All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH v6] PCI: Store PCIe bus address in struct of_pci_range
@ 2015-07-27 15:17 ` Gabriele Paoloni
  0 siblings, 0 replies; 73+ messages in thread
From: Gabriele Paoloni @ 2015-07-27 15:17 UTC (permalink / raw)
  To: gabriele.paoloni, arnd, lorenzo.pieralisi, wangzhou1, bhelgaas,
	robh+dt, james.morse, Liviu.Dudau
  Cc: linux-pci, linux-arm-kernel, devicetree, yuanzhichang, zhudacai,
	zhangjukuo, qiuzhenfa, liguozhu

From: gabriele paoloni <gabriele.paoloni@huawei.com>

    This patch is needed port PCIe designware to new DT parsing API
    As discussed in
    http://lists.infradead.org/pipermail/linux-arm-kernel/2015-January/317743.html
    in designware we have a problem as the PCI addresses in the PCIe controller
    address space are required in order to perform correct HW operation.

    In order to solve this problem commit f4c55c5a3 "PCI: designware:
    Program ATU with untranslated address" added code to read the PCIe
    controller start address directly from the DT ranges.

    In the new DT parsing API of_pci_get_host_bridge_resources() hides the
    DT parser from the host controller drivers, so it is not possible
    for drivers to parse values directly from the DT.

    In http://www.spinics.net/lists/linux-pci/msg42540.html we already tried
    to use the new DT parsing API but there is a bug (obviously) in setting
    the <*>_mod_base addresses
    Applying this patch we can easily set "<*>_mod_base = win->__res.start"

    This patch adds a new field in "struct of_pci_range" to store the
    pci bus start address; it fills the field in of_pci_range_parser_one();
    in of_pci_get_host_bridge_resources() it retrieves the resource entry
    after it is created and added to the resource list and uses
    entry->__res.start to store the pci controller address

    the patch is based on 4.2-rc1

    Signed-off-by: Gabriele Paoloni <gabriele.paoloni@huawei.com>
    Acked-by: Liviu Dudau <Liviu.Dudau@arm.com>
    Acked-by: Rob Herring <robh@kernel.org>
---
 drivers/of/address.c       | 2 ++
 drivers/of/of_pci.c        | 4 ++++
 include/linux/of_address.h | 1 +
 3 files changed, 7 insertions(+)

diff --git a/drivers/of/address.c b/drivers/of/address.c
index 8bfda6a..23a5793 100644
--- a/drivers/of/address.c
+++ b/drivers/of/address.c
@@ -253,6 +253,7 @@ struct of_pci_range *of_pci_range_parser_one(struct of_pci_range_parser *parser,
 						struct of_pci_range *range)
 {
 	const int na = 3, ns = 2;
+	const int p_ns = of_n_size_cells(parser->node);
 
 	if (!range)
 		return NULL;
@@ -265,6 +266,7 @@ struct of_pci_range *of_pci_range_parser_one(struct of_pci_range_parser *parser,
 	range->pci_addr = of_read_number(parser->range + 1, ns);
 	range->cpu_addr = of_translate_address(parser->node,
 				parser->range + na);
+	range->bus_addr = of_read_number(parser->range + na, p_ns);
 	range->size = of_read_number(parser->range + parser->pna + na, ns);
 
 	parser->range += parser->np;
diff --git a/drivers/of/of_pci.c b/drivers/of/of_pci.c
index 5751dc5..fe57030 100644
--- a/drivers/of/of_pci.c
+++ b/drivers/of/of_pci.c
@@ -198,6 +198,7 @@ int of_pci_get_host_bridge_resources(struct device_node *dev,
 
 	pr_debug("Parsing ranges property...\n");
 	for_each_of_pci_range(&parser, &range) {
+		struct resource_entry *entry;
 		/* Read next ranges element */
 		if ((range.flags & IORESOURCE_TYPE_BITS) == IORESOURCE_IO)
 			snprintf(range_type, 4, " IO");
@@ -240,6 +241,9 @@ int of_pci_get_host_bridge_resources(struct device_node *dev,
 		}
 
 		pci_add_resource_offset(resources, res,	res->start - range.pci_addr);
+		entry = list_last_entry(resources, struct resource_entry, node);
+		/* we are using __res for storing the PCI controller address */
+		entry->__res.start = range.bus_addr;
 	}
 
 	return 0;
diff --git a/include/linux/of_address.h b/include/linux/of_address.h
index d88e81b..865f96e 100644
--- a/include/linux/of_address.h
+++ b/include/linux/of_address.h
@@ -16,6 +16,7 @@ struct of_pci_range {
 	u32 pci_space;
 	u64 pci_addr;
 	u64 cpu_addr;
+	u64 bus_addr;
 	u64 size;
 	u32 flags;
 };
-- 
1.9.1

^ permalink raw reply related	[flat|nested] 73+ messages in thread
* [PATCH v6] PCI: Store PCIe bus address in struct of_pci_range
@ 2015-07-27 15:17 ` Gabriele Paoloni
  0 siblings, 0 replies; 73+ messages in thread
From: Gabriele Paoloni @ 2015-07-27 15:17 UTC (permalink / raw)
  To: gabriele.paoloni, arnd, lorenzo.pieralisi, wangzhou1, bhelgaas,
	robh+dt, james.morse, Liviu.Dudau
  Cc: linux-pci, linux-arm-kernel, devicetree, yuanzhichang, zhudacai,
	zhangjukuo, qiuzhenfa, liguozhu

From: gabriele paoloni <gabriele.paoloni@huawei.com>

    This patch is needed port PCIe designware to new DT parsing API
    As discussed in
    http://lists.infradead.org/pipermail/linux-arm-kernel/2015-January/317743.html
    in designware we have a problem as the PCI addresses in the PCIe controller
    address space are required in order to perform correct HW operation.

    In order to solve this problem commit f4c55c5a3 "PCI: designware:
    Program ATU with untranslated address" added code to read the PCIe
    controller start address directly from the DT ranges.

    In the new DT parsing API of_pci_get_host_bridge_resources() hides the
    DT parser from the host controller drivers, so it is not possible
    for drivers to parse values directly from the DT.

    In http://www.spinics.net/lists/linux-pci/msg42540.html we already tried
    to use the new DT parsing API but there is a bug (obviously) in setting
    the <*>_mod_base addresses
    Applying this patch we can easily set "<*>_mod_base = win->__res.start"

    This patch adds a new field in "struct of_pci_range" to store the
    pci bus start address; it fills the field in of_pci_range_parser_one();
    in of_pci_get_host_bridge_resources() it retrieves the resource entry
    after it is created and added to the resource list and uses
    entry->__res.start to store the pci controller address

    the patch is based on 4.2-rc1

    Signed-off-by: Gabriele Paoloni <gabriele.paoloni@huawei.com>
    Acked-by: Liviu Dudau <Liviu.Dudau@arm.com>
    Acked-by: Rob Herring <robh@kernel.org>
---
 drivers/of/address.c       | 2 ++
 drivers/of/of_pci.c        | 4 ++++
 include/linux/of_address.h | 1 +
 3 files changed, 7 insertions(+)

diff --git a/drivers/of/address.c b/drivers/of/address.c
index 8bfda6a..23a5793 100644
--- a/drivers/of/address.c
+++ b/drivers/of/address.c
@@ -253,6 +253,7 @@ struct of_pci_range *of_pci_range_parser_one(struct of_pci_range_parser *parser,
 						struct of_pci_range *range)
 {
 	const int na = 3, ns = 2;
+	const int p_ns = of_n_size_cells(parser->node);
 
 	if (!range)
 		return NULL;
@@ -265,6 +266,7 @@ struct of_pci_range *of_pci_range_parser_one(struct of_pci_range_parser *parser,
 	range->pci_addr = of_read_number(parser->range + 1, ns);
 	range->cpu_addr = of_translate_address(parser->node,
 				parser->range + na);
+	range->bus_addr = of_read_number(parser->range + na, p_ns);
 	range->size = of_read_number(parser->range + parser->pna + na, ns);
 
 	parser->range += parser->np;
diff --git a/drivers/of/of_pci.c b/drivers/of/of_pci.c
index 5751dc5..fe57030 100644
--- a/drivers/of/of_pci.c
+++ b/drivers/of/of_pci.c
@@ -198,6 +198,7 @@ int of_pci_get_host_bridge_resources(struct device_node *dev,
 
 	pr_debug("Parsing ranges property...\n");
 	for_each_of_pci_range(&parser, &range) {
+		struct resource_entry *entry;
 		/* Read next ranges element */
 		if ((range.flags & IORESOURCE_TYPE_BITS) == IORESOURCE_IO)
 			snprintf(range_type, 4, " IO");
@@ -240,6 +241,9 @@ int of_pci_get_host_bridge_resources(struct device_node *dev,
 		}
 
 		pci_add_resource_offset(resources, res,	res->start - range.pci_addr);
+		entry = list_last_entry(resources, struct resource_entry, node);
+		/* we are using __res for storing the PCI controller address */
+		entry->__res.start = range.bus_addr;
 	}
 
 	return 0;
diff --git a/include/linux/of_address.h b/include/linux/of_address.h
index d88e81b..865f96e 100644
--- a/include/linux/of_address.h
+++ b/include/linux/of_address.h
@@ -16,6 +16,7 @@ struct of_pci_range {
 	u32 pci_space;
 	u64 pci_addr;
 	u64 cpu_addr;
+	u64 bus_addr;
 	u64 size;
 	u32 flags;
 };
-- 
1.9.1


^ permalink raw reply related	[flat|nested] 73+ messages in thread
* [PATCH v6] PCI: Store PCIe bus address in struct of_pci_range
@ 2015-07-27 15:17 ` Gabriele Paoloni
  0 siblings, 0 replies; 73+ messages in thread
From: Gabriele Paoloni @ 2015-07-27 15:17 UTC (permalink / raw)
  To: linux-arm-kernel

From: gabriele paoloni <gabriele.paoloni@huawei.com>

    This patch is needed port PCIe designware to new DT parsing API
    As discussed in
    http://lists.infradead.org/pipermail/linux-arm-kernel/2015-January/317743.html
    in designware we have a problem as the PCI addresses in the PCIe controller
    address space are required in order to perform correct HW operation.

    In order to solve this problem commit f4c55c5a3 "PCI: designware:
    Program ATU with untranslated address" added code to read the PCIe
    controller start address directly from the DT ranges.

    In the new DT parsing API of_pci_get_host_bridge_resources() hides the
    DT parser from the host controller drivers, so it is not possible
    for drivers to parse values directly from the DT.

    In http://www.spinics.net/lists/linux-pci/msg42540.html we already tried
    to use the new DT parsing API but there is a bug (obviously) in setting
    the <*>_mod_base addresses
    Applying this patch we can easily set "<*>_mod_base = win->__res.start"

    This patch adds a new field in "struct of_pci_range" to store the
    pci bus start address; it fills the field in of_pci_range_parser_one();
    in of_pci_get_host_bridge_resources() it retrieves the resource entry
    after it is created and added to the resource list and uses
    entry->__res.start to store the pci controller address

    the patch is based on 4.2-rc1

    Signed-off-by: Gabriele Paoloni <gabriele.paoloni@huawei.com>
    Acked-by: Liviu Dudau <Liviu.Dudau@arm.com>
    Acked-by: Rob Herring <robh@kernel.org>
---
 drivers/of/address.c       | 2 ++
 drivers/of/of_pci.c        | 4 ++++
 include/linux/of_address.h | 1 +
 3 files changed, 7 insertions(+)

diff --git a/drivers/of/address.c b/drivers/of/address.c
index 8bfda6a..23a5793 100644
--- a/drivers/of/address.c
+++ b/drivers/of/address.c
@@ -253,6 +253,7 @@ struct of_pci_range *of_pci_range_parser_one(struct of_pci_range_parser *parser,
 						struct of_pci_range *range)
 {
 	const int na = 3, ns = 2;
+	const int p_ns = of_n_size_cells(parser->node);
 
 	if (!range)
 		return NULL;
@@ -265,6 +266,7 @@ struct of_pci_range *of_pci_range_parser_one(struct of_pci_range_parser *parser,
 	range->pci_addr = of_read_number(parser->range + 1, ns);
 	range->cpu_addr = of_translate_address(parser->node,
 				parser->range + na);
+	range->bus_addr = of_read_number(parser->range + na, p_ns);
 	range->size = of_read_number(parser->range + parser->pna + na, ns);
 
 	parser->range += parser->np;
diff --git a/drivers/of/of_pci.c b/drivers/of/of_pci.c
index 5751dc5..fe57030 100644
--- a/drivers/of/of_pci.c
+++ b/drivers/of/of_pci.c
@@ -198,6 +198,7 @@ int of_pci_get_host_bridge_resources(struct device_node *dev,
 
 	pr_debug("Parsing ranges property...\n");
 	for_each_of_pci_range(&parser, &range) {
+		struct resource_entry *entry;
 		/* Read next ranges element */
 		if ((range.flags & IORESOURCE_TYPE_BITS) == IORESOURCE_IO)
 			snprintf(range_type, 4, " IO");
@@ -240,6 +241,9 @@ int of_pci_get_host_bridge_resources(struct device_node *dev,
 		}
 
 		pci_add_resource_offset(resources, res,	res->start - range.pci_addr);
+		entry = list_last_entry(resources, struct resource_entry, node);
+		/* we are using __res for storing the PCI controller address */
+		entry->__res.start = range.bus_addr;
 	}
 
 	return 0;
diff --git a/include/linux/of_address.h b/include/linux/of_address.h
index d88e81b..865f96e 100644
--- a/include/linux/of_address.h
+++ b/include/linux/of_address.h
@@ -16,6 +16,7 @@ struct of_pci_range {
 	u32 pci_space;
 	u64 pci_addr;
 	u64 cpu_addr;
+	u64 bus_addr;
 	u64 size;
 	u32 flags;
 };
-- 
1.9.1

^ permalink raw reply related	[flat|nested] 73+ messages in thread
* RE: [PATCH v6] PCI: Store PCIe bus address in struct of_pci_range
  2015-07-27 15:17 ` Gabriele Paoloni
  (?)
@ 2015-07-29 16:04   ` Gabriele Paoloni
  -1 siblings, 0 replies; 73+ messages in thread
From: Gabriele Paoloni @ 2015-07-29 16:04 UTC (permalink / raw)
  To: Gabriele Paoloni, arnd, lorenzo.pieralisi, Wangzhou (B),
	bhelgaas, robh+dt, james.morse, Liviu.Dudau
  Cc: linux-pci, linux-arm-kernel, devicetree, Yuanzhichang, Zhudacai,
	zhangjukuo, qiuzhenfa, Liguozhu (Kenneth)

Hi Bjorn

Liviu and Rob already acked, do you think it is ok to merge this?

Cheers

Gab

> -----Original Message-----
> From: Gabriele Paoloni
> Sent: Monday, July 27, 2015 4:17 PM
> To: Gabriele Paoloni; arnd@arndb.de; lorenzo.pieralisi@arm.com;
> Wangzhou (B); bhelgaas@google.com; robh+dt@kernel.org;
> james.morse@arm.com; Liviu.Dudau@arm.com
> Cc: linux-pci@vger.kernel.org; linux-arm-kernel@lists.infradead.org;
> devicetree@vger.kernel.org; Yuanzhichang; Zhudacai; zhangjukuo;
> qiuzhenfa; Liguozhu (Kenneth)
> Subject: [PATCH v6] PCI: Store PCIe bus address in struct of_pci_range
> 
> From: gabriele paoloni <gabriele.paoloni@huawei.com>
> 
>     This patch is needed port PCIe designware to new DT parsing API
>     As discussed in
>     http://lists.infradead.org/pipermail/linux-arm-kernel/2015-
> January/317743.html
>     in designware we have a problem as the PCI addresses in the PCIe
> controller
>     address space are required in order to perform correct HW operation.
> 
>     In order to solve this problem commit f4c55c5a3 "PCI: designware:
>     Program ATU with untranslated address" added code to read the PCIe
>     controller start address directly from the DT ranges.
> 
>     In the new DT parsing API of_pci_get_host_bridge_resources() hides
> the
>     DT parser from the host controller drivers, so it is not possible
>     for drivers to parse values directly from the DT.
> 
>     In http://www.spinics.net/lists/linux-pci/msg42540.html we already
> tried
>     to use the new DT parsing API but there is a bug (obviously) in
> setting
>     the <*>_mod_base addresses
>     Applying this patch we can easily set "<*>_mod_base = win-
> >__res.start"
> 
>     This patch adds a new field in "struct of_pci_range" to store the
>     pci bus start address; it fills the field in
> of_pci_range_parser_one();
>     in of_pci_get_host_bridge_resources() it retrieves the resource
> entry
>     after it is created and added to the resource list and uses
>     entry->__res.start to store the pci controller address
> 
>     the patch is based on 4.2-rc1
> 
>     Signed-off-by: Gabriele Paoloni <gabriele.paoloni@huawei.com>
>     Acked-by: Liviu Dudau <Liviu.Dudau@arm.com>
>     Acked-by: Rob Herring <robh@kernel.org>
> ---
>  drivers/of/address.c       | 2 ++
>  drivers/of/of_pci.c        | 4 ++++
>  include/linux/of_address.h | 1 +
>  3 files changed, 7 insertions(+)
> 
> diff --git a/drivers/of/address.c b/drivers/of/address.c
> index 8bfda6a..23a5793 100644
> --- a/drivers/of/address.c
> +++ b/drivers/of/address.c
> @@ -253,6 +253,7 @@ struct of_pci_range *of_pci_range_parser_one(struct
> of_pci_range_parser *parser,
>  						struct of_pci_range *range)
>  {
>  	const int na = 3, ns = 2;
> +	const int p_ns = of_n_size_cells(parser->node);
> 
>  	if (!range)
>  		return NULL;
> @@ -265,6 +266,7 @@ struct of_pci_range *of_pci_range_parser_one(struct
> of_pci_range_parser *parser,
>  	range->pci_addr = of_read_number(parser->range + 1, ns);
>  	range->cpu_addr = of_translate_address(parser->node,
>  				parser->range + na);
> +	range->bus_addr = of_read_number(parser->range + na, p_ns);
>  	range->size = of_read_number(parser->range + parser->pna + na,
> ns);
> 
>  	parser->range += parser->np;
> diff --git a/drivers/of/of_pci.c b/drivers/of/of_pci.c
> index 5751dc5..fe57030 100644
> --- a/drivers/of/of_pci.c
> +++ b/drivers/of/of_pci.c
> @@ -198,6 +198,7 @@ int of_pci_get_host_bridge_resources(struct
> device_node *dev,
> 
>  	pr_debug("Parsing ranges property...\n");
>  	for_each_of_pci_range(&parser, &range) {
> +		struct resource_entry *entry;
>  		/* Read next ranges element */
>  		if ((range.flags & IORESOURCE_TYPE_BITS) == IORESOURCE_IO)
>  			snprintf(range_type, 4, " IO");
> @@ -240,6 +241,9 @@ int of_pci_get_host_bridge_resources(struct
> device_node *dev,
>  		}
> 
>  		pci_add_resource_offset(resources, res,	res->start -
> range.pci_addr);
> +		entry = list_last_entry(resources, struct resource_entry,
> node);
> +		/* we are using __res for storing the PCI controller
> address */
> +		entry->__res.start = range.bus_addr;
>  	}
> 
>  	return 0;
> diff --git a/include/linux/of_address.h b/include/linux/of_address.h
> index d88e81b..865f96e 100644
> --- a/include/linux/of_address.h
> +++ b/include/linux/of_address.h
> @@ -16,6 +16,7 @@ struct of_pci_range {
>  	u32 pci_space;
>  	u64 pci_addr;
>  	u64 cpu_addr;
> +	u64 bus_addr;
>  	u64 size;
>  	u32 flags;
>  };
> --
> 1.9.1


^ permalink raw reply	[flat|nested] 73+ messages in thread
* RE: [PATCH v6] PCI: Store PCIe bus address in struct of_pci_range
@ 2015-07-29 16:04   ` Gabriele Paoloni
  0 siblings, 0 replies; 73+ messages in thread
From: Gabriele Paoloni @ 2015-07-29 16:04 UTC (permalink / raw)
  To: Gabriele Paoloni, arnd, lorenzo.pieralisi, Wangzhou (B),
	bhelgaas, robh+dt, james.morse, Liviu.Dudau
  Cc: linux-pci, linux-arm-kernel, devicetree, Yuanzhichang, Zhudacai,
	zhangjukuo, qiuzhenfa, Liguozhu (Kenneth)

SGkgQmpvcm4NCg0KTGl2aXUgYW5kIFJvYiBhbHJlYWR5IGFja2VkLCBkbyB5b3UgdGhpbmsgaXQg
aXMgb2sgdG8gbWVyZ2UgdGhpcz8NCg0KQ2hlZXJzDQoNCkdhYg0KDQo+IC0tLS0tT3JpZ2luYWwg
TWVzc2FnZS0tLS0tDQo+IEZyb206IEdhYnJpZWxlIFBhb2xvbmkNCj4gU2VudDogTW9uZGF5LCBK
dWx5IDI3LCAyMDE1IDQ6MTcgUE0NCj4gVG86IEdhYnJpZWxlIFBhb2xvbmk7IGFybmRAYXJuZGIu
ZGU7IGxvcmVuem8ucGllcmFsaXNpQGFybS5jb207DQo+IFdhbmd6aG91IChCKTsgYmhlbGdhYXNA
Z29vZ2xlLmNvbTsgcm9iaCtkdEBrZXJuZWwub3JnOw0KPiBqYW1lcy5tb3JzZUBhcm0uY29tOyBM
aXZpdS5EdWRhdUBhcm0uY29tDQo+IENjOiBsaW51eC1wY2lAdmdlci5rZXJuZWwub3JnOyBsaW51
eC1hcm0ta2VybmVsQGxpc3RzLmluZnJhZGVhZC5vcmc7DQo+IGRldmljZXRyZWVAdmdlci5rZXJu
ZWwub3JnOyBZdWFuemhpY2hhbmc7IFpodWRhY2FpOyB6aGFuZ2p1a3VvOw0KPiBxaXV6aGVuZmE7
IExpZ3Vvemh1IChLZW5uZXRoKQ0KPiBTdWJqZWN0OiBbUEFUQ0ggdjZdIFBDSTogU3RvcmUgUENJ
ZSBidXMgYWRkcmVzcyBpbiBzdHJ1Y3Qgb2ZfcGNpX3JhbmdlDQo+IA0KPiBGcm9tOiBnYWJyaWVs
ZSBwYW9sb25pIDxnYWJyaWVsZS5wYW9sb25pQGh1YXdlaS5jb20+DQo+IA0KPiAgICAgVGhpcyBw
YXRjaCBpcyBuZWVkZWQgcG9ydCBQQ0llIGRlc2lnbndhcmUgdG8gbmV3IERUIHBhcnNpbmcgQVBJ
DQo+ICAgICBBcyBkaXNjdXNzZWQgaW4NCj4gICAgIGh0dHA6Ly9saXN0cy5pbmZyYWRlYWQub3Jn
L3BpcGVybWFpbC9saW51eC1hcm0ta2VybmVsLzIwMTUtDQo+IEphbnVhcnkvMzE3NzQzLmh0bWwN
Cj4gICAgIGluIGRlc2lnbndhcmUgd2UgaGF2ZSBhIHByb2JsZW0gYXMgdGhlIFBDSSBhZGRyZXNz
ZXMgaW4gdGhlIFBDSWUNCj4gY29udHJvbGxlcg0KPiAgICAgYWRkcmVzcyBzcGFjZSBhcmUgcmVx
dWlyZWQgaW4gb3JkZXIgdG8gcGVyZm9ybSBjb3JyZWN0IEhXIG9wZXJhdGlvbi4NCj4gDQo+ICAg
ICBJbiBvcmRlciB0byBzb2x2ZSB0aGlzIHByb2JsZW0gY29tbWl0IGY0YzU1YzVhMyAiUENJOiBk
ZXNpZ253YXJlOg0KPiAgICAgUHJvZ3JhbSBBVFUgd2l0aCB1bnRyYW5zbGF0ZWQgYWRkcmVzcyIg
YWRkZWQgY29kZSB0byByZWFkIHRoZSBQQ0llDQo+ICAgICBjb250cm9sbGVyIHN0YXJ0IGFkZHJl
c3MgZGlyZWN0bHkgZnJvbSB0aGUgRFQgcmFuZ2VzLg0KPiANCj4gICAgIEluIHRoZSBuZXcgRFQg
cGFyc2luZyBBUEkgb2ZfcGNpX2dldF9ob3N0X2JyaWRnZV9yZXNvdXJjZXMoKSBoaWRlcw0KPiB0
aGUNCj4gICAgIERUIHBhcnNlciBmcm9tIHRoZSBob3N0IGNvbnRyb2xsZXIgZHJpdmVycywgc28g
aXQgaXMgbm90IHBvc3NpYmxlDQo+ICAgICBmb3IgZHJpdmVycyB0byBwYXJzZSB2YWx1ZXMgZGly
ZWN0bHkgZnJvbSB0aGUgRFQuDQo+IA0KPiAgICAgSW4gaHR0cDovL3d3dy5zcGluaWNzLm5ldC9s
aXN0cy9saW51eC1wY2kvbXNnNDI1NDAuaHRtbCB3ZSBhbHJlYWR5DQo+IHRyaWVkDQo+ICAgICB0
byB1c2UgdGhlIG5ldyBEVCBwYXJzaW5nIEFQSSBidXQgdGhlcmUgaXMgYSBidWcgKG9idmlvdXNs
eSkgaW4NCj4gc2V0dGluZw0KPiAgICAgdGhlIDwqPl9tb2RfYmFzZSBhZGRyZXNzZXMNCj4gICAg
IEFwcGx5aW5nIHRoaXMgcGF0Y2ggd2UgY2FuIGVhc2lseSBzZXQgIjwqPl9tb2RfYmFzZSA9IHdp
bi0NCj4gPl9fcmVzLnN0YXJ0Ig0KPiANCj4gICAgIFRoaXMgcGF0Y2ggYWRkcyBhIG5ldyBmaWVs
ZCBpbiAic3RydWN0IG9mX3BjaV9yYW5nZSIgdG8gc3RvcmUgdGhlDQo+ICAgICBwY2kgYnVzIHN0
YXJ0IGFkZHJlc3M7IGl0IGZpbGxzIHRoZSBmaWVsZCBpbg0KPiBvZl9wY2lfcmFuZ2VfcGFyc2Vy
X29uZSgpOw0KPiAgICAgaW4gb2ZfcGNpX2dldF9ob3N0X2JyaWRnZV9yZXNvdXJjZXMoKSBpdCBy
ZXRyaWV2ZXMgdGhlIHJlc291cmNlDQo+IGVudHJ5DQo+ICAgICBhZnRlciBpdCBpcyBjcmVhdGVk
IGFuZCBhZGRlZCB0byB0aGUgcmVzb3VyY2UgbGlzdCBhbmQgdXNlcw0KPiAgICAgZW50cnktPl9f
cmVzLnN0YXJ0IHRvIHN0b3JlIHRoZSBwY2kgY29udHJvbGxlciBhZGRyZXNzDQo+IA0KPiAgICAg
dGhlIHBhdGNoIGlzIGJhc2VkIG9uIDQuMi1yYzENCj4gDQo+ICAgICBTaWduZWQtb2ZmLWJ5OiBH
YWJyaWVsZSBQYW9sb25pIDxnYWJyaWVsZS5wYW9sb25pQGh1YXdlaS5jb20+DQo+ICAgICBBY2tl
ZC1ieTogTGl2aXUgRHVkYXUgPExpdml1LkR1ZGF1QGFybS5jb20+DQo+ICAgICBBY2tlZC1ieTog
Um9iIEhlcnJpbmcgPHJvYmhAa2VybmVsLm9yZz4NCj4gLS0tDQo+ICBkcml2ZXJzL29mL2FkZHJl
c3MuYyAgICAgICB8IDIgKysNCj4gIGRyaXZlcnMvb2Yvb2ZfcGNpLmMgICAgICAgIHwgNCArKysr
DQo+ICBpbmNsdWRlL2xpbnV4L29mX2FkZHJlc3MuaCB8IDEgKw0KPiAgMyBmaWxlcyBjaGFuZ2Vk
LCA3IGluc2VydGlvbnMoKykNCj4gDQo+IGRpZmYgLS1naXQgYS9kcml2ZXJzL29mL2FkZHJlc3Mu
YyBiL2RyaXZlcnMvb2YvYWRkcmVzcy5jDQo+IGluZGV4IDhiZmRhNmEuLjIzYTU3OTMgMTAwNjQ0
DQo+IC0tLSBhL2RyaXZlcnMvb2YvYWRkcmVzcy5jDQo+ICsrKyBiL2RyaXZlcnMvb2YvYWRkcmVz
cy5jDQo+IEBAIC0yNTMsNiArMjUzLDcgQEAgc3RydWN0IG9mX3BjaV9yYW5nZSAqb2ZfcGNpX3Jh
bmdlX3BhcnNlcl9vbmUoc3RydWN0DQo+IG9mX3BjaV9yYW5nZV9wYXJzZXIgKnBhcnNlciwNCj4g
IAkJCQkJCXN0cnVjdCBvZl9wY2lfcmFuZ2UgKnJhbmdlKQ0KPiAgew0KPiAgCWNvbnN0IGludCBu
YSA9IDMsIG5zID0gMjsNCj4gKwljb25zdCBpbnQgcF9ucyA9IG9mX25fc2l6ZV9jZWxscyhwYXJz
ZXItPm5vZGUpOw0KPiANCj4gIAlpZiAoIXJhbmdlKQ0KPiAgCQlyZXR1cm4gTlVMTDsNCj4gQEAg
LTI2NSw2ICsyNjYsNyBAQCBzdHJ1Y3Qgb2ZfcGNpX3JhbmdlICpvZl9wY2lfcmFuZ2VfcGFyc2Vy
X29uZShzdHJ1Y3QNCj4gb2ZfcGNpX3JhbmdlX3BhcnNlciAqcGFyc2VyLA0KPiAgCXJhbmdlLT5w
Y2lfYWRkciA9IG9mX3JlYWRfbnVtYmVyKHBhcnNlci0+cmFuZ2UgKyAxLCBucyk7DQo+ICAJcmFu
Z2UtPmNwdV9hZGRyID0gb2ZfdHJhbnNsYXRlX2FkZHJlc3MocGFyc2VyLT5ub2RlLA0KPiAgCQkJ
CXBhcnNlci0+cmFuZ2UgKyBuYSk7DQo+ICsJcmFuZ2UtPmJ1c19hZGRyID0gb2ZfcmVhZF9udW1i
ZXIocGFyc2VyLT5yYW5nZSArIG5hLCBwX25zKTsNCj4gIAlyYW5nZS0+c2l6ZSA9IG9mX3JlYWRf
bnVtYmVyKHBhcnNlci0+cmFuZ2UgKyBwYXJzZXItPnBuYSArIG5hLA0KPiBucyk7DQo+IA0KPiAg
CXBhcnNlci0+cmFuZ2UgKz0gcGFyc2VyLT5ucDsNCj4gZGlmZiAtLWdpdCBhL2RyaXZlcnMvb2Yv
b2ZfcGNpLmMgYi9kcml2ZXJzL29mL29mX3BjaS5jDQo+IGluZGV4IDU3NTFkYzUuLmZlNTcwMzAg
MTAwNjQ0DQo+IC0tLSBhL2RyaXZlcnMvb2Yvb2ZfcGNpLmMNCj4gKysrIGIvZHJpdmVycy9vZi9v
Zl9wY2kuYw0KPiBAQCAtMTk4LDYgKzE5OCw3IEBAIGludCBvZl9wY2lfZ2V0X2hvc3RfYnJpZGdl
X3Jlc291cmNlcyhzdHJ1Y3QNCj4gZGV2aWNlX25vZGUgKmRldiwNCj4gDQo+ICAJcHJfZGVidWco
IlBhcnNpbmcgcmFuZ2VzIHByb3BlcnR5Li4uXG4iKTsNCj4gIAlmb3JfZWFjaF9vZl9wY2lfcmFu
Z2UoJnBhcnNlciwgJnJhbmdlKSB7DQo+ICsJCXN0cnVjdCByZXNvdXJjZV9lbnRyeSAqZW50cnk7
DQo+ICAJCS8qIFJlYWQgbmV4dCByYW5nZXMgZWxlbWVudCAqLw0KPiAgCQlpZiAoKHJhbmdlLmZs
YWdzICYgSU9SRVNPVVJDRV9UWVBFX0JJVFMpID09IElPUkVTT1VSQ0VfSU8pDQo+ICAJCQlzbnBy
aW50ZihyYW5nZV90eXBlLCA0LCAiIElPIik7DQo+IEBAIC0yNDAsNiArMjQxLDkgQEAgaW50IG9m
X3BjaV9nZXRfaG9zdF9icmlkZ2VfcmVzb3VyY2VzKHN0cnVjdA0KPiBkZXZpY2Vfbm9kZSAqZGV2
LA0KPiAgCQl9DQo+IA0KPiAgCQlwY2lfYWRkX3Jlc291cmNlX29mZnNldChyZXNvdXJjZXMsIHJl
cywJcmVzLT5zdGFydCAtDQo+IHJhbmdlLnBjaV9hZGRyKTsNCj4gKwkJZW50cnkgPSBsaXN0X2xh
c3RfZW50cnkocmVzb3VyY2VzLCBzdHJ1Y3QgcmVzb3VyY2VfZW50cnksDQo+IG5vZGUpOw0KPiAr
CQkvKiB3ZSBhcmUgdXNpbmcgX19yZXMgZm9yIHN0b3JpbmcgdGhlIFBDSSBjb250cm9sbGVyDQo+
IGFkZHJlc3MgKi8NCj4gKwkJZW50cnktPl9fcmVzLnN0YXJ0ID0gcmFuZ2UuYnVzX2FkZHI7DQo+
ICAJfQ0KPiANCj4gIAlyZXR1cm4gMDsNCj4gZGlmZiAtLWdpdCBhL2luY2x1ZGUvbGludXgvb2Zf
YWRkcmVzcy5oIGIvaW5jbHVkZS9saW51eC9vZl9hZGRyZXNzLmgNCj4gaW5kZXggZDg4ZTgxYi4u
ODY1Zjk2ZSAxMDA2NDQNCj4gLS0tIGEvaW5jbHVkZS9saW51eC9vZl9hZGRyZXNzLmgNCj4gKysr
IGIvaW5jbHVkZS9saW51eC9vZl9hZGRyZXNzLmgNCj4gQEAgLTE2LDYgKzE2LDcgQEAgc3RydWN0
IG9mX3BjaV9yYW5nZSB7DQo+ICAJdTMyIHBjaV9zcGFjZTsNCj4gIAl1NjQgcGNpX2FkZHI7DQo+
ICAJdTY0IGNwdV9hZGRyOw0KPiArCXU2NCBidXNfYWRkcjsNCj4gIAl1NjQgc2l6ZTsNCj4gIAl1
MzIgZmxhZ3M7DQo+ICB9Ow0KPiAtLQ0KPiAxLjkuMQ0KDQo=

^ permalink raw reply	[flat|nested] 73+ messages in thread
* [PATCH v6] PCI: Store PCIe bus address in struct of_pci_range
@ 2015-07-29 16:04   ` Gabriele Paoloni
  0 siblings, 0 replies; 73+ messages in thread
From: Gabriele Paoloni @ 2015-07-29 16:04 UTC (permalink / raw)
  To: linux-arm-kernel

Hi Bjorn

Liviu and Rob already acked, do you think it is ok to merge this?

Cheers

Gab

> -----Original Message-----
> From: Gabriele Paoloni
> Sent: Monday, July 27, 2015 4:17 PM
> To: Gabriele Paoloni; arnd at arndb.de; lorenzo.pieralisi at arm.com;
> Wangzhou (B); bhelgaas at google.com; robh+dt at kernel.org;
> james.morse at arm.com; Liviu.Dudau at arm.com
> Cc: linux-pci at vger.kernel.org; linux-arm-kernel at lists.infradead.org;
> devicetree at vger.kernel.org; Yuanzhichang; Zhudacai; zhangjukuo;
> qiuzhenfa; Liguozhu (Kenneth)
> Subject: [PATCH v6] PCI: Store PCIe bus address in struct of_pci_range
> 
> From: gabriele paoloni <gabriele.paoloni@huawei.com>
> 
>     This patch is needed port PCIe designware to new DT parsing API
>     As discussed in
>     http://lists.infradead.org/pipermail/linux-arm-kernel/2015-
> January/317743.html
>     in designware we have a problem as the PCI addresses in the PCIe
> controller
>     address space are required in order to perform correct HW operation.
> 
>     In order to solve this problem commit f4c55c5a3 "PCI: designware:
>     Program ATU with untranslated address" added code to read the PCIe
>     controller start address directly from the DT ranges.
> 
>     In the new DT parsing API of_pci_get_host_bridge_resources() hides
> the
>     DT parser from the host controller drivers, so it is not possible
>     for drivers to parse values directly from the DT.
> 
>     In http://www.spinics.net/lists/linux-pci/msg42540.html we already
> tried
>     to use the new DT parsing API but there is a bug (obviously) in
> setting
>     the <*>_mod_base addresses
>     Applying this patch we can easily set "<*>_mod_base = win-
> >__res.start"
> 
>     This patch adds a new field in "struct of_pci_range" to store the
>     pci bus start address; it fills the field in
> of_pci_range_parser_one();
>     in of_pci_get_host_bridge_resources() it retrieves the resource
> entry
>     after it is created and added to the resource list and uses
>     entry->__res.start to store the pci controller address
> 
>     the patch is based on 4.2-rc1
> 
>     Signed-off-by: Gabriele Paoloni <gabriele.paoloni@huawei.com>
>     Acked-by: Liviu Dudau <Liviu.Dudau@arm.com>
>     Acked-by: Rob Herring <robh@kernel.org>
> ---
>  drivers/of/address.c       | 2 ++
>  drivers/of/of_pci.c        | 4 ++++
>  include/linux/of_address.h | 1 +
>  3 files changed, 7 insertions(+)
> 
> diff --git a/drivers/of/address.c b/drivers/of/address.c
> index 8bfda6a..23a5793 100644
> --- a/drivers/of/address.c
> +++ b/drivers/of/address.c
> @@ -253,6 +253,7 @@ struct of_pci_range *of_pci_range_parser_one(struct
> of_pci_range_parser *parser,
>  						struct of_pci_range *range)
>  {
>  	const int na = 3, ns = 2;
> +	const int p_ns = of_n_size_cells(parser->node);
> 
>  	if (!range)
>  		return NULL;
> @@ -265,6 +266,7 @@ struct of_pci_range *of_pci_range_parser_one(struct
> of_pci_range_parser *parser,
>  	range->pci_addr = of_read_number(parser->range + 1, ns);
>  	range->cpu_addr = of_translate_address(parser->node,
>  				parser->range + na);
> +	range->bus_addr = of_read_number(parser->range + na, p_ns);
>  	range->size = of_read_number(parser->range + parser->pna + na,
> ns);
> 
>  	parser->range += parser->np;
> diff --git a/drivers/of/of_pci.c b/drivers/of/of_pci.c
> index 5751dc5..fe57030 100644
> --- a/drivers/of/of_pci.c
> +++ b/drivers/of/of_pci.c
> @@ -198,6 +198,7 @@ int of_pci_get_host_bridge_resources(struct
> device_node *dev,
> 
>  	pr_debug("Parsing ranges property...\n");
>  	for_each_of_pci_range(&parser, &range) {
> +		struct resource_entry *entry;
>  		/* Read next ranges element */
>  		if ((range.flags & IORESOURCE_TYPE_BITS) == IORESOURCE_IO)
>  			snprintf(range_type, 4, " IO");
> @@ -240,6 +241,9 @@ int of_pci_get_host_bridge_resources(struct
> device_node *dev,
>  		}
> 
>  		pci_add_resource_offset(resources, res,	res->start -
> range.pci_addr);
> +		entry = list_last_entry(resources, struct resource_entry,
> node);
> +		/* we are using __res for storing the PCI controller
> address */
> +		entry->__res.start = range.bus_addr;
>  	}
> 
>  	return 0;
> diff --git a/include/linux/of_address.h b/include/linux/of_address.h
> index d88e81b..865f96e 100644
> --- a/include/linux/of_address.h
> +++ b/include/linux/of_address.h
> @@ -16,6 +16,7 @@ struct of_pci_range {
>  	u32 pci_space;
>  	u64 pci_addr;
>  	u64 cpu_addr;
> +	u64 bus_addr;
>  	u64 size;
>  	u32 flags;
>  };
> --
> 1.9.1

^ permalink raw reply	[flat|nested] 73+ messages in thread
* Re: [PATCH v6] PCI: Store PCIe bus address in struct of_pci_range
  2015-07-27 15:17 ` Gabriele Paoloni
  (?)
@ 2015-07-29 17:20     ` Bjorn Helgaas
  -1 siblings, 0 replies; 73+ messages in thread
From: Bjorn Helgaas @ 2015-07-29 17:20 UTC (permalink / raw)
  To: Gabriele Paoloni
  Cc: arnd-r2nGTMty4D4, lorenzo.pieralisi-5wv7dgnIgG8,
	wangzhou1-C8/M+/jPZTeaMJb+Lgu22Q, robh+dt-DgEjT+Ai2ygdnm+yROfE0A,
	james.morse-5wv7dgnIgG8, Liviu.Dudau-5wv7dgnIgG8,
	linux-pci-u79uwXL29TY76Z2rM5mHXA,
	linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
	devicetree-u79uwXL29TY76Z2rM5mHXA,
	yuanzhichang-C8/M+/jPZTeaMJb+Lgu22Q,
	zhudacai-C8/M+/jPZTeaMJb+Lgu22Q,
	zhangjukuo-hv44wF8Li93QT0dZR+AlfA,
	qiuzhenfa-C8/M+/jPZTeaMJb+Lgu22Q,
	liguozhu-C8/M+/jPZTeaMJb+Lgu22Q

Hi Gabriele,

As far as I can tell, this is not specific to PCIe, so please use "PCI" in
the subject as a generic term that includes both PCI and PCIe.

On Mon, Jul 27, 2015 at 11:17:03PM +0800, Gabriele Paoloni wrote:
> From: gabriele paoloni <gabriele.paoloni-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
> 
>     This patch is needed port PCIe designware to new DT parsing API
>     As discussed in
>     http://lists.infradead.org/pipermail/linux-arm-kernel/2015-January/317743.html
>     in designware we have a problem as the PCI addresses in the PCIe controller
>     address space are required in order to perform correct HW operation.
> 
>     In order to solve this problem commit f4c55c5a3 "PCI: designware:
>     Program ATU with untranslated address" added code to read the PCIe

Conventional reference is 12-char SHA1, like this:

  f4c55c5a3f7f ("PCI: designware: Program ATU with untranslated address")

>     controller start address directly from the DT ranges.
> 
>     In the new DT parsing API of_pci_get_host_bridge_resources() hides the
>     DT parser from the host controller drivers, so it is not possible
>     for drivers to parse values directly from the DT.
> 
>     In http://www.spinics.net/lists/linux-pci/msg42540.html we already tried
>     to use the new DT parsing API but there is a bug (obviously) in setting
>     the <*>_mod_base addresses
>     Applying this patch we can easily set "<*>_mod_base = win->__res.start"

By itself, this patch adds something.  It would help me understand it if
the *user* of this new something were in the same patch series.

>     This patch adds a new field in "struct of_pci_range" to store the
>     pci bus start address; it fills the field in of_pci_range_parser_one();
>     in of_pci_get_host_bridge_resources() it retrieves the resource entry
>     after it is created and added to the resource list and uses
>     entry->__res.start to store the pci controller address

struct of_pci_range is starting to get confusing to non-OF folks like me.
It now contains:

  u32 pci_space;
  u64 pci_addr;
  u64 cpu_addr;
  u64 bus_addr;

Can you explain what all these things mean, and maybe even add one-line
comments to the structure?

pci_space: The only uses I see are to determine whether to print
"Prefetch".  I don't see any real functionality that uses this.

pci_addr: I assume this is a PCI bus address, like what you would see if
you put an analyzer on the bus/link.  This address could go in a BAR.

cpu_addr: I assume this is a CPU physical address, like what you would see
in /proc/iomem and what you would pass to ioremap().

bus_addr: ?

I'm trying to imagine how this might be expressed in ACPI.  A host bridge
ACPI _CRS contains a CPU physical address and applying a _TRA (translation
offset) to the CPU address gives you a PCI bus address.  I know this code
is OF, not ACPI, but I assume that it should be possible to describe your
hardware via ACPI as well as by OF.

>     the patch is based on 4.2-rc1

You can put this after the "---" line because it's not relevant in the
permanent changelog.

>     Signed-off-by: Gabriele Paoloni <gabriele.paoloni-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
>     Acked-by: Liviu Dudau <Liviu.Dudau-5wv7dgnIgG8@public.gmane.org>
>     Acked-by: Rob Herring <robh-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>

Please un-indent your changelog.

> ---
>  drivers/of/address.c       | 2 ++
>  drivers/of/of_pci.c        | 4 ++++
>  include/linux/of_address.h | 1 +
>  3 files changed, 7 insertions(+)
> 
> diff --git a/drivers/of/address.c b/drivers/of/address.c
> index 8bfda6a..23a5793 100644
> --- a/drivers/of/address.c
> +++ b/drivers/of/address.c
> @@ -253,6 +253,7 @@ struct of_pci_range *of_pci_range_parser_one(struct of_pci_range_parser *parser,
>  						struct of_pci_range *range)
>  {
>  	const int na = 3, ns = 2;
> +	const int p_ns = of_n_size_cells(parser->node);
>  
>  	if (!range)
>  		return NULL;
> @@ -265,6 +266,7 @@ struct of_pci_range *of_pci_range_parser_one(struct of_pci_range_parser *parser,
>  	range->pci_addr = of_read_number(parser->range + 1, ns);
>  	range->cpu_addr = of_translate_address(parser->node,
>  				parser->range + na);
> +	range->bus_addr = of_read_number(parser->range + na, p_ns);
>  	range->size = of_read_number(parser->range + parser->pna + na, ns);
>  
>  	parser->range += parser->np;
> diff --git a/drivers/of/of_pci.c b/drivers/of/of_pci.c
> index 5751dc5..fe57030 100644
> --- a/drivers/of/of_pci.c
> +++ b/drivers/of/of_pci.c
> @@ -198,6 +198,7 @@ int of_pci_get_host_bridge_resources(struct device_node *dev,
>  
>  	pr_debug("Parsing ranges property...\n");
>  	for_each_of_pci_range(&parser, &range) {
> +		struct resource_entry *entry;
>  		/* Read next ranges element */
>  		if ((range.flags & IORESOURCE_TYPE_BITS) == IORESOURCE_IO)
>  			snprintf(range_type, 4, " IO");
> @@ -240,6 +241,9 @@ int of_pci_get_host_bridge_resources(struct device_node *dev,
>  		}
>  
>  		pci_add_resource_offset(resources, res,	res->start - range.pci_addr);
> +		entry = list_last_entry(resources, struct resource_entry, node);
> +		/* we are using __res for storing the PCI controller address */
> +		entry->__res.start = range.bus_addr;
>  	}
>  
>  	return 0;
> diff --git a/include/linux/of_address.h b/include/linux/of_address.h
> index d88e81b..865f96e 100644
> --- a/include/linux/of_address.h
> +++ b/include/linux/of_address.h
> @@ -16,6 +16,7 @@ struct of_pci_range {
>  	u32 pci_space;
>  	u64 pci_addr;
>  	u64 cpu_addr;
> +	u64 bus_addr;
>  	u64 size;
>  	u32 flags;
>  };
> -- 
> 1.9.1
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-pci" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe devicetree" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 73+ messages in thread
* Re: [PATCH v6] PCI: Store PCIe bus address in struct of_pci_range
@ 2015-07-29 17:20     ` Bjorn Helgaas
  0 siblings, 0 replies; 73+ messages in thread
From: Bjorn Helgaas @ 2015-07-29 17:20 UTC (permalink / raw)
  To: Gabriele Paoloni
  Cc: arnd, lorenzo.pieralisi, wangzhou1, robh+dt, james.morse,
	Liviu.Dudau, linux-pci, linux-arm-kernel, devicetree,
	yuanzhichang, zhudacai, zhangjukuo, qiuzhenfa, liguozhu

Hi Gabriele,

As far as I can tell, this is not specific to PCIe, so please use "PCI" in
the subject as a generic term that includes both PCI and PCIe.

On Mon, Jul 27, 2015 at 11:17:03PM +0800, Gabriele Paoloni wrote:
> From: gabriele paoloni <gabriele.paoloni@huawei.com>
> 
>     This patch is needed port PCIe designware to new DT parsing API
>     As discussed in
>     http://lists.infradead.org/pipermail/linux-arm-kernel/2015-January/317743.html
>     in designware we have a problem as the PCI addresses in the PCIe controller
>     address space are required in order to perform correct HW operation.
> 
>     In order to solve this problem commit f4c55c5a3 "PCI: designware:
>     Program ATU with untranslated address" added code to read the PCIe

Conventional reference is 12-char SHA1, like this:

  f4c55c5a3f7f ("PCI: designware: Program ATU with untranslated address")

>     controller start address directly from the DT ranges.
> 
>     In the new DT parsing API of_pci_get_host_bridge_resources() hides the
>     DT parser from the host controller drivers, so it is not possible
>     for drivers to parse values directly from the DT.
> 
>     In http://www.spinics.net/lists/linux-pci/msg42540.html we already tried
>     to use the new DT parsing API but there is a bug (obviously) in setting
>     the <*>_mod_base addresses
>     Applying this patch we can easily set "<*>_mod_base = win->__res.start"

By itself, this patch adds something.  It would help me understand it if
the *user* of this new something were in the same patch series.

>     This patch adds a new field in "struct of_pci_range" to store the
>     pci bus start address; it fills the field in of_pci_range_parser_one();
>     in of_pci_get_host_bridge_resources() it retrieves the resource entry
>     after it is created and added to the resource list and uses
>     entry->__res.start to store the pci controller address

struct of_pci_range is starting to get confusing to non-OF folks like me.
It now contains:

  u32 pci_space;
  u64 pci_addr;
  u64 cpu_addr;
  u64 bus_addr;

Can you explain what all these things mean, and maybe even add one-line
comments to the structure?

pci_space: The only uses I see are to determine whether to print
"Prefetch".  I don't see any real functionality that uses this.

pci_addr: I assume this is a PCI bus address, like what you would see if
you put an analyzer on the bus/link.  This address could go in a BAR.

cpu_addr: I assume this is a CPU physical address, like what you would see
in /proc/iomem and what you would pass to ioremap().

bus_addr: ?

I'm trying to imagine how this might be expressed in ACPI.  A host bridge
ACPI _CRS contains a CPU physical address and applying a _TRA (translation
offset) to the CPU address gives you a PCI bus address.  I know this code
is OF, not ACPI, but I assume that it should be possible to describe your
hardware via ACPI as well as by OF.

>     the patch is based on 4.2-rc1

You can put this after the "---" line because it's not relevant in the
permanent changelog.

>     Signed-off-by: Gabriele Paoloni <gabriele.paoloni@huawei.com>
>     Acked-by: Liviu Dudau <Liviu.Dudau@arm.com>
>     Acked-by: Rob Herring <robh@kernel.org>

Please un-indent your changelog.

> ---
>  drivers/of/address.c       | 2 ++
>  drivers/of/of_pci.c        | 4 ++++
>  include/linux/of_address.h | 1 +
>  3 files changed, 7 insertions(+)
> 
> diff --git a/drivers/of/address.c b/drivers/of/address.c
> index 8bfda6a..23a5793 100644
> --- a/drivers/of/address.c
> +++ b/drivers/of/address.c
> @@ -253,6 +253,7 @@ struct of_pci_range *of_pci_range_parser_one(struct of_pci_range_parser *parser,
>  						struct of_pci_range *range)
>  {
>  	const int na = 3, ns = 2;
> +	const int p_ns = of_n_size_cells(parser->node);
>  
>  	if (!range)
>  		return NULL;
> @@ -265,6 +266,7 @@ struct of_pci_range *of_pci_range_parser_one(struct of_pci_range_parser *parser,
>  	range->pci_addr = of_read_number(parser->range + 1, ns);
>  	range->cpu_addr = of_translate_address(parser->node,
>  				parser->range + na);
> +	range->bus_addr = of_read_number(parser->range + na, p_ns);
>  	range->size = of_read_number(parser->range + parser->pna + na, ns);
>  
>  	parser->range += parser->np;
> diff --git a/drivers/of/of_pci.c b/drivers/of/of_pci.c
> index 5751dc5..fe57030 100644
> --- a/drivers/of/of_pci.c
> +++ b/drivers/of/of_pci.c
> @@ -198,6 +198,7 @@ int of_pci_get_host_bridge_resources(struct device_node *dev,
>  
>  	pr_debug("Parsing ranges property...\n");
>  	for_each_of_pci_range(&parser, &range) {
> +		struct resource_entry *entry;
>  		/* Read next ranges element */
>  		if ((range.flags & IORESOURCE_TYPE_BITS) == IORESOURCE_IO)
>  			snprintf(range_type, 4, " IO");
> @@ -240,6 +241,9 @@ int of_pci_get_host_bridge_resources(struct device_node *dev,
>  		}
>  
>  		pci_add_resource_offset(resources, res,	res->start - range.pci_addr);
> +		entry = list_last_entry(resources, struct resource_entry, node);
> +		/* we are using __res for storing the PCI controller address */
> +		entry->__res.start = range.bus_addr;
>  	}
>  
>  	return 0;
> diff --git a/include/linux/of_address.h b/include/linux/of_address.h
> index d88e81b..865f96e 100644
> --- a/include/linux/of_address.h
> +++ b/include/linux/of_address.h
> @@ -16,6 +16,7 @@ struct of_pci_range {
>  	u32 pci_space;
>  	u64 pci_addr;
>  	u64 cpu_addr;
> +	u64 bus_addr;
>  	u64 size;
>  	u32 flags;
>  };
> -- 
> 1.9.1
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-pci" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 73+ messages in thread
* [PATCH v6] PCI: Store PCIe bus address in struct of_pci_range
@ 2015-07-29 17:20     ` Bjorn Helgaas
  0 siblings, 0 replies; 73+ messages in thread
From: Bjorn Helgaas @ 2015-07-29 17:20 UTC (permalink / raw)
  To: linux-arm-kernel

Hi Gabriele,

As far as I can tell, this is not specific to PCIe, so please use "PCI" in
the subject as a generic term that includes both PCI and PCIe.

On Mon, Jul 27, 2015 at 11:17:03PM +0800, Gabriele Paoloni wrote:
> From: gabriele paoloni <gabriele.paoloni@huawei.com>
> 
>     This patch is needed port PCIe designware to new DT parsing API
>     As discussed in
>     http://lists.infradead.org/pipermail/linux-arm-kernel/2015-January/317743.html
>     in designware we have a problem as the PCI addresses in the PCIe controller
>     address space are required in order to perform correct HW operation.
> 
>     In order to solve this problem commit f4c55c5a3 "PCI: designware:
>     Program ATU with untranslated address" added code to read the PCIe

Conventional reference is 12-char SHA1, like this:

  f4c55c5a3f7f ("PCI: designware: Program ATU with untranslated address")

>     controller start address directly from the DT ranges.
> 
>     In the new DT parsing API of_pci_get_host_bridge_resources() hides the
>     DT parser from the host controller drivers, so it is not possible
>     for drivers to parse values directly from the DT.
> 
>     In http://www.spinics.net/lists/linux-pci/msg42540.html we already tried
>     to use the new DT parsing API but there is a bug (obviously) in setting
>     the <*>_mod_base addresses
>     Applying this patch we can easily set "<*>_mod_base = win->__res.start"

By itself, this patch adds something.  It would help me understand it if
the *user* of this new something were in the same patch series.

>     This patch adds a new field in "struct of_pci_range" to store the
>     pci bus start address; it fills the field in of_pci_range_parser_one();
>     in of_pci_get_host_bridge_resources() it retrieves the resource entry
>     after it is created and added to the resource list and uses
>     entry->__res.start to store the pci controller address

struct of_pci_range is starting to get confusing to non-OF folks like me.
It now contains:

  u32 pci_space;
  u64 pci_addr;
  u64 cpu_addr;
  u64 bus_addr;

Can you explain what all these things mean, and maybe even add one-line
comments to the structure?

pci_space: The only uses I see are to determine whether to print
"Prefetch".  I don't see any real functionality that uses this.

pci_addr: I assume this is a PCI bus address, like what you would see if
you put an analyzer on the bus/link.  This address could go in a BAR.

cpu_addr: I assume this is a CPU physical address, like what you would see
in /proc/iomem and what you would pass to ioremap().

bus_addr: ?

I'm trying to imagine how this might be expressed in ACPI.  A host bridge
ACPI _CRS contains a CPU physical address and applying a _TRA (translation
offset) to the CPU address gives you a PCI bus address.  I know this code
is OF, not ACPI, but I assume that it should be possible to describe your
hardware via ACPI as well as by OF.

>     the patch is based on 4.2-rc1

You can put this after the "---" line because it's not relevant in the
permanent changelog.

>     Signed-off-by: Gabriele Paoloni <gabriele.paoloni@huawei.com>
>     Acked-by: Liviu Dudau <Liviu.Dudau@arm.com>
>     Acked-by: Rob Herring <robh@kernel.org>

Please un-indent your changelog.

> ---
>  drivers/of/address.c       | 2 ++
>  drivers/of/of_pci.c        | 4 ++++
>  include/linux/of_address.h | 1 +
>  3 files changed, 7 insertions(+)
> 
> diff --git a/drivers/of/address.c b/drivers/of/address.c
> index 8bfda6a..23a5793 100644
> --- a/drivers/of/address.c
> +++ b/drivers/of/address.c
> @@ -253,6 +253,7 @@ struct of_pci_range *of_pci_range_parser_one(struct of_pci_range_parser *parser,
>  						struct of_pci_range *range)
>  {
>  	const int na = 3, ns = 2;
> +	const int p_ns = of_n_size_cells(parser->node);
>  
>  	if (!range)
>  		return NULL;
> @@ -265,6 +266,7 @@ struct of_pci_range *of_pci_range_parser_one(struct of_pci_range_parser *parser,
>  	range->pci_addr = of_read_number(parser->range + 1, ns);
>  	range->cpu_addr = of_translate_address(parser->node,
>  				parser->range + na);
> +	range->bus_addr = of_read_number(parser->range + na, p_ns);
>  	range->size = of_read_number(parser->range + parser->pna + na, ns);
>  
>  	parser->range += parser->np;
> diff --git a/drivers/of/of_pci.c b/drivers/of/of_pci.c
> index 5751dc5..fe57030 100644
> --- a/drivers/of/of_pci.c
> +++ b/drivers/of/of_pci.c
> @@ -198,6 +198,7 @@ int of_pci_get_host_bridge_resources(struct device_node *dev,
>  
>  	pr_debug("Parsing ranges property...\n");
>  	for_each_of_pci_range(&parser, &range) {
> +		struct resource_entry *entry;
>  		/* Read next ranges element */
>  		if ((range.flags & IORESOURCE_TYPE_BITS) == IORESOURCE_IO)
>  			snprintf(range_type, 4, " IO");
> @@ -240,6 +241,9 @@ int of_pci_get_host_bridge_resources(struct device_node *dev,
>  		}
>  
>  		pci_add_resource_offset(resources, res,	res->start - range.pci_addr);
> +		entry = list_last_entry(resources, struct resource_entry, node);
> +		/* we are using __res for storing the PCI controller address */
> +		entry->__res.start = range.bus_addr;
>  	}
>  
>  	return 0;
> diff --git a/include/linux/of_address.h b/include/linux/of_address.h
> index d88e81b..865f96e 100644
> --- a/include/linux/of_address.h
> +++ b/include/linux/of_address.h
> @@ -16,6 +16,7 @@ struct of_pci_range {
>  	u32 pci_space;
>  	u64 pci_addr;
>  	u64 cpu_addr;
> +	u64 bus_addr;
>  	u64 size;
>  	u32 flags;
>  };
> -- 
> 1.9.1
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-pci" in
> the body of a message to majordomo at vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 73+ messages in thread
* RE: [PATCH v6] PCI: Store PCIe bus address in struct of_pci_range
  2015-07-29 17:20     ` Bjorn Helgaas
  (?)
@ 2015-07-29 19:44         ` Gabriele Paoloni
  -1 siblings, 0 replies; 73+ messages in thread
From: Gabriele Paoloni @ 2015-07-29 19:44 UTC (permalink / raw)
  To: Bjorn Helgaas
  Cc: arnd-r2nGTMty4D4, lorenzo.pieralisi-5wv7dgnIgG8, Wangzhou (B),
	robh+dt-DgEjT+Ai2ygdnm+yROfE0A, james.morse-5wv7dgnIgG8,
	Liviu.Dudau-5wv7dgnIgG8, linux-pci-u79uwXL29TY76Z2rM5mHXA,
	linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
	devicetree-u79uwXL29TY76Z2rM5mHXA, Yuanzhichang, Zhudacai,
	zhangjukuo, qiuzhenfa, Liguozhu (Kenneth)

Hi Bjorn

Many Thanks for your reply

I have commented back inline with resolutions from my side.

If you're ok with them I'll send it out a new version in the appropriate patchset

Cheers

Gab

> -----Original Message-----
> From: Bjorn Helgaas [mailto:bhelgaas-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org]
> Sent: Wednesday, July 29, 2015 6:21 PM
> To: Gabriele Paoloni
> Cc: arnd-r2nGTMty4D4@public.gmane.org; lorenzo.pieralisi-5wv7dgnIgG8@public.gmane.org; Wangzhou (B);
> robh+dt-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org; james.morse-5wv7dgnIgG8@public.gmane.org; Liviu.Dudau-5wv7dgnIgG8@public.gmane.org; linux-
> pci-u79uwXL29TY76Z2rM5mHXA@public.gmane.org; linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r@public.gmane.org;
> devicetree-u79uwXL29TY76Z2rM5mHXA@public.gmane.org; Yuanzhichang; Zhudacai; zhangjukuo;
> qiuzhenfa; Liguozhu (Kenneth)
> Subject: Re: [PATCH v6] PCI: Store PCIe bus address in struct
> of_pci_range
> 
> Hi Gabriele,
> 
> As far as I can tell, this is not specific to PCIe, so please use "PCI"
> in
> the subject as a generic term that includes both PCI and PCIe.

sure agreed

> 
> On Mon, Jul 27, 2015 at 11:17:03PM +0800, Gabriele Paoloni wrote:
> > From: gabriele paoloni <gabriele.paoloni-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
> >
> >     This patch is needed port PCIe designware to new DT parsing API
> >     As discussed in
> >     http://lists.infradead.org/pipermail/linux-arm-kernel/2015-
> January/317743.html
> >     in designware we have a problem as the PCI addresses in the PCIe
> controller
> >     address space are required in order to perform correct HW
> operation.
> >
> >     In order to solve this problem commit f4c55c5a3 "PCI: designware:
> >     Program ATU with untranslated address" added code to read the
> PCIe
> 
> Conventional reference is 12-char SHA1, like this:
> 
>   f4c55c5a3f7f ("PCI: designware: Program ATU with untranslated
> address")

Agreed, will change this

> 
> >     controller start address directly from the DT ranges.
> >
> >     In the new DT parsing API of_pci_get_host_bridge_resources()
> hides the
> >     DT parser from the host controller drivers, so it is not possible
> >     for drivers to parse values directly from the DT.
> >
> >     In http://www.spinics.net/lists/linux-pci/msg42540.html we
> already tried
> >     to use the new DT parsing API but there is a bug (obviously) in
> setting
> >     the <*>_mod_base addresses
> >     Applying this patch we can easily set "<*>_mod_base = win-
> >__res.start"
> 
> By itself, this patch adds something.  It would help me understand it
> if
> the *user* of this new something were in the same patch series.

the user is: "[PATCH v5 2/5] PCI: designware: Add ARM64 support"
I will ask Zhou Wang to include this patch in his patchset


> 
> >     This patch adds a new field in "struct of_pci_range" to store the
> >     pci bus start address; it fills the field in
> of_pci_range_parser_one();
> >     in of_pci_get_host_bridge_resources() it retrieves the resource
> entry
> >     after it is created and added to the resource list and uses
> >     entry->__res.start to store the pci controller address
> 
> struct of_pci_range is starting to get confusing to non-OF folks like
> me.
> It now contains:
> 
>   u32 pci_space;
>   u64 pci_addr;
>   u64 cpu_addr;
>   u64 bus_addr;
> 
> Can you explain what all these things mean, and maybe even add one-line
> comments to the structure?

sure I can add comments inline in the code

> 
> pci_space: The only uses I see are to determine whether to print
> "Prefetch".  I don't see any real functionality that uses this.

Looking at the code I agree. it's seems to be used only in powerpc 
and microblaze to print out.
However from my understanding pci_space is the phys.hi field of the 
ranges property: it defines the properties of the address space associated
to the PCI address. if you're curious you can find a nice and quick to read
"guide" in http://devicetree.org/MPC5200:PCI

> 
> pci_addr: I assume this is a PCI bus address, like what you would see
> if
> you put an analyzer on the bus/link.  This address could go in a BAR.

Yes, this is the PCI start address of the range: phys.mid + phys.low in the
guide mentioned above

> 
> cpu_addr: I assume this is a CPU physical address, like what you would
> see
> in /proc/iomem and what you would pass to ioremap().

Yes correct

> 
> bus_addr: ?
> 

According to the guide above, this is the address into which the pci_address 
get translated to and that is passed to the root complex. Between the root 
complex and the CPU there can be intermediate translation layers: see that to 
get pci_address we call "of_translate_address"; this will apply all the 
translation layers (ranges in the DT) that it finds till it comes to the root 
node of the DT (thus retrieving the CPU address).
Now said that, for designware we need the first translated PCI address, that we call
here bus_addr after Rob Herring suggested the name...honestly I cannot think of a 
different name

   

> I'm trying to imagine how this might be expressed in ACPI.  A host
> bridge
> ACPI _CRS contains a CPU physical address and applying a _TRA
> (translation
> offset) to the CPU address gives you a PCI bus address.  I know this
> code
> is OF, not ACPI, but I assume that it should be possible to describe
> your
> hardware via ACPI as well as by OF.
> 
> >     the patch is based on 4.2-rc1
> 
> You can put this after the "---" line because it's not relevant in the
> permanent changelog.

Agreed

> 
> >     Signed-off-by: Gabriele Paoloni <gabriele.paoloni-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
> >     Acked-by: Liviu Dudau <Liviu.Dudau-5wv7dgnIgG8@public.gmane.org>
> >     Acked-by: Rob Herring <robh-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>
> 
> Please un-indent your changelog.

Ok agreed

> 
> > ---
> >  drivers/of/address.c       | 2 ++
> >  drivers/of/of_pci.c        | 4 ++++
> >  include/linux/of_address.h | 1 +
> >  3 files changed, 7 insertions(+)
> >
> > diff --git a/drivers/of/address.c b/drivers/of/address.c
> > index 8bfda6a..23a5793 100644
> > --- a/drivers/of/address.c
> > +++ b/drivers/of/address.c
> > @@ -253,6 +253,7 @@ struct of_pci_range
> *of_pci_range_parser_one(struct of_pci_range_parser *parser,
> >  						struct of_pci_range *range)
> >  {
> >  	const int na = 3, ns = 2;
> > +	const int p_ns = of_n_size_cells(parser->node);
> >
> >  	if (!range)
> >  		return NULL;
> > @@ -265,6 +266,7 @@ struct of_pci_range
> *of_pci_range_parser_one(struct of_pci_range_parser *parser,
> >  	range->pci_addr = of_read_number(parser->range + 1, ns);
> >  	range->cpu_addr = of_translate_address(parser->node,
> >  				parser->range + na);
> > +	range->bus_addr = of_read_number(parser->range + na, p_ns);
> >  	range->size = of_read_number(parser->range + parser->pna + na,
> ns);
> >
> >  	parser->range += parser->np;
> > diff --git a/drivers/of/of_pci.c b/drivers/of/of_pci.c
> > index 5751dc5..fe57030 100644
> > --- a/drivers/of/of_pci.c
> > +++ b/drivers/of/of_pci.c
> > @@ -198,6 +198,7 @@ int of_pci_get_host_bridge_resources(struct
> device_node *dev,
> >
> >  	pr_debug("Parsing ranges property...\n");
> >  	for_each_of_pci_range(&parser, &range) {
> > +		struct resource_entry *entry;
> >  		/* Read next ranges element */
> >  		if ((range.flags & IORESOURCE_TYPE_BITS) == IORESOURCE_IO)
> >  			snprintf(range_type, 4, " IO");
> > @@ -240,6 +241,9 @@ int of_pci_get_host_bridge_resources(struct
> device_node *dev,
> >  		}
> >
> >  		pci_add_resource_offset(resources, res,	res->start -
> range.pci_addr);
> > +		entry = list_last_entry(resources, struct resource_entry,
> node);
> > +		/* we are using __res for storing the PCI controller
> address */
> > +		entry->__res.start = range.bus_addr;
> >  	}
> >
> >  	return 0;
> > diff --git a/include/linux/of_address.h b/include/linux/of_address.h
> > index d88e81b..865f96e 100644
> > --- a/include/linux/of_address.h
> > +++ b/include/linux/of_address.h
> > @@ -16,6 +16,7 @@ struct of_pci_range {
> >  	u32 pci_space;
> >  	u64 pci_addr;
> >  	u64 cpu_addr;
> > +	u64 bus_addr;
> >  	u64 size;
> >  	u32 flags;
> >  };
> > --
> > 1.9.1
> >
> > --
> > To unsubscribe from this list: send the line "unsubscribe linux-pci"
> in
> > the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe devicetree" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 73+ messages in thread
* RE: [PATCH v6] PCI: Store PCIe bus address in struct of_pci_range
@ 2015-07-29 19:44         ` Gabriele Paoloni
  0 siblings, 0 replies; 73+ messages in thread
From: Gabriele Paoloni @ 2015-07-29 19:44 UTC (permalink / raw)
  To: Bjorn Helgaas
  Cc: arnd, lorenzo.pieralisi, Wangzhou (B),
	robh+dt, james.morse, Liviu.Dudau, linux-pci, linux-arm-kernel,
	devicetree, Yuanzhichang, Zhudacai, zhangjukuo, qiuzhenfa,
	Liguozhu (Kenneth)

Hi Bjorn

Many Thanks for your reply

I have commented back inline with resolutions from my side.

If you're ok with them I'll send it out a new version in the appropriate patchset

Cheers

Gab

> -----Original Message-----
> From: Bjorn Helgaas [mailto:bhelgaas@google.com]
> Sent: Wednesday, July 29, 2015 6:21 PM
> To: Gabriele Paoloni
> Cc: arnd@arndb.de; lorenzo.pieralisi@arm.com; Wangzhou (B);
> robh+dt@kernel.org; james.morse@arm.com; Liviu.Dudau@arm.com; linux-
> pci@vger.kernel.org; linux-arm-kernel@lists.infradead.org;
> devicetree@vger.kernel.org; Yuanzhichang; Zhudacai; zhangjukuo;
> qiuzhenfa; Liguozhu (Kenneth)
> Subject: Re: [PATCH v6] PCI: Store PCIe bus address in struct
> of_pci_range
> 
> Hi Gabriele,
> 
> As far as I can tell, this is not specific to PCIe, so please use "PCI"
> in
> the subject as a generic term that includes both PCI and PCIe.

sure agreed

> 
> On Mon, Jul 27, 2015 at 11:17:03PM +0800, Gabriele Paoloni wrote:
> > From: gabriele paoloni <gabriele.paoloni@huawei.com>
> >
> >     This patch is needed port PCIe designware to new DT parsing API
> >     As discussed in
> >     http://lists.infradead.org/pipermail/linux-arm-kernel/2015-
> January/317743.html
> >     in designware we have a problem as the PCI addresses in the PCIe
> controller
> >     address space are required in order to perform correct HW
> operation.
> >
> >     In order to solve this problem commit f4c55c5a3 "PCI: designware:
> >     Program ATU with untranslated address" added code to read the
> PCIe
> 
> Conventional reference is 12-char SHA1, like this:
> 
>   f4c55c5a3f7f ("PCI: designware: Program ATU with untranslated
> address")

Agreed, will change this

> 
> >     controller start address directly from the DT ranges.
> >
> >     In the new DT parsing API of_pci_get_host_bridge_resources()
> hides the
> >     DT parser from the host controller drivers, so it is not possible
> >     for drivers to parse values directly from the DT.
> >
> >     In http://www.spinics.net/lists/linux-pci/msg42540.html we
> already tried
> >     to use the new DT parsing API but there is a bug (obviously) in
> setting
> >     the <*>_mod_base addresses
> >     Applying this patch we can easily set "<*>_mod_base = win-
> >__res.start"
> 
> By itself, this patch adds something.  It would help me understand it
> if
> the *user* of this new something were in the same patch series.

the user is: "[PATCH v5 2/5] PCI: designware: Add ARM64 support"
I will ask Zhou Wang to include this patch in his patchset


> 
> >     This patch adds a new field in "struct of_pci_range" to store the
> >     pci bus start address; it fills the field in
> of_pci_range_parser_one();
> >     in of_pci_get_host_bridge_resources() it retrieves the resource
> entry
> >     after it is created and added to the resource list and uses
> >     entry->__res.start to store the pci controller address
> 
> struct of_pci_range is starting to get confusing to non-OF folks like
> me.
> It now contains:
> 
>   u32 pci_space;
>   u64 pci_addr;
>   u64 cpu_addr;
>   u64 bus_addr;
> 
> Can you explain what all these things mean, and maybe even add one-line
> comments to the structure?

sure I can add comments inline in the code

> 
> pci_space: The only uses I see are to determine whether to print
> "Prefetch".  I don't see any real functionality that uses this.

Looking at the code I agree. it's seems to be used only in powerpc 
and microblaze to print out.
However from my understanding pci_space is the phys.hi field of the 
ranges property: it defines the properties of the address space associated
to the PCI address. if you're curious you can find a nice and quick to read
"guide" in http://devicetree.org/MPC5200:PCI

> 
> pci_addr: I assume this is a PCI bus address, like what you would see
> if
> you put an analyzer on the bus/link.  This address could go in a BAR.

Yes, this is the PCI start address of the range: phys.mid + phys.low in the
guide mentioned above

> 
> cpu_addr: I assume this is a CPU physical address, like what you would
> see
> in /proc/iomem and what you would pass to ioremap().

Yes correct

> 
> bus_addr: ?
> 

According to the guide above, this is the address into which the pci_address 
get translated to and that is passed to the root complex. Between the root 
complex and the CPU there can be intermediate translation layers: see that to 
get pci_address we call "of_translate_address"; this will apply all the 
translation layers (ranges in the DT) that it finds till it comes to the root 
node of the DT (thus retrieving the CPU address).
Now said that, for designware we need the first translated PCI address, that we call
here bus_addr after Rob Herring suggested the name...honestly I cannot think of a 
different name

   

> I'm trying to imagine how this might be expressed in ACPI.  A host
> bridge
> ACPI _CRS contains a CPU physical address and applying a _TRA
> (translation
> offset) to the CPU address gives you a PCI bus address.  I know this
> code
> is OF, not ACPI, but I assume that it should be possible to describe
> your
> hardware via ACPI as well as by OF.
> 
> >     the patch is based on 4.2-rc1
> 
> You can put this after the "---" line because it's not relevant in the
> permanent changelog.

Agreed

> 
> >     Signed-off-by: Gabriele Paoloni <gabriele.paoloni@huawei.com>
> >     Acked-by: Liviu Dudau <Liviu.Dudau@arm.com>
> >     Acked-by: Rob Herring <robh@kernel.org>
> 
> Please un-indent your changelog.

Ok agreed

> 
> > ---
> >  drivers/of/address.c       | 2 ++
> >  drivers/of/of_pci.c        | 4 ++++
> >  include/linux/of_address.h | 1 +
> >  3 files changed, 7 insertions(+)
> >
> > diff --git a/drivers/of/address.c b/drivers/of/address.c
> > index 8bfda6a..23a5793 100644
> > --- a/drivers/of/address.c
> > +++ b/drivers/of/address.c
> > @@ -253,6 +253,7 @@ struct of_pci_range
> *of_pci_range_parser_one(struct of_pci_range_parser *parser,
> >  						struct of_pci_range *range)
> >  {
> >  	const int na = 3, ns = 2;
> > +	const int p_ns = of_n_size_cells(parser->node);
> >
> >  	if (!range)
> >  		return NULL;
> > @@ -265,6 +266,7 @@ struct of_pci_range
> *of_pci_range_parser_one(struct of_pci_range_parser *parser,
> >  	range->pci_addr = of_read_number(parser->range + 1, ns);
> >  	range->cpu_addr = of_translate_address(parser->node,
> >  				parser->range + na);
> > +	range->bus_addr = of_read_number(parser->range + na, p_ns);
> >  	range->size = of_read_number(parser->range + parser->pna + na,
> ns);
> >
> >  	parser->range += parser->np;
> > diff --git a/drivers/of/of_pci.c b/drivers/of/of_pci.c
> > index 5751dc5..fe57030 100644
> > --- a/drivers/of/of_pci.c
> > +++ b/drivers/of/of_pci.c
> > @@ -198,6 +198,7 @@ int of_pci_get_host_bridge_resources(struct
> device_node *dev,
> >
> >  	pr_debug("Parsing ranges property...\n");
> >  	for_each_of_pci_range(&parser, &range) {
> > +		struct resource_entry *entry;
> >  		/* Read next ranges element */
> >  		if ((range.flags & IORESOURCE_TYPE_BITS) == IORESOURCE_IO)
> >  			snprintf(range_type, 4, " IO");
> > @@ -240,6 +241,9 @@ int of_pci_get_host_bridge_resources(struct
> device_node *dev,
> >  		}
> >
> >  		pci_add_resource_offset(resources, res,	res->start -
> range.pci_addr);
> > +		entry = list_last_entry(resources, struct resource_entry,
> node);
> > +		/* we are using __res for storing the PCI controller
> address */
> > +		entry->__res.start = range.bus_addr;
> >  	}
> >
> >  	return 0;
> > diff --git a/include/linux/of_address.h b/include/linux/of_address.h
> > index d88e81b..865f96e 100644
> > --- a/include/linux/of_address.h
> > +++ b/include/linux/of_address.h
> > @@ -16,6 +16,7 @@ struct of_pci_range {
> >  	u32 pci_space;
> >  	u64 pci_addr;
> >  	u64 cpu_addr;
> > +	u64 bus_addr;
> >  	u64 size;
> >  	u32 flags;
> >  };
> > --
> > 1.9.1
> >
> > --
> > To unsubscribe from this list: send the line "unsubscribe linux-pci"
> in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 73+ messages in thread
* [PATCH v6] PCI: Store PCIe bus address in struct of_pci_range
@ 2015-07-29 19:44         ` Gabriele Paoloni
  0 siblings, 0 replies; 73+ messages in thread
From: Gabriele Paoloni @ 2015-07-29 19:44 UTC (permalink / raw)
  To: linux-arm-kernel

Hi Bjorn

Many Thanks for your reply

I have commented back inline with resolutions from my side.

If you're ok with them I'll send it out a new version in the appropriate patchset

Cheers

Gab

> -----Original Message-----
> From: Bjorn Helgaas [mailto:bhelgaas at google.com]
> Sent: Wednesday, July 29, 2015 6:21 PM
> To: Gabriele Paoloni
> Cc: arnd at arndb.de; lorenzo.pieralisi at arm.com; Wangzhou (B);
> robh+dt at kernel.org; james.morse at arm.com; Liviu.Dudau at arm.com; linux-
> pci at vger.kernel.org; linux-arm-kernel at lists.infradead.org;
> devicetree at vger.kernel.org; Yuanzhichang; Zhudacai; zhangjukuo;
> qiuzhenfa; Liguozhu (Kenneth)
> Subject: Re: [PATCH v6] PCI: Store PCIe bus address in struct
> of_pci_range
> 
> Hi Gabriele,
> 
> As far as I can tell, this is not specific to PCIe, so please use "PCI"
> in
> the subject as a generic term that includes both PCI and PCIe.

sure agreed

> 
> On Mon, Jul 27, 2015 at 11:17:03PM +0800, Gabriele Paoloni wrote:
> > From: gabriele paoloni <gabriele.paoloni@huawei.com>
> >
> >     This patch is needed port PCIe designware to new DT parsing API
> >     As discussed in
> >     http://lists.infradead.org/pipermail/linux-arm-kernel/2015-
> January/317743.html
> >     in designware we have a problem as the PCI addresses in the PCIe
> controller
> >     address space are required in order to perform correct HW
> operation.
> >
> >     In order to solve this problem commit f4c55c5a3 "PCI: designware:
> >     Program ATU with untranslated address" added code to read the
> PCIe
> 
> Conventional reference is 12-char SHA1, like this:
> 
>   f4c55c5a3f7f ("PCI: designware: Program ATU with untranslated
> address")

Agreed, will change this

> 
> >     controller start address directly from the DT ranges.
> >
> >     In the new DT parsing API of_pci_get_host_bridge_resources()
> hides the
> >     DT parser from the host controller drivers, so it is not possible
> >     for drivers to parse values directly from the DT.
> >
> >     In http://www.spinics.net/lists/linux-pci/msg42540.html we
> already tried
> >     to use the new DT parsing API but there is a bug (obviously) in
> setting
> >     the <*>_mod_base addresses
> >     Applying this patch we can easily set "<*>_mod_base = win-
> >__res.start"
> 
> By itself, this patch adds something.  It would help me understand it
> if
> the *user* of this new something were in the same patch series.

the user is: "[PATCH v5 2/5] PCI: designware: Add ARM64 support"
I will ask Zhou Wang to include this patch in his patchset


> 
> >     This patch adds a new field in "struct of_pci_range" to store the
> >     pci bus start address; it fills the field in
> of_pci_range_parser_one();
> >     in of_pci_get_host_bridge_resources() it retrieves the resource
> entry
> >     after it is created and added to the resource list and uses
> >     entry->__res.start to store the pci controller address
> 
> struct of_pci_range is starting to get confusing to non-OF folks like
> me.
> It now contains:
> 
>   u32 pci_space;
>   u64 pci_addr;
>   u64 cpu_addr;
>   u64 bus_addr;
> 
> Can you explain what all these things mean, and maybe even add one-line
> comments to the structure?

sure I can add comments inline in the code

> 
> pci_space: The only uses I see are to determine whether to print
> "Prefetch".  I don't see any real functionality that uses this.

Looking at the code I agree. it's seems to be used only in powerpc 
and microblaze to print out.
However from my understanding pci_space is the phys.hi field of the 
ranges property: it defines the properties of the address space associated
to the PCI address. if you're curious you can find a nice and quick to read
"guide" in http://devicetree.org/MPC5200:PCI

> 
> pci_addr: I assume this is a PCI bus address, like what you would see
> if
> you put an analyzer on the bus/link.  This address could go in a BAR.

Yes, this is the PCI start address of the range: phys.mid + phys.low in the
guide mentioned above

> 
> cpu_addr: I assume this is a CPU physical address, like what you would
> see
> in /proc/iomem and what you would pass to ioremap().

Yes correct

> 
> bus_addr: ?
> 

According to the guide above, this is the address into which the pci_address 
get translated to and that is passed to the root complex. Between the root 
complex and the CPU there can be intermediate translation layers: see that to 
get pci_address we call "of_translate_address"; this will apply all the 
translation layers (ranges in the DT) that it finds till it comes to the root 
node of the DT (thus retrieving the CPU address).
Now said that, for designware we need the first translated PCI address, that we call
here bus_addr after Rob Herring suggested the name...honestly I cannot think of a 
different name

   

> I'm trying to imagine how this might be expressed in ACPI.  A host
> bridge
> ACPI _CRS contains a CPU physical address and applying a _TRA
> (translation
> offset) to the CPU address gives you a PCI bus address.  I know this
> code
> is OF, not ACPI, but I assume that it should be possible to describe
> your
> hardware via ACPI as well as by OF.
> 
> >     the patch is based on 4.2-rc1
> 
> You can put this after the "---" line because it's not relevant in the
> permanent changelog.

Agreed

> 
> >     Signed-off-by: Gabriele Paoloni <gabriele.paoloni@huawei.com>
> >     Acked-by: Liviu Dudau <Liviu.Dudau@arm.com>
> >     Acked-by: Rob Herring <robh@kernel.org>
> 
> Please un-indent your changelog.

Ok agreed

> 
> > ---
> >  drivers/of/address.c       | 2 ++
> >  drivers/of/of_pci.c        | 4 ++++
> >  include/linux/of_address.h | 1 +
> >  3 files changed, 7 insertions(+)
> >
> > diff --git a/drivers/of/address.c b/drivers/of/address.c
> > index 8bfda6a..23a5793 100644
> > --- a/drivers/of/address.c
> > +++ b/drivers/of/address.c
> > @@ -253,6 +253,7 @@ struct of_pci_range
> *of_pci_range_parser_one(struct of_pci_range_parser *parser,
> >  						struct of_pci_range *range)
> >  {
> >  	const int na = 3, ns = 2;
> > +	const int p_ns = of_n_size_cells(parser->node);
> >
> >  	if (!range)
> >  		return NULL;
> > @@ -265,6 +266,7 @@ struct of_pci_range
> *of_pci_range_parser_one(struct of_pci_range_parser *parser,
> >  	range->pci_addr = of_read_number(parser->range + 1, ns);
> >  	range->cpu_addr = of_translate_address(parser->node,
> >  				parser->range + na);
> > +	range->bus_addr = of_read_number(parser->range + na, p_ns);
> >  	range->size = of_read_number(parser->range + parser->pna + na,
> ns);
> >
> >  	parser->range += parser->np;
> > diff --git a/drivers/of/of_pci.c b/drivers/of/of_pci.c
> > index 5751dc5..fe57030 100644
> > --- a/drivers/of/of_pci.c
> > +++ b/drivers/of/of_pci.c
> > @@ -198,6 +198,7 @@ int of_pci_get_host_bridge_resources(struct
> device_node *dev,
> >
> >  	pr_debug("Parsing ranges property...\n");
> >  	for_each_of_pci_range(&parser, &range) {
> > +		struct resource_entry *entry;
> >  		/* Read next ranges element */
> >  		if ((range.flags & IORESOURCE_TYPE_BITS) == IORESOURCE_IO)
> >  			snprintf(range_type, 4, " IO");
> > @@ -240,6 +241,9 @@ int of_pci_get_host_bridge_resources(struct
> device_node *dev,
> >  		}
> >
> >  		pci_add_resource_offset(resources, res,	res->start -
> range.pci_addr);
> > +		entry = list_last_entry(resources, struct resource_entry,
> node);
> > +		/* we are using __res for storing the PCI controller
> address */
> > +		entry->__res.start = range.bus_addr;
> >  	}
> >
> >  	return 0;
> > diff --git a/include/linux/of_address.h b/include/linux/of_address.h
> > index d88e81b..865f96e 100644
> > --- a/include/linux/of_address.h
> > +++ b/include/linux/of_address.h
> > @@ -16,6 +16,7 @@ struct of_pci_range {
> >  	u32 pci_space;
> >  	u64 pci_addr;
> >  	u64 cpu_addr;
> > +	u64 bus_addr;
> >  	u64 size;
> >  	u32 flags;
> >  };
> > --
> > 1.9.1
> >
> > --
> > To unsubscribe from this list: send the line "unsubscribe linux-pci"
> in
> > the body of a message to majordomo at vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 73+ messages in thread
* Re: [PATCH v6] PCI: Store PCIe bus address in struct of_pci_range
  2015-07-29 19:44         ` Gabriele Paoloni
@ 2015-07-29 21:47           ` Bjorn Helgaas
  -1 siblings, 0 replies; 73+ messages in thread
From: Bjorn Helgaas @ 2015-07-29 21:47 UTC (permalink / raw)
  To: Gabriele Paoloni
  Cc: arnd, lorenzo.pieralisi, Wangzhou (B),
	robh+dt, james.morse, Liviu.Dudau, linux-pci, linux-arm-kernel,
	devicetree, Yuanzhichang, Zhudacai, zhangjukuo, qiuzhenfa,
	Liguozhu (Kenneth),
	Andrew Murray

[+cc Andrew]

On Wed, Jul 29, 2015 at 07:44:18PM +0000, Gabriele Paoloni wrote:
> > -----Original Message-----
> > From: Bjorn Helgaas [mailto:bhelgaas@google.com]
> > Sent: Wednesday, July 29, 2015 6:21 PM
> > To: Gabriele Paoloni

> > On Mon, Jul 27, 2015 at 11:17:03PM +0800, Gabriele Paoloni wrote:
> > > From: gabriele paoloni <gabriele.paoloni@huawei.com>

> > >     This patch adds a new field in "struct of_pci_range" to store the
> > >     pci bus start address; it fills the field in
> > of_pci_range_parser_one();
> > >     in of_pci_get_host_bridge_resources() it retrieves the resource
> > entry
> > >     after it is created and added to the resource list and uses
> > >     entry->__res.start to store the pci controller address
> > 
> > struct of_pci_range is starting to get confusing to non-OF folks like
> > me.
> > It now contains:
> > 
> >   u32 pci_space;
> >   u64 pci_addr;
> >   u64 cpu_addr;
> >   u64 bus_addr;
> > 
> > Can you explain what all these things mean, and maybe even add one-line
> > comments to the structure?

> > pci_space: The only uses I see are to determine whether to print
> > "Prefetch".  I don't see any real functionality that uses this.
> 
> Looking at the code I agree. it's seems to be used only in powerpc 
> and microblaze to print out.
> However from my understanding pci_space is the phys.hi field of the 
> ranges property: it defines the properties of the address space associated
> to the PCI address. if you're curious you can find a nice and quick to read
> "guide" in http://devicetree.org/MPC5200:PCI

I think pci_space should be removed and the users should test
"range.flags & IORESOURCE_PREFETCH" instead.  That's already set by
of_bus_pci_get_flags().  This is separate from your current patch, of
course.

29b635c00f3e ("of/pci: Provide support for parsing PCI DT ranges
property") added struct of_pci_range, and even at the time,
of_bus_pci_get_flags() set IORESOURCE_PREFETCH in of_pci_range.flags.

654837e8fe8d ("powerpc/pci: Use of_pci_range_parser helper in
pci_process_bridge_OF_ranges") converted powerpc to use
of_pci_range_parser() instead of parsing manually.  It converted other
references to look at struct of_pci_range.flags; I'm not sure why it
didn't do that for the prefetch bit.

I copied Andrew in case there's some subtlety here.

> > pci_addr: I assume this is a PCI bus address, like what you would see
> > if
> > you put an analyzer on the bus/link.  This address could go in a BAR.
> 
> Yes, this is the PCI start address of the range: phys.mid + phys.low in the
> guide mentioned above
> 
> > cpu_addr: I assume this is a CPU physical address, like what you would
> > see
> > in /proc/iomem and what you would pass to ioremap().
> 
> Yes correct
> 
> > bus_addr: ?
> 
> According to the guide above, this is the address into which the pci_address 
> get translated to and that is passed to the root complex. Between the root 
> complex and the CPU there can be intermediate translation layers: 

I can't quite parse this, but I do understand how a host bridge can
translate CPU physical addresses to a different range of PCI bus
addresses.  What I don't understand is the difference between "pci_addr"
and the "bus_addr" you're adding.

> see that to 
> get pci_address we call "of_translate_address"; this will apply all the 
> translation layers (ranges in the DT) that it finds till it comes to the root 
> node of the DT (thus retrieving the CPU address).
> Now said that, for designware we need the first translated PCI address, that we call
> here bus_addr after Rob Herring suggested the name...honestly I cannot think of a 
> different name
> 
>    
> 
> > I'm trying to imagine how this might be expressed in ACPI.  A host
> > bridge
> > ACPI _CRS contains a CPU physical address and applying a _TRA
> > (translation
> > offset) to the CPU address gives you a PCI bus address.  I know this
> > code
> > is OF, not ACPI, but I assume that it should be possible to describe
> > your
> > hardware via ACPI as well as by OF.

> > > diff --git a/include/linux/of_address.h b/include/linux/of_address.h
> > > index d88e81b..865f96e 100644
> > > --- a/include/linux/of_address.h
> > > +++ b/include/linux/of_address.h
> > > @@ -16,6 +16,7 @@ struct of_pci_range {
> > >  	u32 pci_space;
> > >  	u64 pci_addr;
> > >  	u64 cpu_addr;
> > > +	u64 bus_addr;
> > >  	u64 size;
> > >  	u32 flags;
> > >  };

^ permalink raw reply	[flat|nested] 73+ messages in thread
* [PATCH v6] PCI: Store PCIe bus address in struct of_pci_range
@ 2015-07-29 21:47           ` Bjorn Helgaas
  0 siblings, 0 replies; 73+ messages in thread
From: Bjorn Helgaas @ 2015-07-29 21:47 UTC (permalink / raw)
  To: linux-arm-kernel

[+cc Andrew]

On Wed, Jul 29, 2015 at 07:44:18PM +0000, Gabriele Paoloni wrote:
> > -----Original Message-----
> > From: Bjorn Helgaas [mailto:bhelgaas at google.com]
> > Sent: Wednesday, July 29, 2015 6:21 PM
> > To: Gabriele Paoloni

> > On Mon, Jul 27, 2015 at 11:17:03PM +0800, Gabriele Paoloni wrote:
> > > From: gabriele paoloni <gabriele.paoloni@huawei.com>

> > >     This patch adds a new field in "struct of_pci_range" to store the
> > >     pci bus start address; it fills the field in
> > of_pci_range_parser_one();
> > >     in of_pci_get_host_bridge_resources() it retrieves the resource
> > entry
> > >     after it is created and added to the resource list and uses
> > >     entry->__res.start to store the pci controller address
> > 
> > struct of_pci_range is starting to get confusing to non-OF folks like
> > me.
> > It now contains:
> > 
> >   u32 pci_space;
> >   u64 pci_addr;
> >   u64 cpu_addr;
> >   u64 bus_addr;
> > 
> > Can you explain what all these things mean, and maybe even add one-line
> > comments to the structure?

> > pci_space: The only uses I see are to determine whether to print
> > "Prefetch".  I don't see any real functionality that uses this.
> 
> Looking at the code I agree. it's seems to be used only in powerpc 
> and microblaze to print out.
> However from my understanding pci_space is the phys.hi field of the 
> ranges property: it defines the properties of the address space associated
> to the PCI address. if you're curious you can find a nice and quick to read
> "guide" in http://devicetree.org/MPC5200:PCI

I think pci_space should be removed and the users should test
"range.flags & IORESOURCE_PREFETCH" instead.  That's already set by
of_bus_pci_get_flags().  This is separate from your current patch, of
course.

29b635c00f3e ("of/pci: Provide support for parsing PCI DT ranges
property") added struct of_pci_range, and even at the time,
of_bus_pci_get_flags() set IORESOURCE_PREFETCH in of_pci_range.flags.

654837e8fe8d ("powerpc/pci: Use of_pci_range_parser helper in
pci_process_bridge_OF_ranges") converted powerpc to use
of_pci_range_parser() instead of parsing manually.  It converted other
references to look at struct of_pci_range.flags; I'm not sure why it
didn't do that for the prefetch bit.

I copied Andrew in case there's some subtlety here.

> > pci_addr: I assume this is a PCI bus address, like what you would see
> > if
> > you put an analyzer on the bus/link.  This address could go in a BAR.
> 
> Yes, this is the PCI start address of the range: phys.mid + phys.low in the
> guide mentioned above
> 
> > cpu_addr: I assume this is a CPU physical address, like what you would
> > see
> > in /proc/iomem and what you would pass to ioremap().
> 
> Yes correct
> 
> > bus_addr: ?
> 
> According to the guide above, this is the address into which the pci_address 
> get translated to and that is passed to the root complex. Between the root 
> complex and the CPU there can be intermediate translation layers: 

I can't quite parse this, but I do understand how a host bridge can
translate CPU physical addresses to a different range of PCI bus
addresses.  What I don't understand is the difference between "pci_addr"
and the "bus_addr" you're adding.

> see that to 
> get pci_address we call "of_translate_address"; this will apply all the 
> translation layers (ranges in the DT) that it finds till it comes to the root 
> node of the DT (thus retrieving the CPU address).
> Now said that, for designware we need the first translated PCI address, that we call
> here bus_addr after Rob Herring suggested the name...honestly I cannot think of a 
> different name
> 
>    
> 
> > I'm trying to imagine how this might be expressed in ACPI.  A host
> > bridge
> > ACPI _CRS contains a CPU physical address and applying a _TRA
> > (translation
> > offset) to the CPU address gives you a PCI bus address.  I know this
> > code
> > is OF, not ACPI, but I assume that it should be possible to describe
> > your
> > hardware via ACPI as well as by OF.

> > > diff --git a/include/linux/of_address.h b/include/linux/of_address.h
> > > index d88e81b..865f96e 100644
> > > --- a/include/linux/of_address.h
> > > +++ b/include/linux/of_address.h
> > > @@ -16,6 +16,7 @@ struct of_pci_range {
> > >  	u32 pci_space;
> > >  	u64 pci_addr;
> > >  	u64 cpu_addr;
> > > +	u64 bus_addr;
> > >  	u64 size;
> > >  	u32 flags;
> > >  };

^ permalink raw reply	[flat|nested] 73+ messages in thread
* Re: [PATCH v6] PCI: Store PCIe bus address in struct of_pci_range
  2015-07-29 19:44         ` Gabriele Paoloni
@ 2015-07-30  7:16           ` Zhou Wang
  -1 siblings, 0 replies; 73+ messages in thread
From: Zhou Wang @ 2015-07-30  7:16 UTC (permalink / raw)
  To: Gabriele Paoloni
  Cc: Bjorn Helgaas, arnd, lorenzo.pieralisi, robh+dt, james.morse,
	Liviu.Dudau, linux-pci, linux-arm-kernel, devicetree,
	Yuanzhichang, Zhudacai, zhangjukuo, qiuzhenfa, Liguozhu (Kenneth)

On 2015/7/30 3:44, Gabriele Paoloni wrote:
> Hi Bjorn
> 
> Many Thanks for your reply
> 
> I have commented back inline with resolutions from my side.
> 
> If you're ok with them I'll send it out a new version in the appropriate patchset
> 
> Cheers
> 
> Gab
> 
>> -----Original Message-----
>> From: Bjorn Helgaas [mailto:bhelgaas@google.com]
>> Sent: Wednesday, July 29, 2015 6:21 PM
>> To: Gabriele Paoloni
>> Cc: arnd@arndb.de; lorenzo.pieralisi@arm.com; Wangzhou (B);
>> robh+dt@kernel.org; james.morse@arm.com; Liviu.Dudau@arm.com; linux-
>> pci@vger.kernel.org; linux-arm-kernel@lists.infradead.org;
>> devicetree@vger.kernel.org; Yuanzhichang; Zhudacai; zhangjukuo;
>> qiuzhenfa; Liguozhu (Kenneth)
>> Subject: Re: [PATCH v6] PCI: Store PCIe bus address in struct
>> of_pci_range
>>
>> Hi Gabriele,
>>
>> As far as I can tell, this is not specific to PCIe, so please use "PCI"
>> in
>> the subject as a generic term that includes both PCI and PCIe.
> 
> sure agreed
> 
>>
>> On Mon, Jul 27, 2015 at 11:17:03PM +0800, Gabriele Paoloni wrote:
>>> From: gabriele paoloni <gabriele.paoloni@huawei.com>
>>>
>>>     This patch is needed port PCIe designware to new DT parsing API
>>>     As discussed in
>>>     http://lists.infradead.org/pipermail/linux-arm-kernel/2015-
>> January/317743.html
>>>     in designware we have a problem as the PCI addresses in the PCIe
>> controller
>>>     address space are required in order to perform correct HW
>> operation.
>>>
>>>     In order to solve this problem commit f4c55c5a3 "PCI: designware:
>>>     Program ATU with untranslated address" added code to read the
>> PCIe
>>
>> Conventional reference is 12-char SHA1, like this:
>>
>>   f4c55c5a3f7f ("PCI: designware: Program ATU with untranslated
>> address")
> 
> Agreed, will change this
> 
>>
>>>     controller start address directly from the DT ranges.
>>>
>>>     In the new DT parsing API of_pci_get_host_bridge_resources()
>> hides the
>>>     DT parser from the host controller drivers, so it is not possible
>>>     for drivers to parse values directly from the DT.
>>>
>>>     In http://www.spinics.net/lists/linux-pci/msg42540.html we
>> already tried
>>>     to use the new DT parsing API but there is a bug (obviously) in
>> setting
>>>     the <*>_mod_base addresses
>>>     Applying this patch we can easily set "<*>_mod_base = win-
>>> __res.start"
>>
>> By itself, this patch adds something.  It would help me understand it
>> if
>> the *user* of this new something were in the same patch series.
> 
> the user is: "[PATCH v5 2/5] PCI: designware: Add ARM64 support"
> I will ask Zhou Wang to include this patch in his patchset
>

Hi Gab,

I can merge this patch in my series if this make it clearer to understand.

Thanks,
Zhou

> 
>>
>>>     This patch adds a new field in "struct of_pci_range" to store the
>>>     pci bus start address; it fills the field in
>> of_pci_range_parser_one();
>>>     in of_pci_get_host_bridge_resources() it retrieves the resource
>> entry
>>>     after it is created and added to the resource list and uses
>>>     entry->__res.start to store the pci controller address
>>
>> struct of_pci_range is starting to get confusing to non-OF folks like
>> me.
>> It now contains:
>>
>>   u32 pci_space;
>>   u64 pci_addr;
>>   u64 cpu_addr;
>>   u64 bus_addr;
>>
>> Can you explain what all these things mean, and maybe even add one-line
>> comments to the structure?
> 
> sure I can add comments inline in the code
> 
>>
>> pci_space: The only uses I see are to determine whether to print
>> "Prefetch".  I don't see any real functionality that uses this.
> 
> Looking at the code I agree. it's seems to be used only in powerpc 
> and microblaze to print out.
> However from my understanding pci_space is the phys.hi field of the 
> ranges property: it defines the properties of the address space associated
> to the PCI address. if you're curious you can find a nice and quick to read
> "guide" in http://devicetree.org/MPC5200:PCI
> 
>>
>> pci_addr: I assume this is a PCI bus address, like what you would see
>> if
>> you put an analyzer on the bus/link.  This address could go in a BAR.
> 
> Yes, this is the PCI start address of the range: phys.mid + phys.low in the
> guide mentioned above
> 
>>
>> cpu_addr: I assume this is a CPU physical address, like what you would
>> see
>> in /proc/iomem and what you would pass to ioremap().
> 
> Yes correct
> 
>>
>> bus_addr: ?
>>
> 
> According to the guide above, this is the address into which the pci_address 
> get translated to and that is passed to the root complex. Between the root 
> complex and the CPU there can be intermediate translation layers: see that to 
> get pci_address we call "of_translate_address"; this will apply all the 
> translation layers (ranges in the DT) that it finds till it comes to the root 
> node of the DT (thus retrieving the CPU address).
> Now said that, for designware we need the first translated PCI address, that we call
> here bus_addr after Rob Herring suggested the name...honestly I cannot think of a 
> different name
> 
>    
> 
>> I'm trying to imagine how this might be expressed in ACPI.  A host
>> bridge
>> ACPI _CRS contains a CPU physical address and applying a _TRA
>> (translation
>> offset) to the CPU address gives you a PCI bus address.  I know this
>> code
>> is OF, not ACPI, but I assume that it should be possible to describe
>> your
>> hardware via ACPI as well as by OF.
>>
>>>     the patch is based on 4.2-rc1
>>
>> You can put this after the "---" line because it's not relevant in the
>> permanent changelog.
> 
> Agreed
> 
>>
>>>     Signed-off-by: Gabriele Paoloni <gabriele.paoloni@huawei.com>
>>>     Acked-by: Liviu Dudau <Liviu.Dudau@arm.com>
>>>     Acked-by: Rob Herring <robh@kernel.org>
>>
>> Please un-indent your changelog.
> 
> Ok agreed
> 
>>
>>> ---
>>>  drivers/of/address.c       | 2 ++
>>>  drivers/of/of_pci.c        | 4 ++++
>>>  include/linux/of_address.h | 1 +
>>>  3 files changed, 7 insertions(+)
>>>
>>> diff --git a/drivers/of/address.c b/drivers/of/address.c
>>> index 8bfda6a..23a5793 100644
>>> --- a/drivers/of/address.c
>>> +++ b/drivers/of/address.c
>>> @@ -253,6 +253,7 @@ struct of_pci_range
>> *of_pci_range_parser_one(struct of_pci_range_parser *parser,
>>>  						struct of_pci_range *range)
>>>  {
>>>  	const int na = 3, ns = 2;
>>> +	const int p_ns = of_n_size_cells(parser->node);
>>>
>>>  	if (!range)
>>>  		return NULL;
>>> @@ -265,6 +266,7 @@ struct of_pci_range
>> *of_pci_range_parser_one(struct of_pci_range_parser *parser,
>>>  	range->pci_addr = of_read_number(parser->range + 1, ns);
>>>  	range->cpu_addr = of_translate_address(parser->node,
>>>  				parser->range + na);
>>> +	range->bus_addr = of_read_number(parser->range + na, p_ns);
>>>  	range->size = of_read_number(parser->range + parser->pna + na,
>> ns);
>>>
>>>  	parser->range += parser->np;
>>> diff --git a/drivers/of/of_pci.c b/drivers/of/of_pci.c
>>> index 5751dc5..fe57030 100644
>>> --- a/drivers/of/of_pci.c
>>> +++ b/drivers/of/of_pci.c
>>> @@ -198,6 +198,7 @@ int of_pci_get_host_bridge_resources(struct
>> device_node *dev,
>>>
>>>  	pr_debug("Parsing ranges property...\n");
>>>  	for_each_of_pci_range(&parser, &range) {
>>> +		struct resource_entry *entry;
>>>  		/* Read next ranges element */
>>>  		if ((range.flags & IORESOURCE_TYPE_BITS) == IORESOURCE_IO)
>>>  			snprintf(range_type, 4, " IO");
>>> @@ -240,6 +241,9 @@ int of_pci_get_host_bridge_resources(struct
>> device_node *dev,
>>>  		}
>>>
>>>  		pci_add_resource_offset(resources, res,	res->start -
>> range.pci_addr);
>>> +		entry = list_last_entry(resources, struct resource_entry,
>> node);
>>> +		/* we are using __res for storing the PCI controller
>> address */
>>> +		entry->__res.start = range.bus_addr;
>>>  	}
>>>
>>>  	return 0;
>>> diff --git a/include/linux/of_address.h b/include/linux/of_address.h
>>> index d88e81b..865f96e 100644
>>> --- a/include/linux/of_address.h
>>> +++ b/include/linux/of_address.h
>>> @@ -16,6 +16,7 @@ struct of_pci_range {
>>>  	u32 pci_space;
>>>  	u64 pci_addr;
>>>  	u64 cpu_addr;
>>> +	u64 bus_addr;
>>>  	u64 size;
>>>  	u32 flags;
>>>  };
>>> --
>>> 1.9.1
>>>
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe linux-pci"
>> in
>>> the body of a message to majordomo@vger.kernel.org
>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 
> .
> 

^ permalink raw reply	[flat|nested] 73+ messages in thread
* [PATCH v6] PCI: Store PCIe bus address in struct of_pci_range
@ 2015-07-30  7:16           ` Zhou Wang
  0 siblings, 0 replies; 73+ messages in thread
From: Zhou Wang @ 2015-07-30  7:16 UTC (permalink / raw)
  To: linux-arm-kernel

On 2015/7/30 3:44, Gabriele Paoloni wrote:
> Hi Bjorn
> 
> Many Thanks for your reply
> 
> I have commented back inline with resolutions from my side.
> 
> If you're ok with them I'll send it out a new version in the appropriate patchset
> 
> Cheers
> 
> Gab
> 
>> -----Original Message-----
>> From: Bjorn Helgaas [mailto:bhelgaas at google.com]
>> Sent: Wednesday, July 29, 2015 6:21 PM
>> To: Gabriele Paoloni
>> Cc: arnd at arndb.de; lorenzo.pieralisi at arm.com; Wangzhou (B);
>> robh+dt at kernel.org; james.morse at arm.com; Liviu.Dudau at arm.com; linux-
>> pci at vger.kernel.org; linux-arm-kernel at lists.infradead.org;
>> devicetree at vger.kernel.org; Yuanzhichang; Zhudacai; zhangjukuo;
>> qiuzhenfa; Liguozhu (Kenneth)
>> Subject: Re: [PATCH v6] PCI: Store PCIe bus address in struct
>> of_pci_range
>>
>> Hi Gabriele,
>>
>> As far as I can tell, this is not specific to PCIe, so please use "PCI"
>> in
>> the subject as a generic term that includes both PCI and PCIe.
> 
> sure agreed
> 
>>
>> On Mon, Jul 27, 2015 at 11:17:03PM +0800, Gabriele Paoloni wrote:
>>> From: gabriele paoloni <gabriele.paoloni@huawei.com>
>>>
>>>     This patch is needed port PCIe designware to new DT parsing API
>>>     As discussed in
>>>     http://lists.infradead.org/pipermail/linux-arm-kernel/2015-
>> January/317743.html
>>>     in designware we have a problem as the PCI addresses in the PCIe
>> controller
>>>     address space are required in order to perform correct HW
>> operation.
>>>
>>>     In order to solve this problem commit f4c55c5a3 "PCI: designware:
>>>     Program ATU with untranslated address" added code to read the
>> PCIe
>>
>> Conventional reference is 12-char SHA1, like this:
>>
>>   f4c55c5a3f7f ("PCI: designware: Program ATU with untranslated
>> address")
> 
> Agreed, will change this
> 
>>
>>>     controller start address directly from the DT ranges.
>>>
>>>     In the new DT parsing API of_pci_get_host_bridge_resources()
>> hides the
>>>     DT parser from the host controller drivers, so it is not possible
>>>     for drivers to parse values directly from the DT.
>>>
>>>     In http://www.spinics.net/lists/linux-pci/msg42540.html we
>> already tried
>>>     to use the new DT parsing API but there is a bug (obviously) in
>> setting
>>>     the <*>_mod_base addresses
>>>     Applying this patch we can easily set "<*>_mod_base = win-
>>> __res.start"
>>
>> By itself, this patch adds something.  It would help me understand it
>> if
>> the *user* of this new something were in the same patch series.
> 
> the user is: "[PATCH v5 2/5] PCI: designware: Add ARM64 support"
> I will ask Zhou Wang to include this patch in his patchset
>

Hi Gab,

I can merge this patch in my series if this make it clearer to understand.

Thanks,
Zhou

> 
>>
>>>     This patch adds a new field in "struct of_pci_range" to store the
>>>     pci bus start address; it fills the field in
>> of_pci_range_parser_one();
>>>     in of_pci_get_host_bridge_resources() it retrieves the resource
>> entry
>>>     after it is created and added to the resource list and uses
>>>     entry->__res.start to store the pci controller address
>>
>> struct of_pci_range is starting to get confusing to non-OF folks like
>> me.
>> It now contains:
>>
>>   u32 pci_space;
>>   u64 pci_addr;
>>   u64 cpu_addr;
>>   u64 bus_addr;
>>
>> Can you explain what all these things mean, and maybe even add one-line
>> comments to the structure?
> 
> sure I can add comments inline in the code
> 
>>
>> pci_space: The only uses I see are to determine whether to print
>> "Prefetch".  I don't see any real functionality that uses this.
> 
> Looking at the code I agree. it's seems to be used only in powerpc 
> and microblaze to print out.
> However from my understanding pci_space is the phys.hi field of the 
> ranges property: it defines the properties of the address space associated
> to the PCI address. if you're curious you can find a nice and quick to read
> "guide" in http://devicetree.org/MPC5200:PCI
> 
>>
>> pci_addr: I assume this is a PCI bus address, like what you would see
>> if
>> you put an analyzer on the bus/link.  This address could go in a BAR.
> 
> Yes, this is the PCI start address of the range: phys.mid + phys.low in the
> guide mentioned above
> 
>>
>> cpu_addr: I assume this is a CPU physical address, like what you would
>> see
>> in /proc/iomem and what you would pass to ioremap().
> 
> Yes correct
> 
>>
>> bus_addr: ?
>>
> 
> According to the guide above, this is the address into which the pci_address 
> get translated to and that is passed to the root complex. Between the root 
> complex and the CPU there can be intermediate translation layers: see that to 
> get pci_address we call "of_translate_address"; this will apply all the 
> translation layers (ranges in the DT) that it finds till it comes to the root 
> node of the DT (thus retrieving the CPU address).
> Now said that, for designware we need the first translated PCI address, that we call
> here bus_addr after Rob Herring suggested the name...honestly I cannot think of a 
> different name
> 
>    
> 
>> I'm trying to imagine how this might be expressed in ACPI.  A host
>> bridge
>> ACPI _CRS contains a CPU physical address and applying a _TRA
>> (translation
>> offset) to the CPU address gives you a PCI bus address.  I know this
>> code
>> is OF, not ACPI, but I assume that it should be possible to describe
>> your
>> hardware via ACPI as well as by OF.
>>
>>>     the patch is based on 4.2-rc1
>>
>> You can put this after the "---" line because it's not relevant in the
>> permanent changelog.
> 
> Agreed
> 
>>
>>>     Signed-off-by: Gabriele Paoloni <gabriele.paoloni@huawei.com>
>>>     Acked-by: Liviu Dudau <Liviu.Dudau@arm.com>
>>>     Acked-by: Rob Herring <robh@kernel.org>
>>
>> Please un-indent your changelog.
> 
> Ok agreed
> 
>>
>>> ---
>>>  drivers/of/address.c       | 2 ++
>>>  drivers/of/of_pci.c        | 4 ++++
>>>  include/linux/of_address.h | 1 +
>>>  3 files changed, 7 insertions(+)
>>>
>>> diff --git a/drivers/of/address.c b/drivers/of/address.c
>>> index 8bfda6a..23a5793 100644
>>> --- a/drivers/of/address.c
>>> +++ b/drivers/of/address.c
>>> @@ -253,6 +253,7 @@ struct of_pci_range
>> *of_pci_range_parser_one(struct of_pci_range_parser *parser,
>>>  						struct of_pci_range *range)
>>>  {
>>>  	const int na = 3, ns = 2;
>>> +	const int p_ns = of_n_size_cells(parser->node);
>>>
>>>  	if (!range)
>>>  		return NULL;
>>> @@ -265,6 +266,7 @@ struct of_pci_range
>> *of_pci_range_parser_one(struct of_pci_range_parser *parser,
>>>  	range->pci_addr = of_read_number(parser->range + 1, ns);
>>>  	range->cpu_addr = of_translate_address(parser->node,
>>>  				parser->range + na);
>>> +	range->bus_addr = of_read_number(parser->range + na, p_ns);
>>>  	range->size = of_read_number(parser->range + parser->pna + na,
>> ns);
>>>
>>>  	parser->range += parser->np;
>>> diff --git a/drivers/of/of_pci.c b/drivers/of/of_pci.c
>>> index 5751dc5..fe57030 100644
>>> --- a/drivers/of/of_pci.c
>>> +++ b/drivers/of/of_pci.c
>>> @@ -198,6 +198,7 @@ int of_pci_get_host_bridge_resources(struct
>> device_node *dev,
>>>
>>>  	pr_debug("Parsing ranges property...\n");
>>>  	for_each_of_pci_range(&parser, &range) {
>>> +		struct resource_entry *entry;
>>>  		/* Read next ranges element */
>>>  		if ((range.flags & IORESOURCE_TYPE_BITS) == IORESOURCE_IO)
>>>  			snprintf(range_type, 4, " IO");
>>> @@ -240,6 +241,9 @@ int of_pci_get_host_bridge_resources(struct
>> device_node *dev,
>>>  		}
>>>
>>>  		pci_add_resource_offset(resources, res,	res->start -
>> range.pci_addr);
>>> +		entry = list_last_entry(resources, struct resource_entry,
>> node);
>>> +		/* we are using __res for storing the PCI controller
>> address */
>>> +		entry->__res.start = range.bus_addr;
>>>  	}
>>>
>>>  	return 0;
>>> diff --git a/include/linux/of_address.h b/include/linux/of_address.h
>>> index d88e81b..865f96e 100644
>>> --- a/include/linux/of_address.h
>>> +++ b/include/linux/of_address.h
>>> @@ -16,6 +16,7 @@ struct of_pci_range {
>>>  	u32 pci_space;
>>>  	u64 pci_addr;
>>>  	u64 cpu_addr;
>>> +	u64 bus_addr;
>>>  	u64 size;
>>>  	u32 flags;
>>>  };
>>> --
>>> 1.9.1
>>>
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe linux-pci"
>> in
>>> the body of a message to majordomo at vger.kernel.org
>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 
> .
> 

^ permalink raw reply	[flat|nested] 73+ messages in thread
* RE: [PATCH v6] PCI: Store PCIe bus address in struct of_pci_range
  2015-07-29 21:47           ` Bjorn Helgaas
@ 2015-07-30  8:30             ` Gabriele Paoloni
  -1 siblings, 0 replies; 73+ messages in thread
From: Gabriele Paoloni @ 2015-07-30  8:30 UTC (permalink / raw)
  To: Bjorn Helgaas
  Cc: arnd, lorenzo.pieralisi, Wangzhou (B),
	robh+dt, james.morse, Liviu.Dudau, linux-pci, linux-arm-kernel,
	devicetree, Yuanzhichang, Zhudacai, zhangjukuo, qiuzhenfa,
	Liguozhu (Kenneth),
	Andrew Murray

> -----Original Message-----
> From: Bjorn Helgaas [mailto:bhelgaas@google.com]
> Sent: Wednesday, July 29, 2015 10:47 PM
> To: Gabriele Paoloni
> Cc: arnd@arndb.de; lorenzo.pieralisi@arm.com; Wangzhou (B);
> robh+dt@kernel.org; james.morse@arm.com; Liviu.Dudau@arm.com; linux-
> pci@vger.kernel.org; linux-arm-kernel@lists.infradead.org;
> devicetree@vger.kernel.org; Yuanzhichang; Zhudacai; zhangjukuo;
> qiuzhenfa; Liguozhu (Kenneth); Andrew Murray
> Subject: Re: [PATCH v6] PCI: Store PCIe bus address in struct
> of_pci_range
> 
> [+cc Andrew]
> 
> On Wed, Jul 29, 2015 at 07:44:18PM +0000, Gabriele Paoloni wrote:
> > > -----Original Message-----
> > > From: Bjorn Helgaas [mailto:bhelgaas@google.com]
> > > Sent: Wednesday, July 29, 2015 6:21 PM
> > > To: Gabriele Paoloni
> 
> > > On Mon, Jul 27, 2015 at 11:17:03PM +0800, Gabriele Paoloni wrote:
> > > > From: gabriele paoloni <gabriele.paoloni@huawei.com>
> 
> > > >     This patch adds a new field in "struct of_pci_range" to store
> the
> > > >     pci bus start address; it fills the field in
> > > of_pci_range_parser_one();
> > > >     in of_pci_get_host_bridge_resources() it retrieves the
> resource
> > > entry
> > > >     after it is created and added to the resource list and uses
> > > >     entry->__res.start to store the pci controller address
> > >
> > > struct of_pci_range is starting to get confusing to non-OF folks
> like
> > > me.
> > > It now contains:
> > >
> > >   u32 pci_space;
> > >   u64 pci_addr;
> > >   u64 cpu_addr;
> > >   u64 bus_addr;
> > >
> > > Can you explain what all these things mean, and maybe even add one-
> line
> > > comments to the structure?
> 
> > > pci_space: The only uses I see are to determine whether to print
> > > "Prefetch".  I don't see any real functionality that uses this.
> >
> > Looking at the code I agree. it's seems to be used only in powerpc
> > and microblaze to print out.
> > However from my understanding pci_space is the phys.hi field of the
> > ranges property: it defines the properties of the address space
> associated
> > to the PCI address. if you're curious you can find a nice and quick
> to read
> > "guide" in http://devicetree.org/MPC5200:PCI
> 
> I think pci_space should be removed and the users should test
> "range.flags & IORESOURCE_PREFETCH" instead.  That's already set by
> of_bus_pci_get_flags().  This is separate from your current patch, of
> course.

Ok so I'll do nothing in my patch about this; maybe I can propose a separate
patch for this, but I cannot test it (I've got no microblaze and powerpc 
neither....)

> 
> 29b635c00f3e ("of/pci: Provide support for parsing PCI DT ranges
> property") added struct of_pci_range, and even at the time,
> of_bus_pci_get_flags() set IORESOURCE_PREFETCH in of_pci_range.flags.
> 
> 654837e8fe8d ("powerpc/pci: Use of_pci_range_parser helper in
> pci_process_bridge_OF_ranges") converted powerpc to use
> of_pci_range_parser() instead of parsing manually.  It converted other
> references to look at struct of_pci_range.flags; I'm not sure why it
> didn't do that for the prefetch bit.
> 
> I copied Andrew in case there's some subtlety here.
> 
> > > pci_addr: I assume this is a PCI bus address, like what you would
> see
> > > if
> > > you put an analyzer on the bus/link.  This address could go in a
> BAR.
> >
> > Yes, this is the PCI start address of the range: phys.mid + phys.low
> in the
> > guide mentioned above
> >
> > > cpu_addr: I assume this is a CPU physical address, like what you
> would
> > > see
> > > in /proc/iomem and what you would pass to ioremap().
> >
> > Yes correct
> >
> > > bus_addr: ?
> >
> > According to the guide above, this is the address into which the
> pci_address
> > get translated to and that is passed to the root complex. Between the
> root
> > complex and the CPU there can be intermediate translation layers:
> 
> I can't quite parse this, but I do understand how a host bridge can
> translate CPU physical addresses to a different range of PCI bus
> addresses.  What I don't understand is the difference between
> "pci_addr"
> and the "bus_addr" you're adding.

For example see:
http://lxr.free-electrons.com/source/arch/arm/boot/dts/imx6qdl.dtsi#L148

ranges = <0x00000800 0 0x01f00000 0x01f00000 0 0x00080000   /* configuration space */
          0x81000000 0 0          0x01f80000 0 0x00010000   /* downstream I/O */
          0x82000000 0 0x01000000 0x01000000 0 0x00f00000>; /* non-prefetchable memory *
               /\          /\          /\
           pci_space    pci_addr     bus_addr            


The host bridge performs the first translation from pci_addr to bus_addr.
If there are other ranges in the parents nodes in the DT bus_addr gets 
translated further till you get the cpu_addr.

Hope it is a bit clearer now....

> 
> > see that to
> > get pci_address we call "of_translate_address"; this will apply all
> the
> > translation layers (ranges in the DT) that it finds till it comes to
> the root
> > node of the DT (thus retrieving the CPU address).
> > Now said that, for designware we need the first translated PCI
> address, that we call
> > here bus_addr after Rob Herring suggested the name...honestly I
> cannot think of a
> > different name
> >
> >
> >
> > > I'm trying to imagine how this might be expressed in ACPI.  A host
> > > bridge
> > > ACPI _CRS contains a CPU physical address and applying a _TRA
> > > (translation
> > > offset) to the CPU address gives you a PCI bus address.  I know
> this
> > > code
> > > is OF, not ACPI, but I assume that it should be possible to
> describe
> > > your
> > > hardware via ACPI as well as by OF.
> 
> > > > diff --git a/include/linux/of_address.h
> b/include/linux/of_address.h
> > > > index d88e81b..865f96e 100644
> > > > --- a/include/linux/of_address.h
> > > > +++ b/include/linux/of_address.h
> > > > @@ -16,6 +16,7 @@ struct of_pci_range {
> > > >  	u32 pci_space;
> > > >  	u64 pci_addr;
> > > >  	u64 cpu_addr;
> > > > +	u64 bus_addr;
> > > >  	u64 size;
> > > >  	u32 flags;
> > > >  };

^ permalink raw reply	[flat|nested] 73+ messages in thread
* [PATCH v6] PCI: Store PCIe bus address in struct of_pci_range
@ 2015-07-30  8:30             ` Gabriele Paoloni
  0 siblings, 0 replies; 73+ messages in thread
From: Gabriele Paoloni @ 2015-07-30  8:30 UTC (permalink / raw)
  To: linux-arm-kernel

> -----Original Message-----
> From: Bjorn Helgaas [mailto:bhelgaas at google.com]
> Sent: Wednesday, July 29, 2015 10:47 PM
> To: Gabriele Paoloni
> Cc: arnd at arndb.de; lorenzo.pieralisi at arm.com; Wangzhou (B);
> robh+dt at kernel.org; james.morse at arm.com; Liviu.Dudau at arm.com; linux-
> pci at vger.kernel.org; linux-arm-kernel at lists.infradead.org;
> devicetree at vger.kernel.org; Yuanzhichang; Zhudacai; zhangjukuo;
> qiuzhenfa; Liguozhu (Kenneth); Andrew Murray
> Subject: Re: [PATCH v6] PCI: Store PCIe bus address in struct
> of_pci_range
> 
> [+cc Andrew]
> 
> On Wed, Jul 29, 2015 at 07:44:18PM +0000, Gabriele Paoloni wrote:
> > > -----Original Message-----
> > > From: Bjorn Helgaas [mailto:bhelgaas at google.com]
> > > Sent: Wednesday, July 29, 2015 6:21 PM
> > > To: Gabriele Paoloni
> 
> > > On Mon, Jul 27, 2015 at 11:17:03PM +0800, Gabriele Paoloni wrote:
> > > > From: gabriele paoloni <gabriele.paoloni@huawei.com>
> 
> > > >     This patch adds a new field in "struct of_pci_range" to store
> the
> > > >     pci bus start address; it fills the field in
> > > of_pci_range_parser_one();
> > > >     in of_pci_get_host_bridge_resources() it retrieves the
> resource
> > > entry
> > > >     after it is created and added to the resource list and uses
> > > >     entry->__res.start to store the pci controller address
> > >
> > > struct of_pci_range is starting to get confusing to non-OF folks
> like
> > > me.
> > > It now contains:
> > >
> > >   u32 pci_space;
> > >   u64 pci_addr;
> > >   u64 cpu_addr;
> > >   u64 bus_addr;
> > >
> > > Can you explain what all these things mean, and maybe even add one-
> line
> > > comments to the structure?
> 
> > > pci_space: The only uses I see are to determine whether to print
> > > "Prefetch".  I don't see any real functionality that uses this.
> >
> > Looking at the code I agree. it's seems to be used only in powerpc
> > and microblaze to print out.
> > However from my understanding pci_space is the phys.hi field of the
> > ranges property: it defines the properties of the address space
> associated
> > to the PCI address. if you're curious you can find a nice and quick
> to read
> > "guide" in http://devicetree.org/MPC5200:PCI
> 
> I think pci_space should be removed and the users should test
> "range.flags & IORESOURCE_PREFETCH" instead.  That's already set by
> of_bus_pci_get_flags().  This is separate from your current patch, of
> course.

Ok so I'll do nothing in my patch about this; maybe I can propose a separate
patch for this, but I cannot test it (I've got no microblaze and powerpc 
neither....)

> 
> 29b635c00f3e ("of/pci: Provide support for parsing PCI DT ranges
> property") added struct of_pci_range, and even at the time,
> of_bus_pci_get_flags() set IORESOURCE_PREFETCH in of_pci_range.flags.
> 
> 654837e8fe8d ("powerpc/pci: Use of_pci_range_parser helper in
> pci_process_bridge_OF_ranges") converted powerpc to use
> of_pci_range_parser() instead of parsing manually.  It converted other
> references to look at struct of_pci_range.flags; I'm not sure why it
> didn't do that for the prefetch bit.
> 
> I copied Andrew in case there's some subtlety here.
> 
> > > pci_addr: I assume this is a PCI bus address, like what you would
> see
> > > if
> > > you put an analyzer on the bus/link.  This address could go in a
> BAR.
> >
> > Yes, this is the PCI start address of the range: phys.mid + phys.low
> in the
> > guide mentioned above
> >
> > > cpu_addr: I assume this is a CPU physical address, like what you
> would
> > > see
> > > in /proc/iomem and what you would pass to ioremap().
> >
> > Yes correct
> >
> > > bus_addr: ?
> >
> > According to the guide above, this is the address into which the
> pci_address
> > get translated to and that is passed to the root complex. Between the
> root
> > complex and the CPU there can be intermediate translation layers:
> 
> I can't quite parse this, but I do understand how a host bridge can
> translate CPU physical addresses to a different range of PCI bus
> addresses.  What I don't understand is the difference between
> "pci_addr"
> and the "bus_addr" you're adding.

For example see:
http://lxr.free-electrons.com/source/arch/arm/boot/dts/imx6qdl.dtsi#L148

ranges = <0x00000800 0 0x01f00000 0x01f00000 0 0x00080000   /* configuration space */
          0x81000000 0 0          0x01f80000 0 0x00010000   /* downstream I/O */
          0x82000000 0 0x01000000 0x01000000 0 0x00f00000>; /* non-prefetchable memory *
               /\          /\          /\
           pci_space    pci_addr     bus_addr            


The host bridge performs the first translation from pci_addr to bus_addr.
If there are other ranges in the parents nodes in the DT bus_addr gets 
translated further till you get the cpu_addr.

Hope it is a bit clearer now....

> 
> > see that to
> > get pci_address we call "of_translate_address"; this will apply all
> the
> > translation layers (ranges in the DT) that it finds till it comes to
> the root
> > node of the DT (thus retrieving the CPU address).
> > Now said that, for designware we need the first translated PCI
> address, that we call
> > here bus_addr after Rob Herring suggested the name...honestly I
> cannot think of a
> > different name
> >
> >
> >
> > > I'm trying to imagine how this might be expressed in ACPI.  A host
> > > bridge
> > > ACPI _CRS contains a CPU physical address and applying a _TRA
> > > (translation
> > > offset) to the CPU address gives you a PCI bus address.  I know
> this
> > > code
> > > is OF, not ACPI, but I assume that it should be possible to
> describe
> > > your
> > > hardware via ACPI as well as by OF.
> 
> > > > diff --git a/include/linux/of_address.h
> b/include/linux/of_address.h
> > > > index d88e81b..865f96e 100644
> > > > --- a/include/linux/of_address.h
> > > > +++ b/include/linux/of_address.h
> > > > @@ -16,6 +16,7 @@ struct of_pci_range {
> > > >  	u32 pci_space;
> > > >  	u64 pci_addr;
> > > >  	u64 cpu_addr;
> > > > +	u64 bus_addr;
> > > >  	u64 size;
> > > >  	u32 flags;
> > > >  };

^ permalink raw reply	[flat|nested] 73+ messages in thread
* Re: [PATCH v6] PCI: Store PCIe bus address in struct of_pci_range
  2015-07-30  8:30             ` Gabriele Paoloni
@ 2015-07-30 11:20               ` Liviu Dudau
  -1 siblings, 0 replies; 73+ messages in thread
From: Liviu Dudau @ 2015-07-30 11:20 UTC (permalink / raw)
  To: Gabriele Paoloni
  Cc: Bjorn Helgaas, arnd, Lorenzo Pieralisi, Wangzhou (B),
	robh+dt, James Morse, linux-pci, linux-arm-kernel, devicetree,
	Yuanzhichang, Zhudacai, zhangjukuo, qiuzhenfa, liguozhu,
	Andrew Murray

Hello,

On Thu, Jul 30, 2015 at 09:30:41AM +0100, Gabriele Paoloni wrote:
> > -----Original Message-----
> > From: Bjorn Helgaas [mailto:bhelgaas@google.com]
> > Sent: Wednesday, July 29, 2015 10:47 PM
> > To: Gabriele Paoloni
> > Cc: arnd@arndb.de; lorenzo.pieralisi@arm.com; Wangzhou (B);
> > robh+dt@kernel.org; james.morse@arm.com; Liviu.Dudau@arm.com; linux-
> > pci@vger.kernel.org; linux-arm-kernel@lists.infradead.org;
> > devicetree@vger.kernel.org; Yuanzhichang; Zhudacai; zhangjukuo;
> > qiuzhenfa; Liguozhu (Kenneth); Andrew Murray
> > Subject: Re: [PATCH v6] PCI: Store PCIe bus address in struct
> > of_pci_range
> > 
> > [+cc Andrew]
> > 
> > On Wed, Jul 29, 2015 at 07:44:18PM +0000, Gabriele Paoloni wrote:
> > > > -----Original Message-----
> > > > From: Bjorn Helgaas [mailto:bhelgaas@google.com]
> > > > Sent: Wednesday, July 29, 2015 6:21 PM
> > > > To: Gabriele Paoloni
> > 
> > > > On Mon, Jul 27, 2015 at 11:17:03PM +0800, Gabriele Paoloni wrote:
> > > > > From: gabriele paoloni <gabriele.paoloni@huawei.com>
> > 
> > > > >     This patch adds a new field in "struct of_pci_range" to store
> > the
> > > > >     pci bus start address; it fills the field in
> > > > of_pci_range_parser_one();
> > > > >     in of_pci_get_host_bridge_resources() it retrieves the
> > resource
> > > > entry
> > > > >     after it is created and added to the resource list and uses
> > > > >     entry->__res.start to store the pci controller address
> > > >
> > > > struct of_pci_range is starting to get confusing to non-OF folks
> > like
> > > > me.
> > > > It now contains:
> > > >
> > > >   u32 pci_space;
> > > >   u64 pci_addr;
> > > >   u64 cpu_addr;
> > > >   u64 bus_addr;
> > > >
> > > > Can you explain what all these things mean, and maybe even add one-
> > > > line comments to the structure?

I can try to do that, as I worked with Andrew Murray when he did the patch.

- pci_space is indeed the range.flags variable and it was trying to keep
  the original value from the DT mainly to try to identify the config space
  in the ranges described. It has now become clear that declaring config
  space in the ranges area is wrong even if one supports ECAM so pci_space
  could be removed as suggested by Bjorn.
- pci_addr is the address that goes on the PCI(e) bus.
- cpu_addr is the translated address that the CPU uses. It does not necessarily
  means it is the same address that the Root Complex sees when requested to
  do Address Translation between host side and bus side. Also, what gets stored
  in the cpu_addr is not equal to what gets declared in the ranges property if
  there are other address translation parents between the Root Complex and the
  CPU.
- bus_addr is meant to be the un-translated cpu_addr that DesignWare needs
  in order to setup its ATS service. The reason for putting it in the of_pci_range
  is because the struct resource does not have the concept of an untranslated
  address.

Best regards,
Liviu


> > 
> > > > pci_space: The only uses I see are to determine whether to print
> > > > "Prefetch".  I don't see any real functionality that uses this.
> > >
> > > Looking at the code I agree. it's seems to be used only in powerpc
> > > and microblaze to print out.
> > > However from my understanding pci_space is the phys.hi field of the
> > > ranges property: it defines the properties of the address space
> > associated
> > > to the PCI address. if you're curious you can find a nice and quick
> > to read
> > > "guide" in http://devicetree.org/MPC5200:PCI
> > 
> > I think pci_space should be removed and the users should test
> > "range.flags & IORESOURCE_PREFETCH" instead.  That's already set by
> > of_bus_pci_get_flags().  This is separate from your current patch, of
> > course.
> 
> Ok so I'll do nothing in my patch about this; maybe I can propose a separate
> patch for this, but I cannot test it (I've got no microblaze and powerpc 
> neither....)
> 
> > 
> > 29b635c00f3e ("of/pci: Provide support for parsing PCI DT ranges
> > property") added struct of_pci_range, and even at the time,
> > of_bus_pci_get_flags() set IORESOURCE_PREFETCH in of_pci_range.flags.
> > 
> > 654837e8fe8d ("powerpc/pci: Use of_pci_range_parser helper in
> > pci_process_bridge_OF_ranges") converted powerpc to use
> > of_pci_range_parser() instead of parsing manually.  It converted other
> > references to look at struct of_pci_range.flags; I'm not sure why it
> > didn't do that for the prefetch bit.
> > 
> > I copied Andrew in case there's some subtlety here.
> > 
> > > > pci_addr: I assume this is a PCI bus address, like what you would
> > see
> > > > if
> > > > you put an analyzer on the bus/link.  This address could go in a
> > BAR.
> > >
> > > Yes, this is the PCI start address of the range: phys.mid + phys.low
> > in the
> > > guide mentioned above
> > >
> > > > cpu_addr: I assume this is a CPU physical address, like what you
> > would
> > > > see
> > > > in /proc/iomem and what you would pass to ioremap().
> > >
> > > Yes correct
> > >
> > > > bus_addr: ?
> > >
> > > According to the guide above, this is the address into which the
> > pci_address
> > > get translated to and that is passed to the root complex. Between the
> > root
> > > complex and the CPU there can be intermediate translation layers:
> > 
> > I can't quite parse this, but I do understand how a host bridge can
> > translate CPU physical addresses to a different range of PCI bus
> > addresses.  What I don't understand is the difference between
> > "pci_addr"
> > and the "bus_addr" you're adding.
> 
> For example see:
> http://lxr.free-electrons.com/source/arch/arm/boot/dts/imx6qdl.dtsi#L148
> 
> ranges = <0x00000800 0 0x01f00000 0x01f00000 0 0x00080000   /* configuration space */
>           0x81000000 0 0          0x01f80000 0 0x00010000   /* downstream I/O */
>           0x82000000 0 0x01000000 0x01000000 0 0x00f00000>; /* non-prefetchable memory *
>                /\          /\          /\
>            pci_space    pci_addr     bus_addr            
> 
> 
> The host bridge performs the first translation from pci_addr to bus_addr.
> If there are other ranges in the parents nodes in the DT bus_addr gets 
> translated further till you get the cpu_addr.
> 
> Hope it is a bit clearer now....
> 
> > 
> > > see that to
> > > get pci_address we call "of_translate_address"; this will apply all
> > the
> > > translation layers (ranges in the DT) that it finds till it comes to
> > the root
> > > node of the DT (thus retrieving the CPU address).
> > > Now said that, for designware we need the first translated PCI
> > address, that we call
> > > here bus_addr after Rob Herring suggested the name...honestly I
> > cannot think of a
> > > different name
> > >
> > >
> > >
> > > > I'm trying to imagine how this might be expressed in ACPI.  A host
> > > > bridge
> > > > ACPI _CRS contains a CPU physical address and applying a _TRA
> > > > (translation
> > > > offset) to the CPU address gives you a PCI bus address.  I know
> > this
> > > > code
> > > > is OF, not ACPI, but I assume that it should be possible to
> > describe
> > > > your
> > > > hardware via ACPI as well as by OF.
> > 
> > > > > diff --git a/include/linux/of_address.h
> > b/include/linux/of_address.h
> > > > > index d88e81b..865f96e 100644
> > > > > --- a/include/linux/of_address.h
> > > > > +++ b/include/linux/of_address.h
> > > > > @@ -16,6 +16,7 @@ struct of_pci_range {
> > > > >  	u32 pci_space;
> > > > >  	u64 pci_addr;
> > > > >  	u64 cpu_addr;
> > > > > +	u64 bus_addr;
> > > > >  	u64 size;
> > > > >  	u32 flags;
> > > > >  };
> 

-- 
====================
| I would like to |
| fix the world,  |
| but they're not |
| giving me the   |
 \ source code!  /
  ---------------
    ¯\_(ツ)_/¯

^ permalink raw reply	[flat|nested] 73+ messages in thread
* [PATCH v6] PCI: Store PCIe bus address in struct of_pci_range
@ 2015-07-30 11:20               ` Liviu Dudau
  0 siblings, 0 replies; 73+ messages in thread
From: Liviu Dudau @ 2015-07-30 11:20 UTC (permalink / raw)
  To: linux-arm-kernel

Hello,

On Thu, Jul 30, 2015 at 09:30:41AM +0100, Gabriele Paoloni wrote:
> > -----Original Message-----
> > From: Bjorn Helgaas [mailto:bhelgaas at google.com]
> > Sent: Wednesday, July 29, 2015 10:47 PM
> > To: Gabriele Paoloni
> > Cc: arnd at arndb.de; lorenzo.pieralisi at arm.com; Wangzhou (B);
> > robh+dt at kernel.org; james.morse at arm.com; Liviu.Dudau at arm.com; linux-
> > pci at vger.kernel.org; linux-arm-kernel at lists.infradead.org;
> > devicetree at vger.kernel.org; Yuanzhichang; Zhudacai; zhangjukuo;
> > qiuzhenfa; Liguozhu (Kenneth); Andrew Murray
> > Subject: Re: [PATCH v6] PCI: Store PCIe bus address in struct
> > of_pci_range
> > 
> > [+cc Andrew]
> > 
> > On Wed, Jul 29, 2015 at 07:44:18PM +0000, Gabriele Paoloni wrote:
> > > > -----Original Message-----
> > > > From: Bjorn Helgaas [mailto:bhelgaas at google.com]
> > > > Sent: Wednesday, July 29, 2015 6:21 PM
> > > > To: Gabriele Paoloni
> > 
> > > > On Mon, Jul 27, 2015 at 11:17:03PM +0800, Gabriele Paoloni wrote:
> > > > > From: gabriele paoloni <gabriele.paoloni@huawei.com>
> > 
> > > > >     This patch adds a new field in "struct of_pci_range" to store
> > the
> > > > >     pci bus start address; it fills the field in
> > > > of_pci_range_parser_one();
> > > > >     in of_pci_get_host_bridge_resources() it retrieves the
> > resource
> > > > entry
> > > > >     after it is created and added to the resource list and uses
> > > > >     entry->__res.start to store the pci controller address
> > > >
> > > > struct of_pci_range is starting to get confusing to non-OF folks
> > like
> > > > me.
> > > > It now contains:
> > > >
> > > >   u32 pci_space;
> > > >   u64 pci_addr;
> > > >   u64 cpu_addr;
> > > >   u64 bus_addr;
> > > >
> > > > Can you explain what all these things mean, and maybe even add one-
> > > > line comments to the structure?

I can try to do that, as I worked with Andrew Murray when he did the patch.

- pci_space is indeed the range.flags variable and it was trying to keep
  the original value from the DT mainly to try to identify the config space
  in the ranges described. It has now become clear that declaring config
  space in the ranges area is wrong even if one supports ECAM so pci_space
  could be removed as suggested by Bjorn.
- pci_addr is the address that goes on the PCI(e) bus.
- cpu_addr is the translated address that the CPU uses. It does not necessarily
  means it is the same address that the Root Complex sees when requested to
  do Address Translation between host side and bus side. Also, what gets stored
  in the cpu_addr is not equal to what gets declared in the ranges property if
  there are other address translation parents between the Root Complex and the
  CPU.
- bus_addr is meant to be the un-translated cpu_addr that DesignWare needs
  in order to setup its ATS service. The reason for putting it in the of_pci_range
  is because the struct resource does not have the concept of an untranslated
  address.

Best regards,
Liviu


> > 
> > > > pci_space: The only uses I see are to determine whether to print
> > > > "Prefetch".  I don't see any real functionality that uses this.
> > >
> > > Looking at the code I agree. it's seems to be used only in powerpc
> > > and microblaze to print out.
> > > However from my understanding pci_space is the phys.hi field of the
> > > ranges property: it defines the properties of the address space
> > associated
> > > to the PCI address. if you're curious you can find a nice and quick
> > to read
> > > "guide" in http://devicetree.org/MPC5200:PCI
> > 
> > I think pci_space should be removed and the users should test
> > "range.flags & IORESOURCE_PREFETCH" instead.  That's already set by
> > of_bus_pci_get_flags().  This is separate from your current patch, of
> > course.
> 
> Ok so I'll do nothing in my patch about this; maybe I can propose a separate
> patch for this, but I cannot test it (I've got no microblaze and powerpc 
> neither....)
> 
> > 
> > 29b635c00f3e ("of/pci: Provide support for parsing PCI DT ranges
> > property") added struct of_pci_range, and even at the time,
> > of_bus_pci_get_flags() set IORESOURCE_PREFETCH in of_pci_range.flags.
> > 
> > 654837e8fe8d ("powerpc/pci: Use of_pci_range_parser helper in
> > pci_process_bridge_OF_ranges") converted powerpc to use
> > of_pci_range_parser() instead of parsing manually.  It converted other
> > references to look at struct of_pci_range.flags; I'm not sure why it
> > didn't do that for the prefetch bit.
> > 
> > I copied Andrew in case there's some subtlety here.
> > 
> > > > pci_addr: I assume this is a PCI bus address, like what you would
> > see
> > > > if
> > > > you put an analyzer on the bus/link.  This address could go in a
> > BAR.
> > >
> > > Yes, this is the PCI start address of the range: phys.mid + phys.low
> > in the
> > > guide mentioned above
> > >
> > > > cpu_addr: I assume this is a CPU physical address, like what you
> > would
> > > > see
> > > > in /proc/iomem and what you would pass to ioremap().
> > >
> > > Yes correct
> > >
> > > > bus_addr: ?
> > >
> > > According to the guide above, this is the address into which the
> > pci_address
> > > get translated to and that is passed to the root complex. Between the
> > root
> > > complex and the CPU there can be intermediate translation layers:
> > 
> > I can't quite parse this, but I do understand how a host bridge can
> > translate CPU physical addresses to a different range of PCI bus
> > addresses.  What I don't understand is the difference between
> > "pci_addr"
> > and the "bus_addr" you're adding.
> 
> For example see:
> http://lxr.free-electrons.com/source/arch/arm/boot/dts/imx6qdl.dtsi#L148
> 
> ranges = <0x00000800 0 0x01f00000 0x01f00000 0 0x00080000   /* configuration space */
>           0x81000000 0 0          0x01f80000 0 0x00010000   /* downstream I/O */
>           0x82000000 0 0x01000000 0x01000000 0 0x00f00000>; /* non-prefetchable memory *
>                /\          /\          /\
>            pci_space    pci_addr     bus_addr            
> 
> 
> The host bridge performs the first translation from pci_addr to bus_addr.
> If there are other ranges in the parents nodes in the DT bus_addr gets 
> translated further till you get the cpu_addr.
> 
> Hope it is a bit clearer now....
> 
> > 
> > > see that to
> > > get pci_address we call "of_translate_address"; this will apply all
> > the
> > > translation layers (ranges in the DT) that it finds till it comes to
> > the root
> > > node of the DT (thus retrieving the CPU address).
> > > Now said that, for designware we need the first translated PCI
> > address, that we call
> > > here bus_addr after Rob Herring suggested the name...honestly I
> > cannot think of a
> > > different name
> > >
> > >
> > >
> > > > I'm trying to imagine how this might be expressed in ACPI.  A host
> > > > bridge
> > > > ACPI _CRS contains a CPU physical address and applying a _TRA
> > > > (translation
> > > > offset) to the CPU address gives you a PCI bus address.  I know
> > this
> > > > code
> > > > is OF, not ACPI, but I assume that it should be possible to
> > describe
> > > > your
> > > > hardware via ACPI as well as by OF.
> > 
> > > > > diff --git a/include/linux/of_address.h
> > b/include/linux/of_address.h
> > > > > index d88e81b..865f96e 100644
> > > > > --- a/include/linux/of_address.h
> > > > > +++ b/include/linux/of_address.h
> > > > > @@ -16,6 +16,7 @@ struct of_pci_range {
> > > > >  	u32 pci_space;
> > > > >  	u64 pci_addr;
> > > > >  	u64 cpu_addr;
> > > > > +	u64 bus_addr;
> > > > >  	u64 size;
> > > > >  	u32 flags;
> > > > >  };
> 

-- 
====================
| I would like to |
| fix the world,  |
| but they're not |
| giving me the   |
 \ source code!  /
  ---------------
    ?\_(?)_/?

^ permalink raw reply	[flat|nested] 73+ messages in thread
* Re: [PATCH v6] PCI: Store PCIe bus address in struct of_pci_range
  2015-07-29 19:44         ` Gabriele Paoloni
@ 2015-07-30 13:42           ` Rob Herring
  -1 siblings, 0 replies; 73+ messages in thread
From: Rob Herring @ 2015-07-30 13:42 UTC (permalink / raw)
  To: Gabriele Paoloni
  Cc: Bjorn Helgaas, arnd, lorenzo.pieralisi, Wangzhou (B),
	robh+dt, james.morse, Liviu.Dudau, linux-pci, linux-arm-kernel,
	devicetree, Yuanzhichang, Zhudacai, zhangjukuo, qiuzhenfa,
	Liguozhu (Kenneth)

On Wed, Jul 29, 2015 at 2:44 PM, Gabriele Paoloni
<gabriele.paoloni@huawei.com> wrote:
> Hi Bjorn
>
> Many Thanks for your reply
>
> I have commented back inline with resolutions from my side.
>
> If you're ok with them I'll send it out a new version in the appropriate patchset
>
> Cheers
>
> Gab
>
>> -----Original Message-----
>> From: Bjorn Helgaas [mailto:bhelgaas@google.com]
>> Sent: Wednesday, July 29, 2015 6:21 PM
>> To: Gabriele Paoloni
>> Cc: arnd@arndb.de; lorenzo.pieralisi@arm.com; Wangzhou (B);
>> robh+dt@kernel.org; james.morse@arm.com; Liviu.Dudau@arm.com; linux-
>> pci@vger.kernel.org; linux-arm-kernel@lists.infradead.org;
>> devicetree@vger.kernel.org; Yuanzhichang; Zhudacai; zhangjukuo;
>> qiuzhenfa; Liguozhu (Kenneth)
>> Subject: Re: [PATCH v6] PCI: Store PCIe bus address in struct
>> of_pci_range
>>
>> Hi Gabriele,
>>
>> As far as I can tell, this is not specific to PCIe, so please use "PCI"
>> in
>> the subject as a generic term that includes both PCI and PCIe.
>
> sure agreed
>
>>
>> On Mon, Jul 27, 2015 at 11:17:03PM +0800, Gabriele Paoloni wrote:
>> > From: gabriele paoloni <gabriele.paoloni@huawei.com>
>> >
>> >     This patch is needed port PCIe designware to new DT parsing API
>> >     As discussed in
>> >     http://lists.infradead.org/pipermail/linux-arm-kernel/2015-
>> January/317743.html
>> >     in designware we have a problem as the PCI addresses in the PCIe
>> controller
>> >     address space are required in order to perform correct HW
>> operation.
>> >
>> >     In order to solve this problem commit f4c55c5a3 "PCI: designware:
>> >     Program ATU with untranslated address" added code to read the
>> PCIe
>>
>> Conventional reference is 12-char SHA1, like this:
>>
>>   f4c55c5a3f7f ("PCI: designware: Program ATU with untranslated
>> address")
>
> Agreed, will change this
>
>>
>> >     controller start address directly from the DT ranges.
>> >
>> >     In the new DT parsing API of_pci_get_host_bridge_resources()
>> hides the
>> >     DT parser from the host controller drivers, so it is not possible
>> >     for drivers to parse values directly from the DT.
>> >
>> >     In http://www.spinics.net/lists/linux-pci/msg42540.html we
>> already tried
>> >     to use the new DT parsing API but there is a bug (obviously) in
>> setting
>> >     the <*>_mod_base addresses
>> >     Applying this patch we can easily set "<*>_mod_base = win-
>> >__res.start"
>>
>> By itself, this patch adds something.  It would help me understand it
>> if
>> the *user* of this new something were in the same patch series.
>
> the user is: "[PATCH v5 2/5] PCI: designware: Add ARM64 support"
> I will ask Zhou Wang to include this patch in his patchset
>
>
>>
>> >     This patch adds a new field in "struct of_pci_range" to store the
>> >     pci bus start address; it fills the field in
>> of_pci_range_parser_one();
>> >     in of_pci_get_host_bridge_resources() it retrieves the resource
>> entry
>> >     after it is created and added to the resource list and uses
>> >     entry->__res.start to store the pci controller address
>>
>> struct of_pci_range is starting to get confusing to non-OF folks like
>> me.
>> It now contains:
>>
>>   u32 pci_space;
>>   u64 pci_addr;
>>   u64 cpu_addr;
>>   u64 bus_addr;
>>
>> Can you explain what all these things mean, and maybe even add one-line
>> comments to the structure?
>
> sure I can add comments inline in the code
>
>>
>> pci_space: The only uses I see are to determine whether to print
>> "Prefetch".  I don't see any real functionality that uses this.
>
> Looking at the code I agree. it's seems to be used only in powerpc
> and microblaze to print out.
> However from my understanding pci_space is the phys.hi field of the
> ranges property: it defines the properties of the address space associated
> to the PCI address. if you're curious you can find a nice and quick to read
> "guide" in http://devicetree.org/MPC5200:PCI
>
>>
>> pci_addr: I assume this is a PCI bus address, like what you would see
>> if
>> you put an analyzer on the bus/link.  This address could go in a BAR.
>
> Yes, this is the PCI start address of the range: phys.mid + phys.low in the
> guide mentioned above
>
>>
>> cpu_addr: I assume this is a CPU physical address, like what you would
>> see
>> in /proc/iomem and what you would pass to ioremap().
>
> Yes correct
>
>>
>> bus_addr: ?
>>
>
> According to the guide above, this is the address into which the pci_address
> get translated to and that is passed to the root complex. Between the root
> complex and the CPU there can be intermediate translation layers: see that to
> get pci_address we call "of_translate_address"; this will apply all the
> translation layers (ranges in the DT) that it finds till it comes to the root
> node of the DT (thus retrieving the CPU address).
> Now said that, for designware we need the first translated PCI address, that we call

I think you mean "translated CPU address." The flow of addresses looks
like this:

CPU -> CPU bus address -> bus fabric address decoding -> bus address
-> DW PCI -> PCI address

This is quite common that an IP block does not see the full address.
It is unusual that the IP block needs to know its full address on the
slave side. It is quite common for the master side and the kernel
deals with that all the time with DMA mapping

> here bus_addr after Rob Herring suggested the name...honestly I cannot think of a
> different name

Thinking about this some more, is this really a translation versus
just a stripping of high bits? Does the DW IP have less than 32-bits
address? If so, we could express differently and just mask the CPU
address within the driver instead.

Rob

^ permalink raw reply	[flat|nested] 73+ messages in thread
* [PATCH v6] PCI: Store PCIe bus address in struct of_pci_range
@ 2015-07-30 13:42           ` Rob Herring
  0 siblings, 0 replies; 73+ messages in thread
From: Rob Herring @ 2015-07-30 13:42 UTC (permalink / raw)
  To: linux-arm-kernel

On Wed, Jul 29, 2015 at 2:44 PM, Gabriele Paoloni
<gabriele.paoloni@huawei.com> wrote:
> Hi Bjorn
>
> Many Thanks for your reply
>
> I have commented back inline with resolutions from my side.
>
> If you're ok with them I'll send it out a new version in the appropriate patchset
>
> Cheers
>
> Gab
>
>> -----Original Message-----
>> From: Bjorn Helgaas [mailto:bhelgaas at google.com]
>> Sent: Wednesday, July 29, 2015 6:21 PM
>> To: Gabriele Paoloni
>> Cc: arnd at arndb.de; lorenzo.pieralisi at arm.com; Wangzhou (B);
>> robh+dt at kernel.org; james.morse at arm.com; Liviu.Dudau at arm.com; linux-
>> pci at vger.kernel.org; linux-arm-kernel at lists.infradead.org;
>> devicetree at vger.kernel.org; Yuanzhichang; Zhudacai; zhangjukuo;
>> qiuzhenfa; Liguozhu (Kenneth)
>> Subject: Re: [PATCH v6] PCI: Store PCIe bus address in struct
>> of_pci_range
>>
>> Hi Gabriele,
>>
>> As far as I can tell, this is not specific to PCIe, so please use "PCI"
>> in
>> the subject as a generic term that includes both PCI and PCIe.
>
> sure agreed
>
>>
>> On Mon, Jul 27, 2015 at 11:17:03PM +0800, Gabriele Paoloni wrote:
>> > From: gabriele paoloni <gabriele.paoloni@huawei.com>
>> >
>> >     This patch is needed port PCIe designware to new DT parsing API
>> >     As discussed in
>> >     http://lists.infradead.org/pipermail/linux-arm-kernel/2015-
>> January/317743.html
>> >     in designware we have a problem as the PCI addresses in the PCIe
>> controller
>> >     address space are required in order to perform correct HW
>> operation.
>> >
>> >     In order to solve this problem commit f4c55c5a3 "PCI: designware:
>> >     Program ATU with untranslated address" added code to read the
>> PCIe
>>
>> Conventional reference is 12-char SHA1, like this:
>>
>>   f4c55c5a3f7f ("PCI: designware: Program ATU with untranslated
>> address")
>
> Agreed, will change this
>
>>
>> >     controller start address directly from the DT ranges.
>> >
>> >     In the new DT parsing API of_pci_get_host_bridge_resources()
>> hides the
>> >     DT parser from the host controller drivers, so it is not possible
>> >     for drivers to parse values directly from the DT.
>> >
>> >     In http://www.spinics.net/lists/linux-pci/msg42540.html we
>> already tried
>> >     to use the new DT parsing API but there is a bug (obviously) in
>> setting
>> >     the <*>_mod_base addresses
>> >     Applying this patch we can easily set "<*>_mod_base = win-
>> >__res.start"
>>
>> By itself, this patch adds something.  It would help me understand it
>> if
>> the *user* of this new something were in the same patch series.
>
> the user is: "[PATCH v5 2/5] PCI: designware: Add ARM64 support"
> I will ask Zhou Wang to include this patch in his patchset
>
>
>>
>> >     This patch adds a new field in "struct of_pci_range" to store the
>> >     pci bus start address; it fills the field in
>> of_pci_range_parser_one();
>> >     in of_pci_get_host_bridge_resources() it retrieves the resource
>> entry
>> >     after it is created and added to the resource list and uses
>> >     entry->__res.start to store the pci controller address
>>
>> struct of_pci_range is starting to get confusing to non-OF folks like
>> me.
>> It now contains:
>>
>>   u32 pci_space;
>>   u64 pci_addr;
>>   u64 cpu_addr;
>>   u64 bus_addr;
>>
>> Can you explain what all these things mean, and maybe even add one-line
>> comments to the structure?
>
> sure I can add comments inline in the code
>
>>
>> pci_space: The only uses I see are to determine whether to print
>> "Prefetch".  I don't see any real functionality that uses this.
>
> Looking at the code I agree. it's seems to be used only in powerpc
> and microblaze to print out.
> However from my understanding pci_space is the phys.hi field of the
> ranges property: it defines the properties of the address space associated
> to the PCI address. if you're curious you can find a nice and quick to read
> "guide" in http://devicetree.org/MPC5200:PCI
>
>>
>> pci_addr: I assume this is a PCI bus address, like what you would see
>> if
>> you put an analyzer on the bus/link.  This address could go in a BAR.
>
> Yes, this is the PCI start address of the range: phys.mid + phys.low in the
> guide mentioned above
>
>>
>> cpu_addr: I assume this is a CPU physical address, like what you would
>> see
>> in /proc/iomem and what you would pass to ioremap().
>
> Yes correct
>
>>
>> bus_addr: ?
>>
>
> According to the guide above, this is the address into which the pci_address
> get translated to and that is passed to the root complex. Between the root
> complex and the CPU there can be intermediate translation layers: see that to
> get pci_address we call "of_translate_address"; this will apply all the
> translation layers (ranges in the DT) that it finds till it comes to the root
> node of the DT (thus retrieving the CPU address).
> Now said that, for designware we need the first translated PCI address, that we call

I think you mean "translated CPU address." The flow of addresses looks
like this:

CPU -> CPU bus address -> bus fabric address decoding -> bus address
-> DW PCI -> PCI address

This is quite common that an IP block does not see the full address.
It is unusual that the IP block needs to know its full address on the
slave side. It is quite common for the master side and the kernel
deals with that all the time with DMA mapping

> here bus_addr after Rob Herring suggested the name...honestly I cannot think of a
> different name

Thinking about this some more, is this really a translation versus
just a stripping of high bits? Does the DW IP have less than 32-bits
address? If so, we could express differently and just mask the CPU
address within the driver instead.

Rob

^ permalink raw reply	[flat|nested] 73+ messages in thread
* RE: [PATCH v6] PCI: Store PCIe bus address in struct of_pci_range
  2015-07-30 13:42           ` Rob Herring
  (?)
@ 2015-07-30 13:52               ` Gabriele Paoloni
  -1 siblings, 0 replies; 73+ messages in thread
From: Gabriele Paoloni @ 2015-07-30 13:52 UTC (permalink / raw)
  To: Rob Herring
  Cc: Bjorn Helgaas, arnd-r2nGTMty4D4, lorenzo.pieralisi-5wv7dgnIgG8,
	Wangzhou (B),
	robh+dt-DgEjT+Ai2ygdnm+yROfE0A, james.morse-5wv7dgnIgG8,
	Liviu.Dudau-5wv7dgnIgG8, linux-pci-u79uwXL29TY76Z2rM5mHXA,
	linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
	devicetree-u79uwXL29TY76Z2rM5mHXA, Yuanzhichang, Zhudacai,
	zhangjukuo, qiuzhenfa, Liguozhu (Kenneth)



> -----Original Message-----
> From: linux-pci-owner@vger.kernel.org [mailto:linux-pci-
> owner@vger.kernel.org] On Behalf Of Rob Herring
> Sent: Thursday, July 30, 2015 2:43 PM
> To: Gabriele Paoloni
> Cc: Bjorn Helgaas; arnd@arndb.de; lorenzo.pieralisi@arm.com; Wangzhou
> (B); robh+dt@kernel.org; james.morse@arm.com; Liviu.Dudau@arm.com;
> linux-pci@vger.kernel.org; linux-arm-kernel@lists.infradead.org;
> devicetree@vger.kernel.org; Yuanzhichang; Zhudacai; zhangjukuo;
> qiuzhenfa; Liguozhu (Kenneth)
> Subject: Re: [PATCH v6] PCI: Store PCIe bus address in struct
> of_pci_range
> 
> On Wed, Jul 29, 2015 at 2:44 PM, Gabriele Paoloni
> <gabriele.paoloni@huawei.com> wrote:
> > Hi Bjorn
> >
> > Many Thanks for your reply
> >
> > I have commented back inline with resolutions from my side.
> >
> > If you're ok with them I'll send it out a new version in the
> appropriate patchset
> >
> > Cheers
> >
> > Gab
> >
> >> -----Original Message-----
> >> From: Bjorn Helgaas [mailto:bhelgaas@google.com]
> >> Sent: Wednesday, July 29, 2015 6:21 PM
> >> To: Gabriele Paoloni
> >> Cc: arnd@arndb.de; lorenzo.pieralisi@arm.com; Wangzhou (B);
> >> robh+dt@kernel.org; james.morse@arm.com; Liviu.Dudau@arm.com; linux-
> >> pci@vger.kernel.org; linux-arm-kernel@lists.infradead.org;
> >> devicetree@vger.kernel.org; Yuanzhichang; Zhudacai; zhangjukuo;
> >> qiuzhenfa; Liguozhu (Kenneth)
> >> Subject: Re: [PATCH v6] PCI: Store PCIe bus address in struct
> >> of_pci_range
> >>
> >> Hi Gabriele,
> >>
> >> As far as I can tell, this is not specific to PCIe, so please use
> "PCI"
> >> in
> >> the subject as a generic term that includes both PCI and PCIe.
> >
> > sure agreed
> >
> >>
> >> On Mon, Jul 27, 2015 at 11:17:03PM +0800, Gabriele Paoloni wrote:
> >> > From: gabriele paoloni <gabriele.paoloni@huawei.com>
> >> >
> >> >     This patch is needed port PCIe designware to new DT parsing
> API
> >> >     As discussed in
> >> >     http://lists.infradead.org/pipermail/linux-arm-kernel/2015-
> >> January/317743.html
> >> >     in designware we have a problem as the PCI addresses in the
> PCIe
> >> controller
> >> >     address space are required in order to perform correct HW
> >> operation.
> >> >
> >> >     In order to solve this problem commit f4c55c5a3 "PCI:
> designware:
> >> >     Program ATU with untranslated address" added code to read the
> >> PCIe
> >>
> >> Conventional reference is 12-char SHA1, like this:
> >>
> >>   f4c55c5a3f7f ("PCI: designware: Program ATU with untranslated
> >> address")
> >
> > Agreed, will change this
> >
> >>
> >> >     controller start address directly from the DT ranges.
> >> >
> >> >     In the new DT parsing API of_pci_get_host_bridge_resources()
> >> hides the
> >> >     DT parser from the host controller drivers, so it is not
> possible
> >> >     for drivers to parse values directly from the DT.
> >> >
> >> >     In http://www.spinics.net/lists/linux-pci/msg42540.html we
> >> already tried
> >> >     to use the new DT parsing API but there is a bug (obviously)
> in
> >> setting
> >> >     the <*>_mod_base addresses
> >> >     Applying this patch we can easily set "<*>_mod_base = win-
> >> >__res.start"
> >>
> >> By itself, this patch adds something.  It would help me understand
> it
> >> if
> >> the *user* of this new something were in the same patch series.
> >
> > the user is: "[PATCH v5 2/5] PCI: designware: Add ARM64 support"
> > I will ask Zhou Wang to include this patch in his patchset
> >
> >
> >>
> >> >     This patch adds a new field in "struct of_pci_range" to store
> the
> >> >     pci bus start address; it fills the field in
> >> of_pci_range_parser_one();
> >> >     in of_pci_get_host_bridge_resources() it retrieves the
> resource
> >> entry
> >> >     after it is created and added to the resource list and uses
> >> >     entry->__res.start to store the pci controller address
> >>
> >> struct of_pci_range is starting to get confusing to non-OF folks
> like
> >> me.
> >> It now contains:
> >>
> >>   u32 pci_space;
> >>   u64 pci_addr;
> >>   u64 cpu_addr;
> >>   u64 bus_addr;
> >>
> >> Can you explain what all these things mean, and maybe even add one-
> line
> >> comments to the structure?
> >
> > sure I can add comments inline in the code
> >
> >>
> >> pci_space: The only uses I see are to determine whether to print
> >> "Prefetch".  I don't see any real functionality that uses this.
> >
> > Looking at the code I agree. it's seems to be used only in powerpc
> > and microblaze to print out.
> > However from my understanding pci_space is the phys.hi field of the
> > ranges property: it defines the properties of the address space
> associated
> > to the PCI address. if you're curious you can find a nice and quick
> to read
> > "guide" in http://devicetree.org/MPC5200:PCI
> >
> >>
> >> pci_addr: I assume this is a PCI bus address, like what you would
> see
> >> if
> >> you put an analyzer on the bus/link.  This address could go in a BAR.
> >
> > Yes, this is the PCI start address of the range: phys.mid + phys.low
> in the
> > guide mentioned above
> >
> >>
> >> cpu_addr: I assume this is a CPU physical address, like what you
> would
> >> see
> >> in /proc/iomem and what you would pass to ioremap().
> >
> > Yes correct
> >
> >>
> >> bus_addr: ?
> >>
> >
> > According to the guide above, this is the address into which the
> pci_address
> > get translated to and that is passed to the root complex. Between the
> root
> > complex and the CPU there can be intermediate translation layers: see
> that to
> > get pci_address we call "of_translate_address"; this will apply all
> the
> > translation layers (ranges in the DT) that it finds till it comes to
> the root
> > node of the DT (thus retrieving the CPU address).
> > Now said that, for designware we need the first translated PCI
> address, that we call
> 
> I think you mean "translated CPU address." The flow of addresses looks
> like this:
> 
> CPU -> CPU bus address -> bus fabric address decoding -> bus address
> -> DW PCI -> PCI address
> 
> This is quite common that an IP block does not see the full address.
> It is unusual that the IP block needs to know its full address on the
> slave side. It is quite common for the master side and the kernel
> deals with that all the time with DMA mapping
> 
> > here bus_addr after Rob Herring suggested the name...honestly I
> cannot think of a
> > different name
> 
> Thinking about this some more, is this really a translation versus
> just a stripping of high bits? Does the DW IP have less than 32-bits
> address? If so, we could express differently and just mask the CPU
> address within the driver instead.

I don’t think we should rely on PCI addresses...what if the intermediate 
translation layer changes the lower significant bits of the "bus address"
to translate into a cpu address?

In that case we're going to program the DW with a wrong address

What I am saying is that the DW driver should not rely on the 
behavior of external HW....what do you think?

> 
> Rob
> --
> To unsubscribe from this list: send the line "unsubscribe linux-pci" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 73+ messages in thread
* RE: [PATCH v6] PCI: Store PCIe bus address in struct of_pci_range
@ 2015-07-30 13:52               ` Gabriele Paoloni
  0 siblings, 0 replies; 73+ messages in thread
From: Gabriele Paoloni @ 2015-07-30 13:52 UTC (permalink / raw)
  To: Rob Herring
  Cc: Bjorn Helgaas, arnd, lorenzo.pieralisi, Wangzhou (B),
	robh+dt, james.morse, Liviu.Dudau, linux-pci, linux-arm-kernel,
	devicetree, Yuanzhichang, Zhudacai, zhangjukuo, qiuzhenfa,
	Liguozhu (Kenneth)

DQoNCj4gLS0tLS1PcmlnaW5hbCBNZXNzYWdlLS0tLS0NCj4gRnJvbTogbGludXgtcGNpLW93bmVy
QHZnZXIua2VybmVsLm9yZyBbbWFpbHRvOmxpbnV4LXBjaS0NCj4gb3duZXJAdmdlci5rZXJuZWwu
b3JnXSBPbiBCZWhhbGYgT2YgUm9iIEhlcnJpbmcNCj4gU2VudDogVGh1cnNkYXksIEp1bHkgMzAs
IDIwMTUgMjo0MyBQTQ0KPiBUbzogR2FicmllbGUgUGFvbG9uaQ0KPiBDYzogQmpvcm4gSGVsZ2Fh
czsgYXJuZEBhcm5kYi5kZTsgbG9yZW56by5waWVyYWxpc2lAYXJtLmNvbTsgV2FuZ3pob3UNCj4g
KEIpOyByb2JoK2R0QGtlcm5lbC5vcmc7IGphbWVzLm1vcnNlQGFybS5jb207IExpdml1LkR1ZGF1
QGFybS5jb207DQo+IGxpbnV4LXBjaUB2Z2VyLmtlcm5lbC5vcmc7IGxpbnV4LWFybS1rZXJuZWxA
bGlzdHMuaW5mcmFkZWFkLm9yZzsNCj4gZGV2aWNldHJlZUB2Z2VyLmtlcm5lbC5vcmc7IFl1YW56
aGljaGFuZzsgWmh1ZGFjYWk7IHpoYW5nanVrdW87DQo+IHFpdXpoZW5mYTsgTGlndW96aHUgKEtl
bm5ldGgpDQo+IFN1YmplY3Q6IFJlOiBbUEFUQ0ggdjZdIFBDSTogU3RvcmUgUENJZSBidXMgYWRk
cmVzcyBpbiBzdHJ1Y3QNCj4gb2ZfcGNpX3JhbmdlDQo+IA0KPiBPbiBXZWQsIEp1bCAyOSwgMjAx
NSBhdCAyOjQ0IFBNLCBHYWJyaWVsZSBQYW9sb25pDQo+IDxnYWJyaWVsZS5wYW9sb25pQGh1YXdl
aS5jb20+IHdyb3RlOg0KPiA+IEhpIEJqb3JuDQo+ID4NCj4gPiBNYW55IFRoYW5rcyBmb3IgeW91
ciByZXBseQ0KPiA+DQo+ID4gSSBoYXZlIGNvbW1lbnRlZCBiYWNrIGlubGluZSB3aXRoIHJlc29s
dXRpb25zIGZyb20gbXkgc2lkZS4NCj4gPg0KPiA+IElmIHlvdSdyZSBvayB3aXRoIHRoZW0gSSds
bCBzZW5kIGl0IG91dCBhIG5ldyB2ZXJzaW9uIGluIHRoZQ0KPiBhcHByb3ByaWF0ZSBwYXRjaHNl
dA0KPiA+DQo+ID4gQ2hlZXJzDQo+ID4NCj4gPiBHYWINCj4gPg0KPiA+PiAtLS0tLU9yaWdpbmFs
IE1lc3NhZ2UtLS0tLQ0KPiA+PiBGcm9tOiBCam9ybiBIZWxnYWFzIFttYWlsdG86YmhlbGdhYXNA
Z29vZ2xlLmNvbV0NCj4gPj4gU2VudDogV2VkbmVzZGF5LCBKdWx5IDI5LCAyMDE1IDY6MjEgUE0N
Cj4gPj4gVG86IEdhYnJpZWxlIFBhb2xvbmkNCj4gPj4gQ2M6IGFybmRAYXJuZGIuZGU7IGxvcmVu
em8ucGllcmFsaXNpQGFybS5jb207IFdhbmd6aG91IChCKTsNCj4gPj4gcm9iaCtkdEBrZXJuZWwu
b3JnOyBqYW1lcy5tb3JzZUBhcm0uY29tOyBMaXZpdS5EdWRhdUBhcm0uY29tOyBsaW51eC0NCj4g
Pj4gcGNpQHZnZXIua2VybmVsLm9yZzsgbGludXgtYXJtLWtlcm5lbEBsaXN0cy5pbmZyYWRlYWQu
b3JnOw0KPiA+PiBkZXZpY2V0cmVlQHZnZXIua2VybmVsLm9yZzsgWXVhbnpoaWNoYW5nOyBaaHVk
YWNhaTsgemhhbmdqdWt1bzsNCj4gPj4gcWl1emhlbmZhOyBMaWd1b3podSAoS2VubmV0aCkNCj4g
Pj4gU3ViamVjdDogUmU6IFtQQVRDSCB2Nl0gUENJOiBTdG9yZSBQQ0llIGJ1cyBhZGRyZXNzIGlu
IHN0cnVjdA0KPiA+PiBvZl9wY2lfcmFuZ2UNCj4gPj4NCj4gPj4gSGkgR2FicmllbGUsDQo+ID4+
DQo+ID4+IEFzIGZhciBhcyBJIGNhbiB0ZWxsLCB0aGlzIGlzIG5vdCBzcGVjaWZpYyB0byBQQ0ll
LCBzbyBwbGVhc2UgdXNlDQo+ICJQQ0kiDQo+ID4+IGluDQo+ID4+IHRoZSBzdWJqZWN0IGFzIGEg
Z2VuZXJpYyB0ZXJtIHRoYXQgaW5jbHVkZXMgYm90aCBQQ0kgYW5kIFBDSWUuDQo+ID4NCj4gPiBz
dXJlIGFncmVlZA0KPiA+DQo+ID4+DQo+ID4+IE9uIE1vbiwgSnVsIDI3LCAyMDE1IGF0IDExOjE3
OjAzUE0gKzA4MDAsIEdhYnJpZWxlIFBhb2xvbmkgd3JvdGU6DQo+ID4+ID4gRnJvbTogZ2Ficmll
bGUgcGFvbG9uaSA8Z2FicmllbGUucGFvbG9uaUBodWF3ZWkuY29tPg0KPiA+PiA+DQo+ID4+ID4g
ICAgIFRoaXMgcGF0Y2ggaXMgbmVlZGVkIHBvcnQgUENJZSBkZXNpZ253YXJlIHRvIG5ldyBEVCBw
YXJzaW5nDQo+IEFQSQ0KPiA+PiA+ICAgICBBcyBkaXNjdXNzZWQgaW4NCj4gPj4gPiAgICAgaHR0
cDovL2xpc3RzLmluZnJhZGVhZC5vcmcvcGlwZXJtYWlsL2xpbnV4LWFybS1rZXJuZWwvMjAxNS0N
Cj4gPj4gSmFudWFyeS8zMTc3NDMuaHRtbA0KPiA+PiA+ICAgICBpbiBkZXNpZ253YXJlIHdlIGhh
dmUgYSBwcm9ibGVtIGFzIHRoZSBQQ0kgYWRkcmVzc2VzIGluIHRoZQ0KPiBQQ0llDQo+ID4+IGNv
bnRyb2xsZXINCj4gPj4gPiAgICAgYWRkcmVzcyBzcGFjZSBhcmUgcmVxdWlyZWQgaW4gb3JkZXIg
dG8gcGVyZm9ybSBjb3JyZWN0IEhXDQo+ID4+IG9wZXJhdGlvbi4NCj4gPj4gPg0KPiA+PiA+ICAg
ICBJbiBvcmRlciB0byBzb2x2ZSB0aGlzIHByb2JsZW0gY29tbWl0IGY0YzU1YzVhMyAiUENJOg0K
PiBkZXNpZ253YXJlOg0KPiA+PiA+ICAgICBQcm9ncmFtIEFUVSB3aXRoIHVudHJhbnNsYXRlZCBh
ZGRyZXNzIiBhZGRlZCBjb2RlIHRvIHJlYWQgdGhlDQo+ID4+IFBDSWUNCj4gPj4NCj4gPj4gQ29u
dmVudGlvbmFsIHJlZmVyZW5jZSBpcyAxMi1jaGFyIFNIQTEsIGxpa2UgdGhpczoNCj4gPj4NCj4g
Pj4gICBmNGM1NWM1YTNmN2YgKCJQQ0k6IGRlc2lnbndhcmU6IFByb2dyYW0gQVRVIHdpdGggdW50
cmFuc2xhdGVkDQo+ID4+IGFkZHJlc3MiKQ0KPiA+DQo+ID4gQWdyZWVkLCB3aWxsIGNoYW5nZSB0
aGlzDQo+ID4NCj4gPj4NCj4gPj4gPiAgICAgY29udHJvbGxlciBzdGFydCBhZGRyZXNzIGRpcmVj
dGx5IGZyb20gdGhlIERUIHJhbmdlcy4NCj4gPj4gPg0KPiA+PiA+ICAgICBJbiB0aGUgbmV3IERU
IHBhcnNpbmcgQVBJIG9mX3BjaV9nZXRfaG9zdF9icmlkZ2VfcmVzb3VyY2VzKCkNCj4gPj4gaGlk
ZXMgdGhlDQo+ID4+ID4gICAgIERUIHBhcnNlciBmcm9tIHRoZSBob3N0IGNvbnRyb2xsZXIgZHJp
dmVycywgc28gaXQgaXMgbm90DQo+IHBvc3NpYmxlDQo+ID4+ID4gICAgIGZvciBkcml2ZXJzIHRv
IHBhcnNlIHZhbHVlcyBkaXJlY3RseSBmcm9tIHRoZSBEVC4NCj4gPj4gPg0KPiA+PiA+ICAgICBJ
biBodHRwOi8vd3d3LnNwaW5pY3MubmV0L2xpc3RzL2xpbnV4LXBjaS9tc2c0MjU0MC5odG1sIHdl
DQo+ID4+IGFscmVhZHkgdHJpZWQNCj4gPj4gPiAgICAgdG8gdXNlIHRoZSBuZXcgRFQgcGFyc2lu
ZyBBUEkgYnV0IHRoZXJlIGlzIGEgYnVnIChvYnZpb3VzbHkpDQo+IGluDQo+ID4+IHNldHRpbmcN
Cj4gPj4gPiAgICAgdGhlIDwqPl9tb2RfYmFzZSBhZGRyZXNzZXMNCj4gPj4gPiAgICAgQXBwbHlp
bmcgdGhpcyBwYXRjaCB3ZSBjYW4gZWFzaWx5IHNldCAiPCo+X21vZF9iYXNlID0gd2luLQ0KPiA+
PiA+X19yZXMuc3RhcnQiDQo+ID4+DQo+ID4+IEJ5IGl0c2VsZiwgdGhpcyBwYXRjaCBhZGRzIHNv
bWV0aGluZy4gIEl0IHdvdWxkIGhlbHAgbWUgdW5kZXJzdGFuZA0KPiBpdA0KPiA+PiBpZg0KPiA+
PiB0aGUgKnVzZXIqIG9mIHRoaXMgbmV3IHNvbWV0aGluZyB3ZXJlIGluIHRoZSBzYW1lIHBhdGNo
IHNlcmllcy4NCj4gPg0KPiA+IHRoZSB1c2VyIGlzOiAiW1BBVENIIHY1IDIvNV0gUENJOiBkZXNp
Z253YXJlOiBBZGQgQVJNNjQgc3VwcG9ydCINCj4gPiBJIHdpbGwgYXNrIFpob3UgV2FuZyB0byBp
bmNsdWRlIHRoaXMgcGF0Y2ggaW4gaGlzIHBhdGNoc2V0DQo+ID4NCj4gPg0KPiA+Pg0KPiA+PiA+
ICAgICBUaGlzIHBhdGNoIGFkZHMgYSBuZXcgZmllbGQgaW4gInN0cnVjdCBvZl9wY2lfcmFuZ2Ui
IHRvIHN0b3JlDQo+IHRoZQ0KPiA+PiA+ICAgICBwY2kgYnVzIHN0YXJ0IGFkZHJlc3M7IGl0IGZp
bGxzIHRoZSBmaWVsZCBpbg0KPiA+PiBvZl9wY2lfcmFuZ2VfcGFyc2VyX29uZSgpOw0KPiA+PiA+
ICAgICBpbiBvZl9wY2lfZ2V0X2hvc3RfYnJpZGdlX3Jlc291cmNlcygpIGl0IHJldHJpZXZlcyB0
aGUNCj4gcmVzb3VyY2UNCj4gPj4gZW50cnkNCj4gPj4gPiAgICAgYWZ0ZXIgaXQgaXMgY3JlYXRl
ZCBhbmQgYWRkZWQgdG8gdGhlIHJlc291cmNlIGxpc3QgYW5kIHVzZXMNCj4gPj4gPiAgICAgZW50
cnktPl9fcmVzLnN0YXJ0IHRvIHN0b3JlIHRoZSBwY2kgY29udHJvbGxlciBhZGRyZXNzDQo+ID4+
DQo+ID4+IHN0cnVjdCBvZl9wY2lfcmFuZ2UgaXMgc3RhcnRpbmcgdG8gZ2V0IGNvbmZ1c2luZyB0
byBub24tT0YgZm9sa3MNCj4gbGlrZQ0KPiA+PiBtZS4NCj4gPj4gSXQgbm93IGNvbnRhaW5zOg0K
PiA+Pg0KPiA+PiAgIHUzMiBwY2lfc3BhY2U7DQo+ID4+ICAgdTY0IHBjaV9hZGRyOw0KPiA+PiAg
IHU2NCBjcHVfYWRkcjsNCj4gPj4gICB1NjQgYnVzX2FkZHI7DQo+ID4+DQo+ID4+IENhbiB5b3Ug
ZXhwbGFpbiB3aGF0IGFsbCB0aGVzZSB0aGluZ3MgbWVhbiwgYW5kIG1heWJlIGV2ZW4gYWRkIG9u
ZS0NCj4gbGluZQ0KPiA+PiBjb21tZW50cyB0byB0aGUgc3RydWN0dXJlPw0KPiA+DQo+ID4gc3Vy
ZSBJIGNhbiBhZGQgY29tbWVudHMgaW5saW5lIGluIHRoZSBjb2RlDQo+ID4NCj4gPj4NCj4gPj4g
cGNpX3NwYWNlOiBUaGUgb25seSB1c2VzIEkgc2VlIGFyZSB0byBkZXRlcm1pbmUgd2hldGhlciB0
byBwcmludA0KPiA+PiAiUHJlZmV0Y2giLiAgSSBkb24ndCBzZWUgYW55IHJlYWwgZnVuY3Rpb25h
bGl0eSB0aGF0IHVzZXMgdGhpcy4NCj4gPg0KPiA+IExvb2tpbmcgYXQgdGhlIGNvZGUgSSBhZ3Jl
ZS4gaXQncyBzZWVtcyB0byBiZSB1c2VkIG9ubHkgaW4gcG93ZXJwYw0KPiA+IGFuZCBtaWNyb2Js
YXplIHRvIHByaW50IG91dC4NCj4gPiBIb3dldmVyIGZyb20gbXkgdW5kZXJzdGFuZGluZyBwY2lf
c3BhY2UgaXMgdGhlIHBoeXMuaGkgZmllbGQgb2YgdGhlDQo+ID4gcmFuZ2VzIHByb3BlcnR5OiBp
dCBkZWZpbmVzIHRoZSBwcm9wZXJ0aWVzIG9mIHRoZSBhZGRyZXNzIHNwYWNlDQo+IGFzc29jaWF0
ZWQNCj4gPiB0byB0aGUgUENJIGFkZHJlc3MuIGlmIHlvdSdyZSBjdXJpb3VzIHlvdSBjYW4gZmlu
ZCBhIG5pY2UgYW5kIHF1aWNrDQo+IHRvIHJlYWQNCj4gPiAiZ3VpZGUiIGluIGh0dHA6Ly9kZXZp
Y2V0cmVlLm9yZy9NUEM1MjAwOlBDSQ0KPiA+DQo+ID4+DQo+ID4+IHBjaV9hZGRyOiBJIGFzc3Vt
ZSB0aGlzIGlzIGEgUENJIGJ1cyBhZGRyZXNzLCBsaWtlIHdoYXQgeW91IHdvdWxkDQo+IHNlZQ0K
PiA+PiBpZg0KPiA+PiB5b3UgcHV0IGFuIGFuYWx5emVyIG9uIHRoZSBidXMvbGluay4gIFRoaXMg
YWRkcmVzcyBjb3VsZCBnbyBpbiBhIEJBUi4NCj4gPg0KPiA+IFllcywgdGhpcyBpcyB0aGUgUENJ
IHN0YXJ0IGFkZHJlc3Mgb2YgdGhlIHJhbmdlOiBwaHlzLm1pZCArIHBoeXMubG93DQo+IGluIHRo
ZQ0KPiA+IGd1aWRlIG1lbnRpb25lZCBhYm92ZQ0KPiA+DQo+ID4+DQo+ID4+IGNwdV9hZGRyOiBJ
IGFzc3VtZSB0aGlzIGlzIGEgQ1BVIHBoeXNpY2FsIGFkZHJlc3MsIGxpa2Ugd2hhdCB5b3UNCj4g
d291bGQNCj4gPj4gc2VlDQo+ID4+IGluIC9wcm9jL2lvbWVtIGFuZCB3aGF0IHlvdSB3b3VsZCBw
YXNzIHRvIGlvcmVtYXAoKS4NCj4gPg0KPiA+IFllcyBjb3JyZWN0DQo+ID4NCj4gPj4NCj4gPj4g
YnVzX2FkZHI6ID8NCj4gPj4NCj4gPg0KPiA+IEFjY29yZGluZyB0byB0aGUgZ3VpZGUgYWJvdmUs
IHRoaXMgaXMgdGhlIGFkZHJlc3MgaW50byB3aGljaCB0aGUNCj4gcGNpX2FkZHJlc3MNCj4gPiBn
ZXQgdHJhbnNsYXRlZCB0byBhbmQgdGhhdCBpcyBwYXNzZWQgdG8gdGhlIHJvb3QgY29tcGxleC4g
QmV0d2VlbiB0aGUNCj4gcm9vdA0KPiA+IGNvbXBsZXggYW5kIHRoZSBDUFUgdGhlcmUgY2FuIGJl
IGludGVybWVkaWF0ZSB0cmFuc2xhdGlvbiBsYXllcnM6IHNlZQ0KPiB0aGF0IHRvDQo+ID4gZ2V0
IHBjaV9hZGRyZXNzIHdlIGNhbGwgIm9mX3RyYW5zbGF0ZV9hZGRyZXNzIjsgdGhpcyB3aWxsIGFw
cGx5IGFsbA0KPiB0aGUNCj4gPiB0cmFuc2xhdGlvbiBsYXllcnMgKHJhbmdlcyBpbiB0aGUgRFQp
IHRoYXQgaXQgZmluZHMgdGlsbCBpdCBjb21lcyB0bw0KPiB0aGUgcm9vdA0KPiA+IG5vZGUgb2Yg
dGhlIERUICh0aHVzIHJldHJpZXZpbmcgdGhlIENQVSBhZGRyZXNzKS4NCj4gPiBOb3cgc2FpZCB0
aGF0LCBmb3IgZGVzaWdud2FyZSB3ZSBuZWVkIHRoZSBmaXJzdCB0cmFuc2xhdGVkIFBDSQ0KPiBh
ZGRyZXNzLCB0aGF0IHdlIGNhbGwNCj4gDQo+IEkgdGhpbmsgeW91IG1lYW4gInRyYW5zbGF0ZWQg
Q1BVIGFkZHJlc3MuIiBUaGUgZmxvdyBvZiBhZGRyZXNzZXMgbG9va3MNCj4gbGlrZSB0aGlzOg0K
PiANCj4gQ1BVIC0+IENQVSBidXMgYWRkcmVzcyAtPiBidXMgZmFicmljIGFkZHJlc3MgZGVjb2Rp
bmcgLT4gYnVzIGFkZHJlc3MNCj4gLT4gRFcgUENJIC0+IFBDSSBhZGRyZXNzDQo+IA0KPiBUaGlz
IGlzIHF1aXRlIGNvbW1vbiB0aGF0IGFuIElQIGJsb2NrIGRvZXMgbm90IHNlZSB0aGUgZnVsbCBh
ZGRyZXNzLg0KPiBJdCBpcyB1bnVzdWFsIHRoYXQgdGhlIElQIGJsb2NrIG5lZWRzIHRvIGtub3cg
aXRzIGZ1bGwgYWRkcmVzcyBvbiB0aGUNCj4gc2xhdmUgc2lkZS4gSXQgaXMgcXVpdGUgY29tbW9u
IGZvciB0aGUgbWFzdGVyIHNpZGUgYW5kIHRoZSBrZXJuZWwNCj4gZGVhbHMgd2l0aCB0aGF0IGFs
bCB0aGUgdGltZSB3aXRoIERNQSBtYXBwaW5nDQo+IA0KPiA+IGhlcmUgYnVzX2FkZHIgYWZ0ZXIg
Um9iIEhlcnJpbmcgc3VnZ2VzdGVkIHRoZSBuYW1lLi4uaG9uZXN0bHkgSQ0KPiBjYW5ub3QgdGhp
bmsgb2YgYQ0KPiA+IGRpZmZlcmVudCBuYW1lDQo+IA0KPiBUaGlua2luZyBhYm91dCB0aGlzIHNv
bWUgbW9yZSwgaXMgdGhpcyByZWFsbHkgYSB0cmFuc2xhdGlvbiB2ZXJzdXMNCj4ganVzdCBhIHN0
cmlwcGluZyBvZiBoaWdoIGJpdHM/IERvZXMgdGhlIERXIElQIGhhdmUgbGVzcyB0aGFuIDMyLWJp
dHMNCj4gYWRkcmVzcz8gSWYgc28sIHdlIGNvdWxkIGV4cHJlc3MgZGlmZmVyZW50bHkgYW5kIGp1
c3QgbWFzayB0aGUgQ1BVDQo+IGFkZHJlc3Mgd2l0aGluIHRoZSBkcml2ZXIgaW5zdGVhZC4NCg0K
SSBkb27igJl0IHRoaW5rIHdlIHNob3VsZCByZWx5IG9uIFBDSSBhZGRyZXNzZXMuLi53aGF0IGlm
IHRoZSBpbnRlcm1lZGlhdGUgDQp0cmFuc2xhdGlvbiBsYXllciBjaGFuZ2VzIHRoZSBsb3dlciBz
aWduaWZpY2FudCBiaXRzIG9mIHRoZSAiYnVzIGFkZHJlc3MiDQp0byB0cmFuc2xhdGUgaW50byBh
IGNwdSBhZGRyZXNzPw0KDQpJbiB0aGF0IGNhc2Ugd2UncmUgZ29pbmcgdG8gcHJvZ3JhbSB0aGUg
RFcgd2l0aCBhIHdyb25nIGFkZHJlc3MNCg0KV2hhdCBJIGFtIHNheWluZyBpcyB0aGF0IHRoZSBE
VyBkcml2ZXIgc2hvdWxkIG5vdCByZWx5IG9uIHRoZSANCmJlaGF2aW9yIG9mIGV4dGVybmFsIEhX
Li4uLndoYXQgZG8geW91IHRoaW5rPw0KDQo+IA0KPiBSb2INCj4gLS0NCj4gVG8gdW5zdWJzY3Jp
YmUgZnJvbSB0aGlzIGxpc3Q6IHNlbmQgdGhlIGxpbmUgInVuc3Vic2NyaWJlIGxpbnV4LXBjaSIg
aW4NCj4gdGhlIGJvZHkgb2YgYSBtZXNzYWdlIHRvIG1ham9yZG9tb0B2Z2VyLmtlcm5lbC5vcmcN
Cj4gTW9yZSBtYWpvcmRvbW8gaW5mbyBhdCAgaHR0cDovL3ZnZXIua2VybmVsLm9yZy9tYWpvcmRv
bW8taW5mby5odG1sDQo=

^ permalink raw reply	[flat|nested] 73+ messages in thread
* [PATCH v6] PCI: Store PCIe bus address in struct of_pci_range
@ 2015-07-30 13:52               ` Gabriele Paoloni
  0 siblings, 0 replies; 73+ messages in thread
From: Gabriele Paoloni @ 2015-07-30 13:52 UTC (permalink / raw)
  To: linux-arm-kernel



> -----Original Message-----
> From: linux-pci-owner at vger.kernel.org [mailto:linux-pci-
> owner at vger.kernel.org] On Behalf Of Rob Herring
> Sent: Thursday, July 30, 2015 2:43 PM
> To: Gabriele Paoloni
> Cc: Bjorn Helgaas; arnd at arndb.de; lorenzo.pieralisi at arm.com; Wangzhou
> (B); robh+dt at kernel.org; james.morse at arm.com; Liviu.Dudau at arm.com;
> linux-pci at vger.kernel.org; linux-arm-kernel at lists.infradead.org;
> devicetree at vger.kernel.org; Yuanzhichang; Zhudacai; zhangjukuo;
> qiuzhenfa; Liguozhu (Kenneth)
> Subject: Re: [PATCH v6] PCI: Store PCIe bus address in struct
> of_pci_range
> 
> On Wed, Jul 29, 2015 at 2:44 PM, Gabriele Paoloni
> <gabriele.paoloni@huawei.com> wrote:
> > Hi Bjorn
> >
> > Many Thanks for your reply
> >
> > I have commented back inline with resolutions from my side.
> >
> > If you're ok with them I'll send it out a new version in the
> appropriate patchset
> >
> > Cheers
> >
> > Gab
> >
> >> -----Original Message-----
> >> From: Bjorn Helgaas [mailto:bhelgaas at google.com]
> >> Sent: Wednesday, July 29, 2015 6:21 PM
> >> To: Gabriele Paoloni
> >> Cc: arnd at arndb.de; lorenzo.pieralisi at arm.com; Wangzhou (B);
> >> robh+dt at kernel.org; james.morse at arm.com; Liviu.Dudau at arm.com; linux-
> >> pci at vger.kernel.org; linux-arm-kernel at lists.infradead.org;
> >> devicetree at vger.kernel.org; Yuanzhichang; Zhudacai; zhangjukuo;
> >> qiuzhenfa; Liguozhu (Kenneth)
> >> Subject: Re: [PATCH v6] PCI: Store PCIe bus address in struct
> >> of_pci_range
> >>
> >> Hi Gabriele,
> >>
> >> As far as I can tell, this is not specific to PCIe, so please use
> "PCI"
> >> in
> >> the subject as a generic term that includes both PCI and PCIe.
> >
> > sure agreed
> >
> >>
> >> On Mon, Jul 27, 2015 at 11:17:03PM +0800, Gabriele Paoloni wrote:
> >> > From: gabriele paoloni <gabriele.paoloni@huawei.com>
> >> >
> >> >     This patch is needed port PCIe designware to new DT parsing
> API
> >> >     As discussed in
> >> >     http://lists.infradead.org/pipermail/linux-arm-kernel/2015-
> >> January/317743.html
> >> >     in designware we have a problem as the PCI addresses in the
> PCIe
> >> controller
> >> >     address space are required in order to perform correct HW
> >> operation.
> >> >
> >> >     In order to solve this problem commit f4c55c5a3 "PCI:
> designware:
> >> >     Program ATU with untranslated address" added code to read the
> >> PCIe
> >>
> >> Conventional reference is 12-char SHA1, like this:
> >>
> >>   f4c55c5a3f7f ("PCI: designware: Program ATU with untranslated
> >> address")
> >
> > Agreed, will change this
> >
> >>
> >> >     controller start address directly from the DT ranges.
> >> >
> >> >     In the new DT parsing API of_pci_get_host_bridge_resources()
> >> hides the
> >> >     DT parser from the host controller drivers, so it is not
> possible
> >> >     for drivers to parse values directly from the DT.
> >> >
> >> >     In http://www.spinics.net/lists/linux-pci/msg42540.html we
> >> already tried
> >> >     to use the new DT parsing API but there is a bug (obviously)
> in
> >> setting
> >> >     the <*>_mod_base addresses
> >> >     Applying this patch we can easily set "<*>_mod_base = win-
> >> >__res.start"
> >>
> >> By itself, this patch adds something.  It would help me understand
> it
> >> if
> >> the *user* of this new something were in the same patch series.
> >
> > the user is: "[PATCH v5 2/5] PCI: designware: Add ARM64 support"
> > I will ask Zhou Wang to include this patch in his patchset
> >
> >
> >>
> >> >     This patch adds a new field in "struct of_pci_range" to store
> the
> >> >     pci bus start address; it fills the field in
> >> of_pci_range_parser_one();
> >> >     in of_pci_get_host_bridge_resources() it retrieves the
> resource
> >> entry
> >> >     after it is created and added to the resource list and uses
> >> >     entry->__res.start to store the pci controller address
> >>
> >> struct of_pci_range is starting to get confusing to non-OF folks
> like
> >> me.
> >> It now contains:
> >>
> >>   u32 pci_space;
> >>   u64 pci_addr;
> >>   u64 cpu_addr;
> >>   u64 bus_addr;
> >>
> >> Can you explain what all these things mean, and maybe even add one-
> line
> >> comments to the structure?
> >
> > sure I can add comments inline in the code
> >
> >>
> >> pci_space: The only uses I see are to determine whether to print
> >> "Prefetch".  I don't see any real functionality that uses this.
> >
> > Looking at the code I agree. it's seems to be used only in powerpc
> > and microblaze to print out.
> > However from my understanding pci_space is the phys.hi field of the
> > ranges property: it defines the properties of the address space
> associated
> > to the PCI address. if you're curious you can find a nice and quick
> to read
> > "guide" in http://devicetree.org/MPC5200:PCI
> >
> >>
> >> pci_addr: I assume this is a PCI bus address, like what you would
> see
> >> if
> >> you put an analyzer on the bus/link.  This address could go in a BAR.
> >
> > Yes, this is the PCI start address of the range: phys.mid + phys.low
> in the
> > guide mentioned above
> >
> >>
> >> cpu_addr: I assume this is a CPU physical address, like what you
> would
> >> see
> >> in /proc/iomem and what you would pass to ioremap().
> >
> > Yes correct
> >
> >>
> >> bus_addr: ?
> >>
> >
> > According to the guide above, this is the address into which the
> pci_address
> > get translated to and that is passed to the root complex. Between the
> root
> > complex and the CPU there can be intermediate translation layers: see
> that to
> > get pci_address we call "of_translate_address"; this will apply all
> the
> > translation layers (ranges in the DT) that it finds till it comes to
> the root
> > node of the DT (thus retrieving the CPU address).
> > Now said that, for designware we need the first translated PCI
> address, that we call
> 
> I think you mean "translated CPU address." The flow of addresses looks
> like this:
> 
> CPU -> CPU bus address -> bus fabric address decoding -> bus address
> -> DW PCI -> PCI address
> 
> This is quite common that an IP block does not see the full address.
> It is unusual that the IP block needs to know its full address on the
> slave side. It is quite common for the master side and the kernel
> deals with that all the time with DMA mapping
> 
> > here bus_addr after Rob Herring suggested the name...honestly I
> cannot think of a
> > different name
> 
> Thinking about this some more, is this really a translation versus
> just a stripping of high bits? Does the DW IP have less than 32-bits
> address? If so, we could express differently and just mask the CPU
> address within the driver instead.

I don?t think we should rely on PCI addresses...what if the intermediate 
translation layer changes the lower significant bits of the "bus address"
to translate into a cpu address?

In that case we're going to program the DW with a wrong address

What I am saying is that the DW driver should not rely on the 
behavior of external HW....what do you think?

> 
> Rob
> --
> To unsubscribe from this list: send the line "unsubscribe linux-pci" in
> the body of a message to majordomo at vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 73+ messages in thread
* RE: [PATCH v6] PCI: Store PCIe bus address in struct of_pci_range
  2015-07-30 13:52               ` Gabriele Paoloni
  (?)
@ 2015-07-30 14:15                 ` Gabriele Paoloni
  -1 siblings, 0 replies; 73+ messages in thread
From: Gabriele Paoloni @ 2015-07-30 14:15 UTC (permalink / raw)
  To: Gabriele Paoloni, Rob Herring
  Cc: Bjorn Helgaas, arnd-r2nGTMty4D4, lorenzo.pieralisi-5wv7dgnIgG8,
	Wangzhou (B),
	robh+dt-DgEjT+Ai2ygdnm+yROfE0A, james.morse-5wv7dgnIgG8,
	Liviu.Dudau-5wv7dgnIgG8, linux-pci-u79uwXL29TY76Z2rM5mHXA,
	linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
	devicetree-u79uwXL29TY76Z2rM5mHXA, Yuanzhichang, Zhudacai,
	zhangjukuo, qiuzhenfa, Liguozhu (Kenneth)



> -----Original Message-----
> From: linux-pci-owner@vger.kernel.org [mailto:linux-pci-
> owner@vger.kernel.org] On Behalf Of Gabriele Paoloni
> Sent: Thursday, July 30, 2015 2:52 PM
> To: Rob Herring
> Cc: Bjorn Helgaas; arnd@arndb.de; lorenzo.pieralisi@arm.com; Wangzhou
> (B); robh+dt@kernel.org; james.morse@arm.com; Liviu.Dudau@arm.com;
> linux-pci@vger.kernel.org; linux-arm-kernel@lists.infradead.org;
> devicetree@vger.kernel.org; Yuanzhichang; Zhudacai; zhangjukuo;
> qiuzhenfa; Liguozhu (Kenneth)
> Subject: RE: [PATCH v6] PCI: Store PCIe bus address in struct
> of_pci_range
> 
> 
> 
> > -----Original Message-----
> > From: linux-pci-owner@vger.kernel.org [mailto:linux-pci-
> > owner@vger.kernel.org] On Behalf Of Rob Herring
> > Sent: Thursday, July 30, 2015 2:43 PM
> > To: Gabriele Paoloni
> > Cc: Bjorn Helgaas; arnd@arndb.de; lorenzo.pieralisi@arm.com; Wangzhou
> > (B); robh+dt@kernel.org; james.morse@arm.com; Liviu.Dudau@arm.com;
> > linux-pci@vger.kernel.org; linux-arm-kernel@lists.infradead.org;
> > devicetree@vger.kernel.org; Yuanzhichang; Zhudacai; zhangjukuo;
> > qiuzhenfa; Liguozhu (Kenneth)
> > Subject: Re: [PATCH v6] PCI: Store PCIe bus address in struct
> > of_pci_range
> >
> > On Wed, Jul 29, 2015 at 2:44 PM, Gabriele Paoloni
> > <gabriele.paoloni@huawei.com> wrote:
> > > Hi Bjorn
> > >
> > > Many Thanks for your reply
> > >
> > > I have commented back inline with resolutions from my side.
> > >
> > > If you're ok with them I'll send it out a new version in the
> > appropriate patchset
> > >
> > > Cheers
> > >
> > > Gab
> > >
> > >> -----Original Message-----
> > >> From: Bjorn Helgaas [mailto:bhelgaas@google.com]
> > >> Sent: Wednesday, July 29, 2015 6:21 PM
> > >> To: Gabriele Paoloni
> > >> Cc: arnd@arndb.de; lorenzo.pieralisi@arm.com; Wangzhou (B);
> > >> robh+dt@kernel.org; james.morse@arm.com; Liviu.Dudau@arm.com;
> linux-
> > >> pci@vger.kernel.org; linux-arm-kernel@lists.infradead.org;
> > >> devicetree@vger.kernel.org; Yuanzhichang; Zhudacai; zhangjukuo;
> > >> qiuzhenfa; Liguozhu (Kenneth)
> > >> Subject: Re: [PATCH v6] PCI: Store PCIe bus address in struct
> > >> of_pci_range
> > >>
> > >> Hi Gabriele,
> > >>
> > >> As far as I can tell, this is not specific to PCIe, so please use
> > "PCI"
> > >> in
> > >> the subject as a generic term that includes both PCI and PCIe.
> > >
> > > sure agreed
> > >
> > >>
> > >> On Mon, Jul 27, 2015 at 11:17:03PM +0800, Gabriele Paoloni wrote:
> > >> > From: gabriele paoloni <gabriele.paoloni@huawei.com>
> > >> >
> > >> >     This patch is needed port PCIe designware to new DT parsing
> > API
> > >> >     As discussed in
> > >> >     http://lists.infradead.org/pipermail/linux-arm-kernel/2015-
> > >> January/317743.html
> > >> >     in designware we have a problem as the PCI addresses in the
> > PCIe
> > >> controller
> > >> >     address space are required in order to perform correct HW
> > >> operation.
> > >> >
> > >> >     In order to solve this problem commit f4c55c5a3 "PCI:
> > designware:
> > >> >     Program ATU with untranslated address" added code to read
> the
> > >> PCIe
> > >>
> > >> Conventional reference is 12-char SHA1, like this:
> > >>
> > >>   f4c55c5a3f7f ("PCI: designware: Program ATU with untranslated
> > >> address")
> > >
> > > Agreed, will change this
> > >
> > >>
> > >> >     controller start address directly from the DT ranges.
> > >> >
> > >> >     In the new DT parsing API of_pci_get_host_bridge_resources()
> > >> hides the
> > >> >     DT parser from the host controller drivers, so it is not
> > possible
> > >> >     for drivers to parse values directly from the DT.
> > >> >
> > >> >     In http://www.spinics.net/lists/linux-pci/msg42540.html we
> > >> already tried
> > >> >     to use the new DT parsing API but there is a bug (obviously)
> > in
> > >> setting
> > >> >     the <*>_mod_base addresses
> > >> >     Applying this patch we can easily set "<*>_mod_base = win-
> > >> >__res.start"
> > >>
> > >> By itself, this patch adds something.  It would help me understand
> > it
> > >> if
> > >> the *user* of this new something were in the same patch series.
> > >
> > > the user is: "[PATCH v5 2/5] PCI: designware: Add ARM64 support"
> > > I will ask Zhou Wang to include this patch in his patchset
> > >
> > >
> > >>
> > >> >     This patch adds a new field in "struct of_pci_range" to
> store
> > the
> > >> >     pci bus start address; it fills the field in
> > >> of_pci_range_parser_one();
> > >> >     in of_pci_get_host_bridge_resources() it retrieves the
> > resource
> > >> entry
> > >> >     after it is created and added to the resource list and uses
> > >> >     entry->__res.start to store the pci controller address
> > >>
> > >> struct of_pci_range is starting to get confusing to non-OF folks
> > like
> > >> me.
> > >> It now contains:
> > >>
> > >>   u32 pci_space;
> > >>   u64 pci_addr;
> > >>   u64 cpu_addr;
> > >>   u64 bus_addr;
> > >>
> > >> Can you explain what all these things mean, and maybe even add
> one-
> > line
> > >> comments to the structure?
> > >
> > > sure I can add comments inline in the code
> > >
> > >>
> > >> pci_space: The only uses I see are to determine whether to print
> > >> "Prefetch".  I don't see any real functionality that uses this.
> > >
> > > Looking at the code I agree. it's seems to be used only in powerpc
> > > and microblaze to print out.
> > > However from my understanding pci_space is the phys.hi field of the
> > > ranges property: it defines the properties of the address space
> > associated
> > > to the PCI address. if you're curious you can find a nice and quick
> > to read
> > > "guide" in http://devicetree.org/MPC5200:PCI
> > >
> > >>
> > >> pci_addr: I assume this is a PCI bus address, like what you would
> > see
> > >> if
> > >> you put an analyzer on the bus/link.  This address could go in a
> BAR.
> > >
> > > Yes, this is the PCI start address of the range: phys.mid +
> phys.low
> > in the
> > > guide mentioned above
> > >
> > >>
> > >> cpu_addr: I assume this is a CPU physical address, like what you
> > would
> > >> see
> > >> in /proc/iomem and what you would pass to ioremap().
> > >
> > > Yes correct
> > >
> > >>
> > >> bus_addr: ?
> > >>
> > >
> > > According to the guide above, this is the address into which the
> > pci_address
> > > get translated to and that is passed to the root complex. Between
> the
> > root
> > > complex and the CPU there can be intermediate translation layers:
> see
> > that to
> > > get pci_address we call "of_translate_address"; this will apply all
> > the
> > > translation layers (ranges in the DT) that it finds till it comes
> to
> > the root
> > > node of the DT (thus retrieving the CPU address).
> > > Now said that, for designware we need the first translated PCI
> > address, that we call
> >
> > I think you mean "translated CPU address." The flow of addresses
> looks
> > like this:
> >
> > CPU -> CPU bus address -> bus fabric address decoding -> bus address
> > -> DW PCI -> PCI address
> >
> > This is quite common that an IP block does not see the full address.
> > It is unusual that the IP block needs to know its full address on the
> > slave side. It is quite common for the master side and the kernel
> > deals with that all the time with DMA mapping
> >
> > > here bus_addr after Rob Herring suggested the name...honestly I
> > cannot think of a
> > > different name
> >
> > Thinking about this some more, is this really a translation versus
> > just a stripping of high bits? Does the DW IP have less than 32-bits
> > address? If so, we could express differently and just mask the CPU
> > address within the driver instead.
> 
> I don’t think we should rely on PCI addresses...what if the
> intermediate
> translation layer changes the lower significant bits of the "bus
> address"
> to translate into a cpu address?

Sorry above I meant "I don’t think we should rely on CPU addresses"

> 
> In that case we're going to program the DW with a wrong address
> 
> What I am saying is that the DW driver should not rely on the
> behavior of external HW....what do you think?
> 
> >
> > Rob
> > --
> > To unsubscribe from this list: send the line "unsubscribe linux-pci"
> in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> \x13��칻\x1c�&�~�&�\x18��+-
> ��ݶ\x17��w��˛���m�b��ir(��\x17��ܨ}���Ơz�&j:+v���\r����zZ+��+zf���h���~����i���
> z�\x1e�w���?����&�)ߢ^[f

^ permalink raw reply	[flat|nested] 73+ messages in thread
* RE: [PATCH v6] PCI: Store PCIe bus address in struct of_pci_range
@ 2015-07-30 14:15                 ` Gabriele Paoloni
  0 siblings, 0 replies; 73+ messages in thread
From: Gabriele Paoloni @ 2015-07-30 14:15 UTC (permalink / raw)
  To: Gabriele Paoloni, Rob Herring
  Cc: Bjorn Helgaas, arnd, lorenzo.pieralisi, Wangzhou (B),
	robh+dt, james.morse, Liviu.Dudau, linux-pci, linux-arm-kernel,
	devicetree, Yuanzhichang, Zhudacai, zhangjukuo, qiuzhenfa,
	Liguozhu (Kenneth)

DQoNCj4gLS0tLS1PcmlnaW5hbCBNZXNzYWdlLS0tLS0NCj4gRnJvbTogbGludXgtcGNpLW93bmVy
QHZnZXIua2VybmVsLm9yZyBbbWFpbHRvOmxpbnV4LXBjaS0NCj4gb3duZXJAdmdlci5rZXJuZWwu
b3JnXSBPbiBCZWhhbGYgT2YgR2FicmllbGUgUGFvbG9uaQ0KPiBTZW50OiBUaHVyc2RheSwgSnVs
eSAzMCwgMjAxNSAyOjUyIFBNDQo+IFRvOiBSb2IgSGVycmluZw0KPiBDYzogQmpvcm4gSGVsZ2Fh
czsgYXJuZEBhcm5kYi5kZTsgbG9yZW56by5waWVyYWxpc2lAYXJtLmNvbTsgV2FuZ3pob3UNCj4g
KEIpOyByb2JoK2R0QGtlcm5lbC5vcmc7IGphbWVzLm1vcnNlQGFybS5jb207IExpdml1LkR1ZGF1
QGFybS5jb207DQo+IGxpbnV4LXBjaUB2Z2VyLmtlcm5lbC5vcmc7IGxpbnV4LWFybS1rZXJuZWxA
bGlzdHMuaW5mcmFkZWFkLm9yZzsNCj4gZGV2aWNldHJlZUB2Z2VyLmtlcm5lbC5vcmc7IFl1YW56
aGljaGFuZzsgWmh1ZGFjYWk7IHpoYW5nanVrdW87DQo+IHFpdXpoZW5mYTsgTGlndW96aHUgKEtl
bm5ldGgpDQo+IFN1YmplY3Q6IFJFOiBbUEFUQ0ggdjZdIFBDSTogU3RvcmUgUENJZSBidXMgYWRk
cmVzcyBpbiBzdHJ1Y3QNCj4gb2ZfcGNpX3JhbmdlDQo+IA0KPiANCj4gDQo+ID4gLS0tLS1Pcmln
aW5hbCBNZXNzYWdlLS0tLS0NCj4gPiBGcm9tOiBsaW51eC1wY2ktb3duZXJAdmdlci5rZXJuZWwu
b3JnIFttYWlsdG86bGludXgtcGNpLQ0KPiA+IG93bmVyQHZnZXIua2VybmVsLm9yZ10gT24gQmVo
YWxmIE9mIFJvYiBIZXJyaW5nDQo+ID4gU2VudDogVGh1cnNkYXksIEp1bHkgMzAsIDIwMTUgMjo0
MyBQTQ0KPiA+IFRvOiBHYWJyaWVsZSBQYW9sb25pDQo+ID4gQ2M6IEJqb3JuIEhlbGdhYXM7IGFy
bmRAYXJuZGIuZGU7IGxvcmVuem8ucGllcmFsaXNpQGFybS5jb207IFdhbmd6aG91DQo+ID4gKEIp
OyByb2JoK2R0QGtlcm5lbC5vcmc7IGphbWVzLm1vcnNlQGFybS5jb207IExpdml1LkR1ZGF1QGFy
bS5jb207DQo+ID4gbGludXgtcGNpQHZnZXIua2VybmVsLm9yZzsgbGludXgtYXJtLWtlcm5lbEBs
aXN0cy5pbmZyYWRlYWQub3JnOw0KPiA+IGRldmljZXRyZWVAdmdlci5rZXJuZWwub3JnOyBZdWFu
emhpY2hhbmc7IFpodWRhY2FpOyB6aGFuZ2p1a3VvOw0KPiA+IHFpdXpoZW5mYTsgTGlndW96aHUg
KEtlbm5ldGgpDQo+ID4gU3ViamVjdDogUmU6IFtQQVRDSCB2Nl0gUENJOiBTdG9yZSBQQ0llIGJ1
cyBhZGRyZXNzIGluIHN0cnVjdA0KPiA+IG9mX3BjaV9yYW5nZQ0KPiA+DQo+ID4gT24gV2VkLCBK
dWwgMjksIDIwMTUgYXQgMjo0NCBQTSwgR2FicmllbGUgUGFvbG9uaQ0KPiA+IDxnYWJyaWVsZS5w
YW9sb25pQGh1YXdlaS5jb20+IHdyb3RlOg0KPiA+ID4gSGkgQmpvcm4NCj4gPiA+DQo+ID4gPiBN
YW55IFRoYW5rcyBmb3IgeW91ciByZXBseQ0KPiA+ID4NCj4gPiA+IEkgaGF2ZSBjb21tZW50ZWQg
YmFjayBpbmxpbmUgd2l0aCByZXNvbHV0aW9ucyBmcm9tIG15IHNpZGUuDQo+ID4gPg0KPiA+ID4g
SWYgeW91J3JlIG9rIHdpdGggdGhlbSBJJ2xsIHNlbmQgaXQgb3V0IGEgbmV3IHZlcnNpb24gaW4g
dGhlDQo+ID4gYXBwcm9wcmlhdGUgcGF0Y2hzZXQNCj4gPiA+DQo+ID4gPiBDaGVlcnMNCj4gPiA+
DQo+ID4gPiBHYWINCj4gPiA+DQo+ID4gPj4gLS0tLS1PcmlnaW5hbCBNZXNzYWdlLS0tLS0NCj4g
PiA+PiBGcm9tOiBCam9ybiBIZWxnYWFzIFttYWlsdG86YmhlbGdhYXNAZ29vZ2xlLmNvbV0NCj4g
PiA+PiBTZW50OiBXZWRuZXNkYXksIEp1bHkgMjksIDIwMTUgNjoyMSBQTQ0KPiA+ID4+IFRvOiBH
YWJyaWVsZSBQYW9sb25pDQo+ID4gPj4gQ2M6IGFybmRAYXJuZGIuZGU7IGxvcmVuem8ucGllcmFs
aXNpQGFybS5jb207IFdhbmd6aG91IChCKTsNCj4gPiA+PiByb2JoK2R0QGtlcm5lbC5vcmc7IGph
bWVzLm1vcnNlQGFybS5jb207IExpdml1LkR1ZGF1QGFybS5jb207DQo+IGxpbnV4LQ0KPiA+ID4+
IHBjaUB2Z2VyLmtlcm5lbC5vcmc7IGxpbnV4LWFybS1rZXJuZWxAbGlzdHMuaW5mcmFkZWFkLm9y
ZzsNCj4gPiA+PiBkZXZpY2V0cmVlQHZnZXIua2VybmVsLm9yZzsgWXVhbnpoaWNoYW5nOyBaaHVk
YWNhaTsgemhhbmdqdWt1bzsNCj4gPiA+PiBxaXV6aGVuZmE7IExpZ3Vvemh1IChLZW5uZXRoKQ0K
PiA+ID4+IFN1YmplY3Q6IFJlOiBbUEFUQ0ggdjZdIFBDSTogU3RvcmUgUENJZSBidXMgYWRkcmVz
cyBpbiBzdHJ1Y3QNCj4gPiA+PiBvZl9wY2lfcmFuZ2UNCj4gPiA+Pg0KPiA+ID4+IEhpIEdhYnJp
ZWxlLA0KPiA+ID4+DQo+ID4gPj4gQXMgZmFyIGFzIEkgY2FuIHRlbGwsIHRoaXMgaXMgbm90IHNw
ZWNpZmljIHRvIFBDSWUsIHNvIHBsZWFzZSB1c2UNCj4gPiAiUENJIg0KPiA+ID4+IGluDQo+ID4g
Pj4gdGhlIHN1YmplY3QgYXMgYSBnZW5lcmljIHRlcm0gdGhhdCBpbmNsdWRlcyBib3RoIFBDSSBh
bmQgUENJZS4NCj4gPiA+DQo+ID4gPiBzdXJlIGFncmVlZA0KPiA+ID4NCj4gPiA+Pg0KPiA+ID4+
IE9uIE1vbiwgSnVsIDI3LCAyMDE1IGF0IDExOjE3OjAzUE0gKzA4MDAsIEdhYnJpZWxlIFBhb2xv
bmkgd3JvdGU6DQo+ID4gPj4gPiBGcm9tOiBnYWJyaWVsZSBwYW9sb25pIDxnYWJyaWVsZS5wYW9s
b25pQGh1YXdlaS5jb20+DQo+ID4gPj4gPg0KPiA+ID4+ID4gICAgIFRoaXMgcGF0Y2ggaXMgbmVl
ZGVkIHBvcnQgUENJZSBkZXNpZ253YXJlIHRvIG5ldyBEVCBwYXJzaW5nDQo+ID4gQVBJDQo+ID4g
Pj4gPiAgICAgQXMgZGlzY3Vzc2VkIGluDQo+ID4gPj4gPiAgICAgaHR0cDovL2xpc3RzLmluZnJh
ZGVhZC5vcmcvcGlwZXJtYWlsL2xpbnV4LWFybS1rZXJuZWwvMjAxNS0NCj4gPiA+PiBKYW51YXJ5
LzMxNzc0My5odG1sDQo+ID4gPj4gPiAgICAgaW4gZGVzaWdud2FyZSB3ZSBoYXZlIGEgcHJvYmxl
bSBhcyB0aGUgUENJIGFkZHJlc3NlcyBpbiB0aGUNCj4gPiBQQ0llDQo+ID4gPj4gY29udHJvbGxl
cg0KPiA+ID4+ID4gICAgIGFkZHJlc3Mgc3BhY2UgYXJlIHJlcXVpcmVkIGluIG9yZGVyIHRvIHBl
cmZvcm0gY29ycmVjdCBIVw0KPiA+ID4+IG9wZXJhdGlvbi4NCj4gPiA+PiA+DQo+ID4gPj4gPiAg
ICAgSW4gb3JkZXIgdG8gc29sdmUgdGhpcyBwcm9ibGVtIGNvbW1pdCBmNGM1NWM1YTMgIlBDSToN
Cj4gPiBkZXNpZ253YXJlOg0KPiA+ID4+ID4gICAgIFByb2dyYW0gQVRVIHdpdGggdW50cmFuc2xh
dGVkIGFkZHJlc3MiIGFkZGVkIGNvZGUgdG8gcmVhZA0KPiB0aGUNCj4gPiA+PiBQQ0llDQo+ID4g
Pj4NCj4gPiA+PiBDb252ZW50aW9uYWwgcmVmZXJlbmNlIGlzIDEyLWNoYXIgU0hBMSwgbGlrZSB0
aGlzOg0KPiA+ID4+DQo+ID4gPj4gICBmNGM1NWM1YTNmN2YgKCJQQ0k6IGRlc2lnbndhcmU6IFBy
b2dyYW0gQVRVIHdpdGggdW50cmFuc2xhdGVkDQo+ID4gPj4gYWRkcmVzcyIpDQo+ID4gPg0KPiA+
ID4gQWdyZWVkLCB3aWxsIGNoYW5nZSB0aGlzDQo+ID4gPg0KPiA+ID4+DQo+ID4gPj4gPiAgICAg
Y29udHJvbGxlciBzdGFydCBhZGRyZXNzIGRpcmVjdGx5IGZyb20gdGhlIERUIHJhbmdlcy4NCj4g
PiA+PiA+DQo+ID4gPj4gPiAgICAgSW4gdGhlIG5ldyBEVCBwYXJzaW5nIEFQSSBvZl9wY2lfZ2V0
X2hvc3RfYnJpZGdlX3Jlc291cmNlcygpDQo+ID4gPj4gaGlkZXMgdGhlDQo+ID4gPj4gPiAgICAg
RFQgcGFyc2VyIGZyb20gdGhlIGhvc3QgY29udHJvbGxlciBkcml2ZXJzLCBzbyBpdCBpcyBub3QN
Cj4gPiBwb3NzaWJsZQ0KPiA+ID4+ID4gICAgIGZvciBkcml2ZXJzIHRvIHBhcnNlIHZhbHVlcyBk
aXJlY3RseSBmcm9tIHRoZSBEVC4NCj4gPiA+PiA+DQo+ID4gPj4gPiAgICAgSW4gaHR0cDovL3d3
dy5zcGluaWNzLm5ldC9saXN0cy9saW51eC1wY2kvbXNnNDI1NDAuaHRtbCB3ZQ0KPiA+ID4+IGFs
cmVhZHkgdHJpZWQNCj4gPiA+PiA+ICAgICB0byB1c2UgdGhlIG5ldyBEVCBwYXJzaW5nIEFQSSBi
dXQgdGhlcmUgaXMgYSBidWcgKG9idmlvdXNseSkNCj4gPiBpbg0KPiA+ID4+IHNldHRpbmcNCj4g
PiA+PiA+ICAgICB0aGUgPCo+X21vZF9iYXNlIGFkZHJlc3Nlcw0KPiA+ID4+ID4gICAgIEFwcGx5
aW5nIHRoaXMgcGF0Y2ggd2UgY2FuIGVhc2lseSBzZXQgIjwqPl9tb2RfYmFzZSA9IHdpbi0NCj4g
PiA+PiA+X19yZXMuc3RhcnQiDQo+ID4gPj4NCj4gPiA+PiBCeSBpdHNlbGYsIHRoaXMgcGF0Y2gg
YWRkcyBzb21ldGhpbmcuICBJdCB3b3VsZCBoZWxwIG1lIHVuZGVyc3RhbmQNCj4gPiBpdA0KPiA+
ID4+IGlmDQo+ID4gPj4gdGhlICp1c2VyKiBvZiB0aGlzIG5ldyBzb21ldGhpbmcgd2VyZSBpbiB0
aGUgc2FtZSBwYXRjaCBzZXJpZXMuDQo+ID4gPg0KPiA+ID4gdGhlIHVzZXIgaXM6ICJbUEFUQ0gg
djUgMi81XSBQQ0k6IGRlc2lnbndhcmU6IEFkZCBBUk02NCBzdXBwb3J0Ig0KPiA+ID4gSSB3aWxs
IGFzayBaaG91IFdhbmcgdG8gaW5jbHVkZSB0aGlzIHBhdGNoIGluIGhpcyBwYXRjaHNldA0KPiA+
ID4NCj4gPiA+DQo+ID4gPj4NCj4gPiA+PiA+ICAgICBUaGlzIHBhdGNoIGFkZHMgYSBuZXcgZmll
bGQgaW4gInN0cnVjdCBvZl9wY2lfcmFuZ2UiIHRvDQo+IHN0b3JlDQo+ID4gdGhlDQo+ID4gPj4g
PiAgICAgcGNpIGJ1cyBzdGFydCBhZGRyZXNzOyBpdCBmaWxscyB0aGUgZmllbGQgaW4NCj4gPiA+
PiBvZl9wY2lfcmFuZ2VfcGFyc2VyX29uZSgpOw0KPiA+ID4+ID4gICAgIGluIG9mX3BjaV9nZXRf
aG9zdF9icmlkZ2VfcmVzb3VyY2VzKCkgaXQgcmV0cmlldmVzIHRoZQ0KPiA+IHJlc291cmNlDQo+
ID4gPj4gZW50cnkNCj4gPiA+PiA+ICAgICBhZnRlciBpdCBpcyBjcmVhdGVkIGFuZCBhZGRlZCB0
byB0aGUgcmVzb3VyY2UgbGlzdCBhbmQgdXNlcw0KPiA+ID4+ID4gICAgIGVudHJ5LT5fX3Jlcy5z
dGFydCB0byBzdG9yZSB0aGUgcGNpIGNvbnRyb2xsZXIgYWRkcmVzcw0KPiA+ID4+DQo+ID4gPj4g
c3RydWN0IG9mX3BjaV9yYW5nZSBpcyBzdGFydGluZyB0byBnZXQgY29uZnVzaW5nIHRvIG5vbi1P
RiBmb2xrcw0KPiA+IGxpa2UNCj4gPiA+PiBtZS4NCj4gPiA+PiBJdCBub3cgY29udGFpbnM6DQo+
ID4gPj4NCj4gPiA+PiAgIHUzMiBwY2lfc3BhY2U7DQo+ID4gPj4gICB1NjQgcGNpX2FkZHI7DQo+
ID4gPj4gICB1NjQgY3B1X2FkZHI7DQo+ID4gPj4gICB1NjQgYnVzX2FkZHI7DQo+ID4gPj4NCj4g
PiA+PiBDYW4geW91IGV4cGxhaW4gd2hhdCBhbGwgdGhlc2UgdGhpbmdzIG1lYW4sIGFuZCBtYXli
ZSBldmVuIGFkZA0KPiBvbmUtDQo+ID4gbGluZQ0KPiA+ID4+IGNvbW1lbnRzIHRvIHRoZSBzdHJ1
Y3R1cmU/DQo+ID4gPg0KPiA+ID4gc3VyZSBJIGNhbiBhZGQgY29tbWVudHMgaW5saW5lIGluIHRo
ZSBjb2RlDQo+ID4gPg0KPiA+ID4+DQo+ID4gPj4gcGNpX3NwYWNlOiBUaGUgb25seSB1c2VzIEkg
c2VlIGFyZSB0byBkZXRlcm1pbmUgd2hldGhlciB0byBwcmludA0KPiA+ID4+ICJQcmVmZXRjaCIu
ICBJIGRvbid0IHNlZSBhbnkgcmVhbCBmdW5jdGlvbmFsaXR5IHRoYXQgdXNlcyB0aGlzLg0KPiA+
ID4NCj4gPiA+IExvb2tpbmcgYXQgdGhlIGNvZGUgSSBhZ3JlZS4gaXQncyBzZWVtcyB0byBiZSB1
c2VkIG9ubHkgaW4gcG93ZXJwYw0KPiA+ID4gYW5kIG1pY3JvYmxhemUgdG8gcHJpbnQgb3V0Lg0K
PiA+ID4gSG93ZXZlciBmcm9tIG15IHVuZGVyc3RhbmRpbmcgcGNpX3NwYWNlIGlzIHRoZSBwaHlz
LmhpIGZpZWxkIG9mIHRoZQ0KPiA+ID4gcmFuZ2VzIHByb3BlcnR5OiBpdCBkZWZpbmVzIHRoZSBw
cm9wZXJ0aWVzIG9mIHRoZSBhZGRyZXNzIHNwYWNlDQo+ID4gYXNzb2NpYXRlZA0KPiA+ID4gdG8g
dGhlIFBDSSBhZGRyZXNzLiBpZiB5b3UncmUgY3VyaW91cyB5b3UgY2FuIGZpbmQgYSBuaWNlIGFu
ZCBxdWljaw0KPiA+IHRvIHJlYWQNCj4gPiA+ICJndWlkZSIgaW4gaHR0cDovL2RldmljZXRyZWUu
b3JnL01QQzUyMDA6UENJDQo+ID4gPg0KPiA+ID4+DQo+ID4gPj4gcGNpX2FkZHI6IEkgYXNzdW1l
IHRoaXMgaXMgYSBQQ0kgYnVzIGFkZHJlc3MsIGxpa2Ugd2hhdCB5b3Ugd291bGQNCj4gPiBzZWUN
Cj4gPiA+PiBpZg0KPiA+ID4+IHlvdSBwdXQgYW4gYW5hbHl6ZXIgb24gdGhlIGJ1cy9saW5rLiAg
VGhpcyBhZGRyZXNzIGNvdWxkIGdvIGluIGENCj4gQkFSLg0KPiA+ID4NCj4gPiA+IFllcywgdGhp
cyBpcyB0aGUgUENJIHN0YXJ0IGFkZHJlc3Mgb2YgdGhlIHJhbmdlOiBwaHlzLm1pZCArDQo+IHBo
eXMubG93DQo+ID4gaW4gdGhlDQo+ID4gPiBndWlkZSBtZW50aW9uZWQgYWJvdmUNCj4gPiA+DQo+
ID4gPj4NCj4gPiA+PiBjcHVfYWRkcjogSSBhc3N1bWUgdGhpcyBpcyBhIENQVSBwaHlzaWNhbCBh
ZGRyZXNzLCBsaWtlIHdoYXQgeW91DQo+ID4gd291bGQNCj4gPiA+PiBzZWUNCj4gPiA+PiBpbiAv
cHJvYy9pb21lbSBhbmQgd2hhdCB5b3Ugd291bGQgcGFzcyB0byBpb3JlbWFwKCkuDQo+ID4gPg0K
PiA+ID4gWWVzIGNvcnJlY3QNCj4gPiA+DQo+ID4gPj4NCj4gPiA+PiBidXNfYWRkcjogPw0KPiA+
ID4+DQo+ID4gPg0KPiA+ID4gQWNjb3JkaW5nIHRvIHRoZSBndWlkZSBhYm92ZSwgdGhpcyBpcyB0
aGUgYWRkcmVzcyBpbnRvIHdoaWNoIHRoZQ0KPiA+IHBjaV9hZGRyZXNzDQo+ID4gPiBnZXQgdHJh
bnNsYXRlZCB0byBhbmQgdGhhdCBpcyBwYXNzZWQgdG8gdGhlIHJvb3QgY29tcGxleC4gQmV0d2Vl
bg0KPiB0aGUNCj4gPiByb290DQo+ID4gPiBjb21wbGV4IGFuZCB0aGUgQ1BVIHRoZXJlIGNhbiBi
ZSBpbnRlcm1lZGlhdGUgdHJhbnNsYXRpb24gbGF5ZXJzOg0KPiBzZWUNCj4gPiB0aGF0IHRvDQo+
ID4gPiBnZXQgcGNpX2FkZHJlc3Mgd2UgY2FsbCAib2ZfdHJhbnNsYXRlX2FkZHJlc3MiOyB0aGlz
IHdpbGwgYXBwbHkgYWxsDQo+ID4gdGhlDQo+ID4gPiB0cmFuc2xhdGlvbiBsYXllcnMgKHJhbmdl
cyBpbiB0aGUgRFQpIHRoYXQgaXQgZmluZHMgdGlsbCBpdCBjb21lcw0KPiB0bw0KPiA+IHRoZSBy
b290DQo+ID4gPiBub2RlIG9mIHRoZSBEVCAodGh1cyByZXRyaWV2aW5nIHRoZSBDUFUgYWRkcmVz
cykuDQo+ID4gPiBOb3cgc2FpZCB0aGF0LCBmb3IgZGVzaWdud2FyZSB3ZSBuZWVkIHRoZSBmaXJz
dCB0cmFuc2xhdGVkIFBDSQ0KPiA+IGFkZHJlc3MsIHRoYXQgd2UgY2FsbA0KPiA+DQo+ID4gSSB0
aGluayB5b3UgbWVhbiAidHJhbnNsYXRlZCBDUFUgYWRkcmVzcy4iIFRoZSBmbG93IG9mIGFkZHJl
c3Nlcw0KPiBsb29rcw0KPiA+IGxpa2UgdGhpczoNCj4gPg0KPiA+IENQVSAtPiBDUFUgYnVzIGFk
ZHJlc3MgLT4gYnVzIGZhYnJpYyBhZGRyZXNzIGRlY29kaW5nIC0+IGJ1cyBhZGRyZXNzDQo+ID4g
LT4gRFcgUENJIC0+IFBDSSBhZGRyZXNzDQo+ID4NCj4gPiBUaGlzIGlzIHF1aXRlIGNvbW1vbiB0
aGF0IGFuIElQIGJsb2NrIGRvZXMgbm90IHNlZSB0aGUgZnVsbCBhZGRyZXNzLg0KPiA+IEl0IGlz
IHVudXN1YWwgdGhhdCB0aGUgSVAgYmxvY2sgbmVlZHMgdG8ga25vdyBpdHMgZnVsbCBhZGRyZXNz
IG9uIHRoZQ0KPiA+IHNsYXZlIHNpZGUuIEl0IGlzIHF1aXRlIGNvbW1vbiBmb3IgdGhlIG1hc3Rl
ciBzaWRlIGFuZCB0aGUga2VybmVsDQo+ID4gZGVhbHMgd2l0aCB0aGF0IGFsbCB0aGUgdGltZSB3
aXRoIERNQSBtYXBwaW5nDQo+ID4NCj4gPiA+IGhlcmUgYnVzX2FkZHIgYWZ0ZXIgUm9iIEhlcnJp
bmcgc3VnZ2VzdGVkIHRoZSBuYW1lLi4uaG9uZXN0bHkgSQ0KPiA+IGNhbm5vdCB0aGluayBvZiBh
DQo+ID4gPiBkaWZmZXJlbnQgbmFtZQ0KPiA+DQo+ID4gVGhpbmtpbmcgYWJvdXQgdGhpcyBzb21l
IG1vcmUsIGlzIHRoaXMgcmVhbGx5IGEgdHJhbnNsYXRpb24gdmVyc3VzDQo+ID4ganVzdCBhIHN0
cmlwcGluZyBvZiBoaWdoIGJpdHM/IERvZXMgdGhlIERXIElQIGhhdmUgbGVzcyB0aGFuIDMyLWJp
dHMNCj4gPiBhZGRyZXNzPyBJZiBzbywgd2UgY291bGQgZXhwcmVzcyBkaWZmZXJlbnRseSBhbmQg
anVzdCBtYXNrIHRoZSBDUFUNCj4gPiBhZGRyZXNzIHdpdGhpbiB0aGUgZHJpdmVyIGluc3RlYWQu
DQo+IA0KPiBJIGRvbuKAmXQgdGhpbmsgd2Ugc2hvdWxkIHJlbHkgb24gUENJIGFkZHJlc3Nlcy4u
LndoYXQgaWYgdGhlDQo+IGludGVybWVkaWF0ZQ0KPiB0cmFuc2xhdGlvbiBsYXllciBjaGFuZ2Vz
IHRoZSBsb3dlciBzaWduaWZpY2FudCBiaXRzIG9mIHRoZSAiYnVzDQo+IGFkZHJlc3MiDQo+IHRv
IHRyYW5zbGF0ZSBpbnRvIGEgY3B1IGFkZHJlc3M/DQoNClNvcnJ5IGFib3ZlIEkgbWVhbnQgIkkg
ZG9u4oCZdCB0aGluayB3ZSBzaG91bGQgcmVseSBvbiBDUFUgYWRkcmVzc2VzIg0KDQo+IA0KPiBJ
biB0aGF0IGNhc2Ugd2UncmUgZ29pbmcgdG8gcHJvZ3JhbSB0aGUgRFcgd2l0aCBhIHdyb25nIGFk
ZHJlc3MNCj4gDQo+IFdoYXQgSSBhbSBzYXlpbmcgaXMgdGhhdCB0aGUgRFcgZHJpdmVyIHNob3Vs
ZCBub3QgcmVseSBvbiB0aGUNCj4gYmVoYXZpb3Igb2YgZXh0ZXJuYWwgSFcuLi4ud2hhdCBkbyB5
b3UgdGhpbms/DQo+IA0KPiA+DQo+ID4gUm9iDQo+ID4gLS0NCj4gPiBUbyB1bnN1YnNjcmliZSBm
cm9tIHRoaXMgbGlzdDogc2VuZCB0aGUgbGluZSAidW5zdWJzY3JpYmUgbGludXgtcGNpIg0KPiBp
bg0KPiA+IHRoZSBib2R5IG9mIGEgbWVzc2FnZSB0byBtYWpvcmRvbW9Admdlci5rZXJuZWwub3Jn
DQo+ID4gTW9yZSBtYWpvcmRvbW8gaW5mbyBhdCAgaHR0cDovL3ZnZXIua2VybmVsLm9yZy9tYWpv
cmRvbW8taW5mby5odG1sDQo+IBPvv73vv73subsc77+9Ju+/vX7vv70m77+9GO+/ve+/vSstDQo+
IO+/ve+/vd22F++/ve+/vXfvv73vv73Lm++/ve+/ve+/vW3vv71i77+977+9aXIo77+977+9F++/
ve+/vdyofe+/ve+/ve+/vcageu+/vSZqOit277+977+977+9De+/ve+/ve+/ve+/vXpaK++/ve+/
vSt6Zu+/ve+/ve+/vWjvv73vv73vv71+77+977+977+977+9ae+/ve+/ve+/vQ0KPiB677+9Hu+/
vXfvv73vv73vv70/77+977+977+977+9Ju+/vSnfohtmDQo=

^ permalink raw reply	[flat|nested] 73+ messages in thread
* [PATCH v6] PCI: Store PCIe bus address in struct of_pci_range
@ 2015-07-30 14:15                 ` Gabriele Paoloni
  0 siblings, 0 replies; 73+ messages in thread
From: Gabriele Paoloni @ 2015-07-30 14:15 UTC (permalink / raw)
  To: linux-arm-kernel



> -----Original Message-----
> From: linux-pci-owner at vger.kernel.org [mailto:linux-pci-
> owner at vger.kernel.org] On Behalf Of Gabriele Paoloni
> Sent: Thursday, July 30, 2015 2:52 PM
> To: Rob Herring
> Cc: Bjorn Helgaas; arnd at arndb.de; lorenzo.pieralisi at arm.com; Wangzhou
> (B); robh+dt at kernel.org; james.morse at arm.com; Liviu.Dudau at arm.com;
> linux-pci at vger.kernel.org; linux-arm-kernel at lists.infradead.org;
> devicetree at vger.kernel.org; Yuanzhichang; Zhudacai; zhangjukuo;
> qiuzhenfa; Liguozhu (Kenneth)
> Subject: RE: [PATCH v6] PCI: Store PCIe bus address in struct
> of_pci_range
> 
> 
> 
> > -----Original Message-----
> > From: linux-pci-owner at vger.kernel.org [mailto:linux-pci-
> > owner at vger.kernel.org] On Behalf Of Rob Herring
> > Sent: Thursday, July 30, 2015 2:43 PM
> > To: Gabriele Paoloni
> > Cc: Bjorn Helgaas; arnd at arndb.de; lorenzo.pieralisi at arm.com; Wangzhou
> > (B); robh+dt at kernel.org; james.morse at arm.com; Liviu.Dudau at arm.com;
> > linux-pci at vger.kernel.org; linux-arm-kernel at lists.infradead.org;
> > devicetree at vger.kernel.org; Yuanzhichang; Zhudacai; zhangjukuo;
> > qiuzhenfa; Liguozhu (Kenneth)
> > Subject: Re: [PATCH v6] PCI: Store PCIe bus address in struct
> > of_pci_range
> >
> > On Wed, Jul 29, 2015 at 2:44 PM, Gabriele Paoloni
> > <gabriele.paoloni@huawei.com> wrote:
> > > Hi Bjorn
> > >
> > > Many Thanks for your reply
> > >
> > > I have commented back inline with resolutions from my side.
> > >
> > > If you're ok with them I'll send it out a new version in the
> > appropriate patchset
> > >
> > > Cheers
> > >
> > > Gab
> > >
> > >> -----Original Message-----
> > >> From: Bjorn Helgaas [mailto:bhelgaas at google.com]
> > >> Sent: Wednesday, July 29, 2015 6:21 PM
> > >> To: Gabriele Paoloni
> > >> Cc: arnd at arndb.de; lorenzo.pieralisi at arm.com; Wangzhou (B);
> > >> robh+dt at kernel.org; james.morse at arm.com; Liviu.Dudau at arm.com;
> linux-
> > >> pci at vger.kernel.org; linux-arm-kernel at lists.infradead.org;
> > >> devicetree at vger.kernel.org; Yuanzhichang; Zhudacai; zhangjukuo;
> > >> qiuzhenfa; Liguozhu (Kenneth)
> > >> Subject: Re: [PATCH v6] PCI: Store PCIe bus address in struct
> > >> of_pci_range
> > >>
> > >> Hi Gabriele,
> > >>
> > >> As far as I can tell, this is not specific to PCIe, so please use
> > "PCI"
> > >> in
> > >> the subject as a generic term that includes both PCI and PCIe.
> > >
> > > sure agreed
> > >
> > >>
> > >> On Mon, Jul 27, 2015 at 11:17:03PM +0800, Gabriele Paoloni wrote:
> > >> > From: gabriele paoloni <gabriele.paoloni@huawei.com>
> > >> >
> > >> >     This patch is needed port PCIe designware to new DT parsing
> > API
> > >> >     As discussed in
> > >> >     http://lists.infradead.org/pipermail/linux-arm-kernel/2015-
> > >> January/317743.html
> > >> >     in designware we have a problem as the PCI addresses in the
> > PCIe
> > >> controller
> > >> >     address space are required in order to perform correct HW
> > >> operation.
> > >> >
> > >> >     In order to solve this problem commit f4c55c5a3 "PCI:
> > designware:
> > >> >     Program ATU with untranslated address" added code to read
> the
> > >> PCIe
> > >>
> > >> Conventional reference is 12-char SHA1, like this:
> > >>
> > >>   f4c55c5a3f7f ("PCI: designware: Program ATU with untranslated
> > >> address")
> > >
> > > Agreed, will change this
> > >
> > >>
> > >> >     controller start address directly from the DT ranges.
> > >> >
> > >> >     In the new DT parsing API of_pci_get_host_bridge_resources()
> > >> hides the
> > >> >     DT parser from the host controller drivers, so it is not
> > possible
> > >> >     for drivers to parse values directly from the DT.
> > >> >
> > >> >     In http://www.spinics.net/lists/linux-pci/msg42540.html we
> > >> already tried
> > >> >     to use the new DT parsing API but there is a bug (obviously)
> > in
> > >> setting
> > >> >     the <*>_mod_base addresses
> > >> >     Applying this patch we can easily set "<*>_mod_base = win-
> > >> >__res.start"
> > >>
> > >> By itself, this patch adds something.  It would help me understand
> > it
> > >> if
> > >> the *user* of this new something were in the same patch series.
> > >
> > > the user is: "[PATCH v5 2/5] PCI: designware: Add ARM64 support"
> > > I will ask Zhou Wang to include this patch in his patchset
> > >
> > >
> > >>
> > >> >     This patch adds a new field in "struct of_pci_range" to
> store
> > the
> > >> >     pci bus start address; it fills the field in
> > >> of_pci_range_parser_one();
> > >> >     in of_pci_get_host_bridge_resources() it retrieves the
> > resource
> > >> entry
> > >> >     after it is created and added to the resource list and uses
> > >> >     entry->__res.start to store the pci controller address
> > >>
> > >> struct of_pci_range is starting to get confusing to non-OF folks
> > like
> > >> me.
> > >> It now contains:
> > >>
> > >>   u32 pci_space;
> > >>   u64 pci_addr;
> > >>   u64 cpu_addr;
> > >>   u64 bus_addr;
> > >>
> > >> Can you explain what all these things mean, and maybe even add
> one-
> > line
> > >> comments to the structure?
> > >
> > > sure I can add comments inline in the code
> > >
> > >>
> > >> pci_space: The only uses I see are to determine whether to print
> > >> "Prefetch".  I don't see any real functionality that uses this.
> > >
> > > Looking at the code I agree. it's seems to be used only in powerpc
> > > and microblaze to print out.
> > > However from my understanding pci_space is the phys.hi field of the
> > > ranges property: it defines the properties of the address space
> > associated
> > > to the PCI address. if you're curious you can find a nice and quick
> > to read
> > > "guide" in http://devicetree.org/MPC5200:PCI
> > >
> > >>
> > >> pci_addr: I assume this is a PCI bus address, like what you would
> > see
> > >> if
> > >> you put an analyzer on the bus/link.  This address could go in a
> BAR.
> > >
> > > Yes, this is the PCI start address of the range: phys.mid +
> phys.low
> > in the
> > > guide mentioned above
> > >
> > >>
> > >> cpu_addr: I assume this is a CPU physical address, like what you
> > would
> > >> see
> > >> in /proc/iomem and what you would pass to ioremap().
> > >
> > > Yes correct
> > >
> > >>
> > >> bus_addr: ?
> > >>
> > >
> > > According to the guide above, this is the address into which the
> > pci_address
> > > get translated to and that is passed to the root complex. Between
> the
> > root
> > > complex and the CPU there can be intermediate translation layers:
> see
> > that to
> > > get pci_address we call "of_translate_address"; this will apply all
> > the
> > > translation layers (ranges in the DT) that it finds till it comes
> to
> > the root
> > > node of the DT (thus retrieving the CPU address).
> > > Now said that, for designware we need the first translated PCI
> > address, that we call
> >
> > I think you mean "translated CPU address." The flow of addresses
> looks
> > like this:
> >
> > CPU -> CPU bus address -> bus fabric address decoding -> bus address
> > -> DW PCI -> PCI address
> >
> > This is quite common that an IP block does not see the full address.
> > It is unusual that the IP block needs to know its full address on the
> > slave side. It is quite common for the master side and the kernel
> > deals with that all the time with DMA mapping
> >
> > > here bus_addr after Rob Herring suggested the name...honestly I
> > cannot think of a
> > > different name
> >
> > Thinking about this some more, is this really a translation versus
> > just a stripping of high bits? Does the DW IP have less than 32-bits
> > address? If so, we could express differently and just mask the CPU
> > address within the driver instead.
> 
> I don?t think we should rely on PCI addresses...what if the
> intermediate
> translation layer changes the lower significant bits of the "bus
> address"
> to translate into a cpu address?

Sorry above I meant "I don?t think we should rely on CPU addresses"

> 
> In that case we're going to program the DW with a wrong address
> 
> What I am saying is that the DW driver should not rely on the
> behavior of external HW....what do you think?
> 
> >
> > Rob
> > --
> > To unsubscribe from this list: send the line "unsubscribe linux-pci"
> in
> > the body of a message to majordomo at vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> \x13???
?&?~?&?\x18??+-
> ???\x17??w??????m?b??ir(??\x17???}????z?&j:+v???
????zZ+??+zf???h???~????i???
> z?
?w????????&?)?^[f

^ permalink raw reply	[flat|nested] 73+ messages in thread
* Re: [PATCH v6] PCI: Store PCIe bus address in struct of_pci_range
  2015-07-30 13:42           ` Rob Herring
@ 2015-07-30 16:06             ` Bjorn Helgaas
  -1 siblings, 0 replies; 73+ messages in thread
From: Bjorn Helgaas @ 2015-07-30 16:06 UTC (permalink / raw)
  To: Rob Herring
  Cc: Gabriele Paoloni, arnd, lorenzo.pieralisi, Wangzhou (B),
	robh+dt, james.morse, Liviu.Dudau, linux-pci, linux-arm-kernel,
	devicetree, Yuanzhichang, Zhudacai, zhangjukuo, qiuzhenfa,
	Liguozhu (Kenneth)

On Thu, Jul 30, 2015 at 08:42:53AM -0500, Rob Herring wrote:
> On Wed, Jul 29, 2015 at 2:44 PM, Gabriele Paoloni
> <gabriele.paoloni@huawei.com> wrote:
> > Hi Bjorn
> >
> > Many Thanks for your reply
> >
> > I have commented back inline with resolutions from my side.
> >
> > If you're ok with them I'll send it out a new version in the appropriate patchset
> >
> > Cheers
> >
> > Gab
> >
> >> -----Original Message-----
> >> From: Bjorn Helgaas [mailto:bhelgaas@google.com]
> >> Sent: Wednesday, July 29, 2015 6:21 PM
> >> To: Gabriele Paoloni
> >> Cc: arnd@arndb.de; lorenzo.pieralisi@arm.com; Wangzhou (B);
> >> robh+dt@kernel.org; james.morse@arm.com; Liviu.Dudau@arm.com; linux-
> >> pci@vger.kernel.org; linux-arm-kernel@lists.infradead.org;
> >> devicetree@vger.kernel.org; Yuanzhichang; Zhudacai; zhangjukuo;
> >> qiuzhenfa; Liguozhu (Kenneth)
> >> Subject: Re: [PATCH v6] PCI: Store PCIe bus address in struct
> >> of_pci_range
> >>
> >> Hi Gabriele,
> >>
> >> As far as I can tell, this is not specific to PCIe, so please use "PCI"
> >> in
> >> the subject as a generic term that includes both PCI and PCIe.
> >
> > sure agreed
> >
> >>
> >> On Mon, Jul 27, 2015 at 11:17:03PM +0800, Gabriele Paoloni wrote:
> >> > From: gabriele paoloni <gabriele.paoloni@huawei.com>
> >> >
> >> >     This patch is needed port PCIe designware to new DT parsing API
> >> >     As discussed in
> >> >     http://lists.infradead.org/pipermail/linux-arm-kernel/2015-
> >> January/317743.html
> >> >     in designware we have a problem as the PCI addresses in the PCIe
> >> controller
> >> >     address space are required in order to perform correct HW
> >> operation.
> >> >
> >> >     In order to solve this problem commit f4c55c5a3 "PCI: designware:
> >> >     Program ATU with untranslated address" added code to read the
> >> PCIe
> >>
> >> Conventional reference is 12-char SHA1, like this:
> >>
> >>   f4c55c5a3f7f ("PCI: designware: Program ATU with untranslated
> >> address")
> >
> > Agreed, will change this
> >
> >>
> >> >     controller start address directly from the DT ranges.
> >> >
> >> >     In the new DT parsing API of_pci_get_host_bridge_resources()
> >> hides the
> >> >     DT parser from the host controller drivers, so it is not possible
> >> >     for drivers to parse values directly from the DT.
> >> >
> >> >     In http://www.spinics.net/lists/linux-pci/msg42540.html we
> >> already tried
> >> >     to use the new DT parsing API but there is a bug (obviously) in
> >> setting
> >> >     the <*>_mod_base addresses
> >> >     Applying this patch we can easily set "<*>_mod_base = win-
> >> >__res.start"
> >>
> >> By itself, this patch adds something.  It would help me understand it
> >> if
> >> the *user* of this new something were in the same patch series.
> >
> > the user is: "[PATCH v5 2/5] PCI: designware: Add ARM64 support"
> > I will ask Zhou Wang to include this patch in his patchset
> >
> >
> >>
> >> >     This patch adds a new field in "struct of_pci_range" to store the
> >> >     pci bus start address; it fills the field in
> >> of_pci_range_parser_one();
> >> >     in of_pci_get_host_bridge_resources() it retrieves the resource
> >> entry
> >> >     after it is created and added to the resource list and uses
> >> >     entry->__res.start to store the pci controller address
> >>
> >> struct of_pci_range is starting to get confusing to non-OF folks like
> >> me.
> >> It now contains:
> >>
> >>   u32 pci_space;
> >>   u64 pci_addr;
> >>   u64 cpu_addr;
> >>   u64 bus_addr;
> >>
> >> Can you explain what all these things mean, and maybe even add one-line
> >> comments to the structure?
> >
> > sure I can add comments inline in the code
> >
> >>
> >> pci_space: The only uses I see are to determine whether to print
> >> "Prefetch".  I don't see any real functionality that uses this.
> >
> > Looking at the code I agree. it's seems to be used only in powerpc
> > and microblaze to print out.
> > However from my understanding pci_space is the phys.hi field of the
> > ranges property: it defines the properties of the address space associated
> > to the PCI address. if you're curious you can find a nice and quick to read
> > "guide" in http://devicetree.org/MPC5200:PCI
> >
> >>
> >> pci_addr: I assume this is a PCI bus address, like what you would see
> >> if
> >> you put an analyzer on the bus/link.  This address could go in a BAR.
> >
> > Yes, this is the PCI start address of the range: phys.mid + phys.low in the
> > guide mentioned above
> >
> >>
> >> cpu_addr: I assume this is a CPU physical address, like what you would
> >> see
> >> in /proc/iomem and what you would pass to ioremap().
> >
> > Yes correct
> >
> >>
> >> bus_addr: ?
> >>
> >
> > According to the guide above, this is the address into which the pci_address
> > get translated to and that is passed to the root complex. Between the root
> > complex and the CPU there can be intermediate translation layers: see that to
> > get pci_address we call "of_translate_address"; this will apply all the
> > translation layers (ranges in the DT) that it finds till it comes to the root
> > node of the DT (thus retrieving the CPU address).
> > Now said that, for designware we need the first translated PCI address, that we call
> 
> I think you mean "translated CPU address." The flow of addresses looks
> like this:
> 
> CPU -> CPU bus address -> bus fabric address decoding -> bus address
> -> DW PCI -> PCI address
> 
> This is quite common that an IP block does not see the full address.
> It is unusual that the IP block needs to know its full address on the
> slave side. It is quite common for the master side and the kernel
> deals with that all the time with DMA mapping
> 
> > here bus_addr after Rob Herring suggested the name...honestly I cannot think of a
> > different name
> 
> Thinking about this some more, is this really a translation versus
> just a stripping of high bits? Does the DW IP have less than 32-bits
> address? If so, we could express differently and just mask the CPU
> address within the driver instead.

I would like this much better if it would be sufficient.

If I understand this correctly, this is all for the MMIO path, i.e., it has
nothing to do with DMA addresses generated by the device being translated
to main memory addresses.

ACPI has a fairly good abstract model for describing the address
translations the kernel and drivers need to know about.  We should assume
we'll need to describe this hardware via ACPI in addition to DT eventually,
so I'm trying to figure out how we would map this into ACPI terms.

If the driver really needs this intermediate address (the "bus address"
above), I think that would mean adding a second ACPI bridge device.  The
first bridge would have a _TRA showing the offset from CPU physical address
to bus address, and the second bridge would have a _TRA showing the offset
from bus address to PCI address.

If you only had a single ACPI bridge device, you'd only have one _TRA (from
CPU physical to PCI bus address), and there'd be no way for the driver to
learn about the intermediate bus address.

^ permalink raw reply	[flat|nested] 73+ messages in thread
* [PATCH v6] PCI: Store PCIe bus address in struct of_pci_range
@ 2015-07-30 16:06             ` Bjorn Helgaas
  0 siblings, 0 replies; 73+ messages in thread
From: Bjorn Helgaas @ 2015-07-30 16:06 UTC (permalink / raw)
  To: linux-arm-kernel

On Thu, Jul 30, 2015 at 08:42:53AM -0500, Rob Herring wrote:
> On Wed, Jul 29, 2015 at 2:44 PM, Gabriele Paoloni
> <gabriele.paoloni@huawei.com> wrote:
> > Hi Bjorn
> >
> > Many Thanks for your reply
> >
> > I have commented back inline with resolutions from my side.
> >
> > If you're ok with them I'll send it out a new version in the appropriate patchset
> >
> > Cheers
> >
> > Gab
> >
> >> -----Original Message-----
> >> From: Bjorn Helgaas [mailto:bhelgaas at google.com]
> >> Sent: Wednesday, July 29, 2015 6:21 PM
> >> To: Gabriele Paoloni
> >> Cc: arnd at arndb.de; lorenzo.pieralisi at arm.com; Wangzhou (B);
> >> robh+dt at kernel.org; james.morse at arm.com; Liviu.Dudau at arm.com; linux-
> >> pci at vger.kernel.org; linux-arm-kernel at lists.infradead.org;
> >> devicetree at vger.kernel.org; Yuanzhichang; Zhudacai; zhangjukuo;
> >> qiuzhenfa; Liguozhu (Kenneth)
> >> Subject: Re: [PATCH v6] PCI: Store PCIe bus address in struct
> >> of_pci_range
> >>
> >> Hi Gabriele,
> >>
> >> As far as I can tell, this is not specific to PCIe, so please use "PCI"
> >> in
> >> the subject as a generic term that includes both PCI and PCIe.
> >
> > sure agreed
> >
> >>
> >> On Mon, Jul 27, 2015 at 11:17:03PM +0800, Gabriele Paoloni wrote:
> >> > From: gabriele paoloni <gabriele.paoloni@huawei.com>
> >> >
> >> >     This patch is needed port PCIe designware to new DT parsing API
> >> >     As discussed in
> >> >     http://lists.infradead.org/pipermail/linux-arm-kernel/2015-
> >> January/317743.html
> >> >     in designware we have a problem as the PCI addresses in the PCIe
> >> controller
> >> >     address space are required in order to perform correct HW
> >> operation.
> >> >
> >> >     In order to solve this problem commit f4c55c5a3 "PCI: designware:
> >> >     Program ATU with untranslated address" added code to read the
> >> PCIe
> >>
> >> Conventional reference is 12-char SHA1, like this:
> >>
> >>   f4c55c5a3f7f ("PCI: designware: Program ATU with untranslated
> >> address")
> >
> > Agreed, will change this
> >
> >>
> >> >     controller start address directly from the DT ranges.
> >> >
> >> >     In the new DT parsing API of_pci_get_host_bridge_resources()
> >> hides the
> >> >     DT parser from the host controller drivers, so it is not possible
> >> >     for drivers to parse values directly from the DT.
> >> >
> >> >     In http://www.spinics.net/lists/linux-pci/msg42540.html we
> >> already tried
> >> >     to use the new DT parsing API but there is a bug (obviously) in
> >> setting
> >> >     the <*>_mod_base addresses
> >> >     Applying this patch we can easily set "<*>_mod_base = win-
> >> >__res.start"
> >>
> >> By itself, this patch adds something.  It would help me understand it
> >> if
> >> the *user* of this new something were in the same patch series.
> >
> > the user is: "[PATCH v5 2/5] PCI: designware: Add ARM64 support"
> > I will ask Zhou Wang to include this patch in his patchset
> >
> >
> >>
> >> >     This patch adds a new field in "struct of_pci_range" to store the
> >> >     pci bus start address; it fills the field in
> >> of_pci_range_parser_one();
> >> >     in of_pci_get_host_bridge_resources() it retrieves the resource
> >> entry
> >> >     after it is created and added to the resource list and uses
> >> >     entry->__res.start to store the pci controller address
> >>
> >> struct of_pci_range is starting to get confusing to non-OF folks like
> >> me.
> >> It now contains:
> >>
> >>   u32 pci_space;
> >>   u64 pci_addr;
> >>   u64 cpu_addr;
> >>   u64 bus_addr;
> >>
> >> Can you explain what all these things mean, and maybe even add one-line
> >> comments to the structure?
> >
> > sure I can add comments inline in the code
> >
> >>
> >> pci_space: The only uses I see are to determine whether to print
> >> "Prefetch".  I don't see any real functionality that uses this.
> >
> > Looking at the code I agree. it's seems to be used only in powerpc
> > and microblaze to print out.
> > However from my understanding pci_space is the phys.hi field of the
> > ranges property: it defines the properties of the address space associated
> > to the PCI address. if you're curious you can find a nice and quick to read
> > "guide" in http://devicetree.org/MPC5200:PCI
> >
> >>
> >> pci_addr: I assume this is a PCI bus address, like what you would see
> >> if
> >> you put an analyzer on the bus/link.  This address could go in a BAR.
> >
> > Yes, this is the PCI start address of the range: phys.mid + phys.low in the
> > guide mentioned above
> >
> >>
> >> cpu_addr: I assume this is a CPU physical address, like what you would
> >> see
> >> in /proc/iomem and what you would pass to ioremap().
> >
> > Yes correct
> >
> >>
> >> bus_addr: ?
> >>
> >
> > According to the guide above, this is the address into which the pci_address
> > get translated to and that is passed to the root complex. Between the root
> > complex and the CPU there can be intermediate translation layers: see that to
> > get pci_address we call "of_translate_address"; this will apply all the
> > translation layers (ranges in the DT) that it finds till it comes to the root
> > node of the DT (thus retrieving the CPU address).
> > Now said that, for designware we need the first translated PCI address, that we call
> 
> I think you mean "translated CPU address." The flow of addresses looks
> like this:
> 
> CPU -> CPU bus address -> bus fabric address decoding -> bus address
> -> DW PCI -> PCI address
> 
> This is quite common that an IP block does not see the full address.
> It is unusual that the IP block needs to know its full address on the
> slave side. It is quite common for the master side and the kernel
> deals with that all the time with DMA mapping
> 
> > here bus_addr after Rob Herring suggested the name...honestly I cannot think of a
> > different name
> 
> Thinking about this some more, is this really a translation versus
> just a stripping of high bits? Does the DW IP have less than 32-bits
> address? If so, we could express differently and just mask the CPU
> address within the driver instead.

I would like this much better if it would be sufficient.

If I understand this correctly, this is all for the MMIO path, i.e., it has
nothing to do with DMA addresses generated by the device being translated
to main memory addresses.

ACPI has a fairly good abstract model for describing the address
translations the kernel and drivers need to know about.  We should assume
we'll need to describe this hardware via ACPI in addition to DT eventually,
so I'm trying to figure out how we would map this into ACPI terms.

If the driver really needs this intermediate address (the "bus address"
above), I think that would mean adding a second ACPI bridge device.  The
first bridge would have a _TRA showing the offset from CPU physical address
to bus address, and the second bridge would have a _TRA showing the offset
from bus address to PCI address.

If you only had a single ACPI bridge device, you'd only have one _TRA (from
CPU physical to PCI bus address), and there'd be no way for the driver to
learn about the intermediate bus address.

^ permalink raw reply	[flat|nested] 73+ messages in thread
* Re: [PATCH v6] PCI: Store PCIe bus address in struct of_pci_range
  2015-07-30 13:52               ` Gabriele Paoloni
@ 2015-07-30 16:14                 ` Bjorn Helgaas
  -1 siblings, 0 replies; 73+ messages in thread
From: Bjorn Helgaas @ 2015-07-30 16:14 UTC (permalink / raw)
  To: Gabriele Paoloni
  Cc: Rob Herring, arnd, lorenzo.pieralisi, Wangzhou (B),
	robh+dt, james.morse, Liviu.Dudau, linux-pci, linux-arm-kernel,
	devicetree, Yuanzhichang, Zhudacai, zhangjukuo, qiuzhenfa,
	Liguozhu (Kenneth)

On Thu, Jul 30, 2015 at 01:52:13PM +0000, Gabriele Paoloni wrote:
> 
> 
> > -----Original Message-----
> > From: linux-pci-owner@vger.kernel.org [mailto:linux-pci-
> > owner@vger.kernel.org] On Behalf Of Rob Herring
> > Sent: Thursday, July 30, 2015 2:43 PM
> > To: Gabriele Paoloni
> > Cc: Bjorn Helgaas; arnd@arndb.de; lorenzo.pieralisi@arm.com; Wangzhou
> > (B); robh+dt@kernel.org; james.morse@arm.com; Liviu.Dudau@arm.com;
> > linux-pci@vger.kernel.org; linux-arm-kernel@lists.infradead.org;
> > devicetree@vger.kernel.org; Yuanzhichang; Zhudacai; zhangjukuo;
> > qiuzhenfa; Liguozhu (Kenneth)
> > Subject: Re: [PATCH v6] PCI: Store PCIe bus address in struct
> > of_pci_range
> > 
> > On Wed, Jul 29, 2015 at 2:44 PM, Gabriele Paoloni
> > <gabriele.paoloni@huawei.com> wrote:
> > > Hi Bjorn
> > >
> > > Many Thanks for your reply
> > >
> > > I have commented back inline with resolutions from my side.
> > >
> > > If you're ok with them I'll send it out a new version in the
> > appropriate patchset
> > >
> > > Cheers
> > >
> > > Gab
> > >
> > >> -----Original Message-----
> > >> From: Bjorn Helgaas [mailto:bhelgaas@google.com]
> > >> Sent: Wednesday, July 29, 2015 6:21 PM
> > >> To: Gabriele Paoloni
> > >> Cc: arnd@arndb.de; lorenzo.pieralisi@arm.com; Wangzhou (B);
> > >> robh+dt@kernel.org; james.morse@arm.com; Liviu.Dudau@arm.com; linux-
> > >> pci@vger.kernel.org; linux-arm-kernel@lists.infradead.org;
> > >> devicetree@vger.kernel.org; Yuanzhichang; Zhudacai; zhangjukuo;
> > >> qiuzhenfa; Liguozhu (Kenneth)
> > >> Subject: Re: [PATCH v6] PCI: Store PCIe bus address in struct
> > >> of_pci_range
> > >>
> > >> Hi Gabriele,
> > >>
> > >> As far as I can tell, this is not specific to PCIe, so please use
> > "PCI"
> > >> in
> > >> the subject as a generic term that includes both PCI and PCIe.
> > >
> > > sure agreed
> > >
> > >>
> > >> On Mon, Jul 27, 2015 at 11:17:03PM +0800, Gabriele Paoloni wrote:
> > >> > From: gabriele paoloni <gabriele.paoloni@huawei.com>
> > >> >
> > >> >     This patch is needed port PCIe designware to new DT parsing
> > API
> > >> >     As discussed in
> > >> >     http://lists.infradead.org/pipermail/linux-arm-kernel/2015-
> > >> January/317743.html
> > >> >     in designware we have a problem as the PCI addresses in the
> > PCIe
> > >> controller
> > >> >     address space are required in order to perform correct HW
> > >> operation.
> > >> >
> > >> >     In order to solve this problem commit f4c55c5a3 "PCI:
> > designware:
> > >> >     Program ATU with untranslated address" added code to read the
> > >> PCIe
> > >>
> > >> Conventional reference is 12-char SHA1, like this:
> > >>
> > >>   f4c55c5a3f7f ("PCI: designware: Program ATU with untranslated
> > >> address")
> > >
> > > Agreed, will change this
> > >
> > >>
> > >> >     controller start address directly from the DT ranges.
> > >> >
> > >> >     In the new DT parsing API of_pci_get_host_bridge_resources()
> > >> hides the
> > >> >     DT parser from the host controller drivers, so it is not
> > possible
> > >> >     for drivers to parse values directly from the DT.
> > >> >
> > >> >     In http://www.spinics.net/lists/linux-pci/msg42540.html we
> > >> already tried
> > >> >     to use the new DT parsing API but there is a bug (obviously)
> > in
> > >> setting
> > >> >     the <*>_mod_base addresses
> > >> >     Applying this patch we can easily set "<*>_mod_base = win-
> > >> >__res.start"
> > >>
> > >> By itself, this patch adds something.  It would help me understand
> > it
> > >> if
> > >> the *user* of this new something were in the same patch series.
> > >
> > > the user is: "[PATCH v5 2/5] PCI: designware: Add ARM64 support"
> > > I will ask Zhou Wang to include this patch in his patchset
> > >
> > >
> > >>
> > >> >     This patch adds a new field in "struct of_pci_range" to store
> > the
> > >> >     pci bus start address; it fills the field in
> > >> of_pci_range_parser_one();
> > >> >     in of_pci_get_host_bridge_resources() it retrieves the
> > resource
> > >> entry
> > >> >     after it is created and added to the resource list and uses
> > >> >     entry->__res.start to store the pci controller address
> > >>
> > >> struct of_pci_range is starting to get confusing to non-OF folks
> > like
> > >> me.
> > >> It now contains:
> > >>
> > >>   u32 pci_space;
> > >>   u64 pci_addr;
> > >>   u64 cpu_addr;
> > >>   u64 bus_addr;
> > >>
> > >> Can you explain what all these things mean, and maybe even add one-
> > line
> > >> comments to the structure?
> > >
> > > sure I can add comments inline in the code
> > >
> > >>
> > >> pci_space: The only uses I see are to determine whether to print
> > >> "Prefetch".  I don't see any real functionality that uses this.
> > >
> > > Looking at the code I agree. it's seems to be used only in powerpc
> > > and microblaze to print out.
> > > However from my understanding pci_space is the phys.hi field of the
> > > ranges property: it defines the properties of the address space
> > associated
> > > to the PCI address. if you're curious you can find a nice and quick
> > to read
> > > "guide" in http://devicetree.org/MPC5200:PCI
> > >
> > >>
> > >> pci_addr: I assume this is a PCI bus address, like what you would
> > see
> > >> if
> > >> you put an analyzer on the bus/link.  This address could go in a BAR.
> > >
> > > Yes, this is the PCI start address of the range: phys.mid + phys.low
> > in the
> > > guide mentioned above
> > >
> > >>
> > >> cpu_addr: I assume this is a CPU physical address, like what you
> > would
> > >> see
> > >> in /proc/iomem and what you would pass to ioremap().
> > >
> > > Yes correct
> > >
> > >>
> > >> bus_addr: ?
> > >>
> > >
> > > According to the guide above, this is the address into which the
> > pci_address
> > > get translated to and that is passed to the root complex. Between the
> > root
> > > complex and the CPU there can be intermediate translation layers: see
> > that to
> > > get pci_address we call "of_translate_address"; this will apply all
> > the
> > > translation layers (ranges in the DT) that it finds till it comes to
> > the root
> > > node of the DT (thus retrieving the CPU address).
> > > Now said that, for designware we need the first translated PCI
> > address, that we call
> > 
> > I think you mean "translated CPU address." The flow of addresses looks
> > like this:
> > 
> > CPU -> CPU bus address -> bus fabric address decoding -> bus address
> > -> DW PCI -> PCI address
> > 
> > This is quite common that an IP block does not see the full address.
> > It is unusual that the IP block needs to know its full address on the
> > slave side. It is quite common for the master side and the kernel
> > deals with that all the time with DMA mapping
> > 
> > > here bus_addr after Rob Herring suggested the name...honestly I
> > cannot think of a
> > > different name
> > 
> > Thinking about this some more, is this really a translation versus
> > just a stripping of high bits? Does the DW IP have less than 32-bits
> > address? If so, we could express differently and just mask the CPU
> > address within the driver instead.
> 
> I don’t think we should rely on [CPU] addresses...what if the intermediate 
> translation layer changes the lower significant bits of the "bus address"
> to translate into a cpu address?

Is it really a possiblity that the lower bits could be changed?

I think we're talking about MMIO here, and a bridge has an MMIO
window.  A window is a range of CPU physical addresses that the
bridge forwards to PCI.  The PCI core assumes that a CPU physical
address range is linearly mapped to PCI bus addresses, i.e., if the
window base is X and maps to PCI address Y, then as long as X+n is
inside the window, it must map to Y+n.

That means the low-order bits (the ones that are the offset into the
window) cannot change.

^ permalink raw reply	[flat|nested] 73+ messages in thread
* [PATCH v6] PCI: Store PCIe bus address in struct of_pci_range
@ 2015-07-30 16:14                 ` Bjorn Helgaas
  0 siblings, 0 replies; 73+ messages in thread
From: Bjorn Helgaas @ 2015-07-30 16:14 UTC (permalink / raw)
  To: linux-arm-kernel

On Thu, Jul 30, 2015 at 01:52:13PM +0000, Gabriele Paoloni wrote:
> 
> 
> > -----Original Message-----
> > From: linux-pci-owner at vger.kernel.org [mailto:linux-pci-
> > owner at vger.kernel.org] On Behalf Of Rob Herring
> > Sent: Thursday, July 30, 2015 2:43 PM
> > To: Gabriele Paoloni
> > Cc: Bjorn Helgaas; arnd at arndb.de; lorenzo.pieralisi at arm.com; Wangzhou
> > (B); robh+dt at kernel.org; james.morse at arm.com; Liviu.Dudau at arm.com;
> > linux-pci at vger.kernel.org; linux-arm-kernel at lists.infradead.org;
> > devicetree at vger.kernel.org; Yuanzhichang; Zhudacai; zhangjukuo;
> > qiuzhenfa; Liguozhu (Kenneth)
> > Subject: Re: [PATCH v6] PCI: Store PCIe bus address in struct
> > of_pci_range
> > 
> > On Wed, Jul 29, 2015 at 2:44 PM, Gabriele Paoloni
> > <gabriele.paoloni@huawei.com> wrote:
> > > Hi Bjorn
> > >
> > > Many Thanks for your reply
> > >
> > > I have commented back inline with resolutions from my side.
> > >
> > > If you're ok with them I'll send it out a new version in the
> > appropriate patchset
> > >
> > > Cheers
> > >
> > > Gab
> > >
> > >> -----Original Message-----
> > >> From: Bjorn Helgaas [mailto:bhelgaas at google.com]
> > >> Sent: Wednesday, July 29, 2015 6:21 PM
> > >> To: Gabriele Paoloni
> > >> Cc: arnd at arndb.de; lorenzo.pieralisi at arm.com; Wangzhou (B);
> > >> robh+dt at kernel.org; james.morse at arm.com; Liviu.Dudau at arm.com; linux-
> > >> pci at vger.kernel.org; linux-arm-kernel at lists.infradead.org;
> > >> devicetree at vger.kernel.org; Yuanzhichang; Zhudacai; zhangjukuo;
> > >> qiuzhenfa; Liguozhu (Kenneth)
> > >> Subject: Re: [PATCH v6] PCI: Store PCIe bus address in struct
> > >> of_pci_range
> > >>
> > >> Hi Gabriele,
> > >>
> > >> As far as I can tell, this is not specific to PCIe, so please use
> > "PCI"
> > >> in
> > >> the subject as a generic term that includes both PCI and PCIe.
> > >
> > > sure agreed
> > >
> > >>
> > >> On Mon, Jul 27, 2015 at 11:17:03PM +0800, Gabriele Paoloni wrote:
> > >> > From: gabriele paoloni <gabriele.paoloni@huawei.com>
> > >> >
> > >> >     This patch is needed port PCIe designware to new DT parsing
> > API
> > >> >     As discussed in
> > >> >     http://lists.infradead.org/pipermail/linux-arm-kernel/2015-
> > >> January/317743.html
> > >> >     in designware we have a problem as the PCI addresses in the
> > PCIe
> > >> controller
> > >> >     address space are required in order to perform correct HW
> > >> operation.
> > >> >
> > >> >     In order to solve this problem commit f4c55c5a3 "PCI:
> > designware:
> > >> >     Program ATU with untranslated address" added code to read the
> > >> PCIe
> > >>
> > >> Conventional reference is 12-char SHA1, like this:
> > >>
> > >>   f4c55c5a3f7f ("PCI: designware: Program ATU with untranslated
> > >> address")
> > >
> > > Agreed, will change this
> > >
> > >>
> > >> >     controller start address directly from the DT ranges.
> > >> >
> > >> >     In the new DT parsing API of_pci_get_host_bridge_resources()
> > >> hides the
> > >> >     DT parser from the host controller drivers, so it is not
> > possible
> > >> >     for drivers to parse values directly from the DT.
> > >> >
> > >> >     In http://www.spinics.net/lists/linux-pci/msg42540.html we
> > >> already tried
> > >> >     to use the new DT parsing API but there is a bug (obviously)
> > in
> > >> setting
> > >> >     the <*>_mod_base addresses
> > >> >     Applying this patch we can easily set "<*>_mod_base = win-
> > >> >__res.start"
> > >>
> > >> By itself, this patch adds something.  It would help me understand
> > it
> > >> if
> > >> the *user* of this new something were in the same patch series.
> > >
> > > the user is: "[PATCH v5 2/5] PCI: designware: Add ARM64 support"
> > > I will ask Zhou Wang to include this patch in his patchset
> > >
> > >
> > >>
> > >> >     This patch adds a new field in "struct of_pci_range" to store
> > the
> > >> >     pci bus start address; it fills the field in
> > >> of_pci_range_parser_one();
> > >> >     in of_pci_get_host_bridge_resources() it retrieves the
> > resource
> > >> entry
> > >> >     after it is created and added to the resource list and uses
> > >> >     entry->__res.start to store the pci controller address
> > >>
> > >> struct of_pci_range is starting to get confusing to non-OF folks
> > like
> > >> me.
> > >> It now contains:
> > >>
> > >>   u32 pci_space;
> > >>   u64 pci_addr;
> > >>   u64 cpu_addr;
> > >>   u64 bus_addr;
> > >>
> > >> Can you explain what all these things mean, and maybe even add one-
> > line
> > >> comments to the structure?
> > >
> > > sure I can add comments inline in the code
> > >
> > >>
> > >> pci_space: The only uses I see are to determine whether to print
> > >> "Prefetch".  I don't see any real functionality that uses this.
> > >
> > > Looking at the code I agree. it's seems to be used only in powerpc
> > > and microblaze to print out.
> > > However from my understanding pci_space is the phys.hi field of the
> > > ranges property: it defines the properties of the address space
> > associated
> > > to the PCI address. if you're curious you can find a nice and quick
> > to read
> > > "guide" in http://devicetree.org/MPC5200:PCI
> > >
> > >>
> > >> pci_addr: I assume this is a PCI bus address, like what you would
> > see
> > >> if
> > >> you put an analyzer on the bus/link.  This address could go in a BAR.
> > >
> > > Yes, this is the PCI start address of the range: phys.mid + phys.low
> > in the
> > > guide mentioned above
> > >
> > >>
> > >> cpu_addr: I assume this is a CPU physical address, like what you
> > would
> > >> see
> > >> in /proc/iomem and what you would pass to ioremap().
> > >
> > > Yes correct
> > >
> > >>
> > >> bus_addr: ?
> > >>
> > >
> > > According to the guide above, this is the address into which the
> > pci_address
> > > get translated to and that is passed to the root complex. Between the
> > root
> > > complex and the CPU there can be intermediate translation layers: see
> > that to
> > > get pci_address we call "of_translate_address"; this will apply all
> > the
> > > translation layers (ranges in the DT) that it finds till it comes to
> > the root
> > > node of the DT (thus retrieving the CPU address).
> > > Now said that, for designware we need the first translated PCI
> > address, that we call
> > 
> > I think you mean "translated CPU address." The flow of addresses looks
> > like this:
> > 
> > CPU -> CPU bus address -> bus fabric address decoding -> bus address
> > -> DW PCI -> PCI address
> > 
> > This is quite common that an IP block does not see the full address.
> > It is unusual that the IP block needs to know its full address on the
> > slave side. It is quite common for the master side and the kernel
> > deals with that all the time with DMA mapping
> > 
> > > here bus_addr after Rob Herring suggested the name...honestly I
> > cannot think of a
> > > different name
> > 
> > Thinking about this some more, is this really a translation versus
> > just a stripping of high bits? Does the DW IP have less than 32-bits
> > address? If so, we could express differently and just mask the CPU
> > address within the driver instead.
> 
> I don?t think we should rely on [CPU] addresses...what if the intermediate 
> translation layer changes the lower significant bits of the "bus address"
> to translate into a cpu address?

Is it really a possiblity that the lower bits could be changed?

I think we're talking about MMIO here, and a bridge has an MMIO
window.  A window is a range of CPU physical addresses that the
bridge forwards to PCI.  The PCI core assumes that a CPU physical
address range is linearly mapped to PCI bus addresses, i.e., if the
window base is X and maps to PCI address Y, then as long as X+n is
inside the window, it must map to Y+n.

That means the low-order bits (the ones that are the offset into the
window) cannot change.

^ permalink raw reply	[flat|nested] 73+ messages in thread
* RE: [PATCH v6] PCI: Store PCIe bus address in struct of_pci_range
  2015-07-30 16:14                 ` Bjorn Helgaas
  (?)
@ 2015-07-30 16:50                   ` Gabriele Paoloni
  -1 siblings, 0 replies; 73+ messages in thread
From: Gabriele Paoloni @ 2015-07-30 16:50 UTC (permalink / raw)
  To: Bjorn Helgaas
  Cc: Rob Herring, arnd, lorenzo.pieralisi, Wangzhou (B),
	robh+dt, james.morse, Liviu.Dudau, linux-pci, linux-arm-kernel,
	devicetree, Yuanzhichang, Zhudacai, zhangjukuo, qiuzhenfa,
	Liguozhu (Kenneth)



> -----Original Message-----
> From: linux-pci-owner@vger.kernel.org [mailto:linux-pci-
> owner@vger.kernel.org] On Behalf Of Bjorn Helgaas
> Sent: Thursday, July 30, 2015 5:15 PM
> To: Gabriele Paoloni
> Cc: Rob Herring; arnd@arndb.de; lorenzo.pieralisi@arm.com; Wangzhou (B);
> robh+dt@kernel.org; james.morse@arm.com; Liviu.Dudau@arm.com; linux-
> pci@vger.kernel.org; linux-arm-kernel@lists.infradead.org;
> devicetree@vger.kernel.org; Yuanzhichang; Zhudacai; zhangjukuo;
> qiuzhenfa; Liguozhu (Kenneth)
> Subject: Re: [PATCH v6] PCI: Store PCIe bus address in struct
> of_pci_range
> 
> On Thu, Jul 30, 2015 at 01:52:13PM +0000, Gabriele Paoloni wrote:
> >
> >
> > > -----Original Message-----
> > > From: linux-pci-owner@vger.kernel.org [mailto:linux-pci-
> > > owner@vger.kernel.org] On Behalf Of Rob Herring
> > > Sent: Thursday, July 30, 2015 2:43 PM
> > > To: Gabriele Paoloni
> > > Cc: Bjorn Helgaas; arnd@arndb.de; lorenzo.pieralisi@arm.com;
> Wangzhou
> > > (B); robh+dt@kernel.org; james.morse@arm.com; Liviu.Dudau@arm.com;
> > > linux-pci@vger.kernel.org; linux-arm-kernel@lists.infradead.org;
> > > devicetree@vger.kernel.org; Yuanzhichang; Zhudacai; zhangjukuo;
> > > qiuzhenfa; Liguozhu (Kenneth)
> > > Subject: Re: [PATCH v6] PCI: Store PCIe bus address in struct
> > > of_pci_range
> > >
> > > On Wed, Jul 29, 2015 at 2:44 PM, Gabriele Paoloni
> > > <gabriele.paoloni@huawei.com> wrote:
> > > > Hi Bjorn
> > > >
> > > > Many Thanks for your reply
> > > >
> > > > I have commented back inline with resolutions from my side.
> > > >
> > > > If you're ok with them I'll send it out a new version in the
> > > appropriate patchset
> > > >
> > > > Cheers
> > > >
> > > > Gab
> > > >
> > > >> -----Original Message-----
> > > >> From: Bjorn Helgaas [mailto:bhelgaas@google.com]
> > > >> Sent: Wednesday, July 29, 2015 6:21 PM
> > > >> To: Gabriele Paoloni
> > > >> Cc: arnd@arndb.de; lorenzo.pieralisi@arm.com; Wangzhou (B);
> > > >> robh+dt@kernel.org; james.morse@arm.com; Liviu.Dudau@arm.com;
> linux-
> > > >> pci@vger.kernel.org; linux-arm-kernel@lists.infradead.org;
> > > >> devicetree@vger.kernel.org; Yuanzhichang; Zhudacai; zhangjukuo;
> > > >> qiuzhenfa; Liguozhu (Kenneth)
> > > >> Subject: Re: [PATCH v6] PCI: Store PCIe bus address in struct
> > > >> of_pci_range
> > > >>
> > > >> Hi Gabriele,
> > > >>
> > > >> As far as I can tell, this is not specific to PCIe, so please
> use
> > > "PCI"
> > > >> in
> > > >> the subject as a generic term that includes both PCI and PCIe.
> > > >
> > > > sure agreed
> > > >
> > > >>
> > > >> On Mon, Jul 27, 2015 at 11:17:03PM +0800, Gabriele Paoloni wrote:
> > > >> > From: gabriele paoloni <gabriele.paoloni@huawei.com>
> > > >> >
> > > >> >     This patch is needed port PCIe designware to new DT
> parsing
> > > API
> > > >> >     As discussed in
> > > >> >     http://lists.infradead.org/pipermail/linux-arm-
> kernel/2015-
> > > >> January/317743.html
> > > >> >     in designware we have a problem as the PCI addresses in
> the
> > > PCIe
> > > >> controller
> > > >> >     address space are required in order to perform correct HW
> > > >> operation.
> > > >> >
> > > >> >     In order to solve this problem commit f4c55c5a3 "PCI:
> > > designware:
> > > >> >     Program ATU with untranslated address" added code to read
> the
> > > >> PCIe
> > > >>
> > > >> Conventional reference is 12-char SHA1, like this:
> > > >>
> > > >>   f4c55c5a3f7f ("PCI: designware: Program ATU with untranslated
> > > >> address")
> > > >
> > > > Agreed, will change this
> > > >
> > > >>
> > > >> >     controller start address directly from the DT ranges.
> > > >> >
> > > >> >     In the new DT parsing API
> of_pci_get_host_bridge_resources()
> > > >> hides the
> > > >> >     DT parser from the host controller drivers, so it is not
> > > possible
> > > >> >     for drivers to parse values directly from the DT.
> > > >> >
> > > >> >     In http://www.spinics.net/lists/linux-pci/msg42540.html we
> > > >> already tried
> > > >> >     to use the new DT parsing API but there is a bug
> (obviously)
> > > in
> > > >> setting
> > > >> >     the <*>_mod_base addresses
> > > >> >     Applying this patch we can easily set "<*>_mod_base = win-
> > > >> >__res.start"
> > > >>
> > > >> By itself, this patch adds something.  It would help me
> understand
> > > it
> > > >> if
> > > >> the *user* of this new something were in the same patch series.
> > > >
> > > > the user is: "[PATCH v5 2/5] PCI: designware: Add ARM64 support"
> > > > I will ask Zhou Wang to include this patch in his patchset
> > > >
> > > >
> > > >>
> > > >> >     This patch adds a new field in "struct of_pci_range" to
> store
> > > the
> > > >> >     pci bus start address; it fills the field in
> > > >> of_pci_range_parser_one();
> > > >> >     in of_pci_get_host_bridge_resources() it retrieves the
> > > resource
> > > >> entry
> > > >> >     after it is created and added to the resource list and
> uses
> > > >> >     entry->__res.start to store the pci controller address
> > > >>
> > > >> struct of_pci_range is starting to get confusing to non-OF folks
> > > like
> > > >> me.
> > > >> It now contains:
> > > >>
> > > >>   u32 pci_space;
> > > >>   u64 pci_addr;
> > > >>   u64 cpu_addr;
> > > >>   u64 bus_addr;
> > > >>
> > > >> Can you explain what all these things mean, and maybe even add
> one-
> > > line
> > > >> comments to the structure?
> > > >
> > > > sure I can add comments inline in the code
> > > >
> > > >>
> > > >> pci_space: The only uses I see are to determine whether to print
> > > >> "Prefetch".  I don't see any real functionality that uses this.
> > > >
> > > > Looking at the code I agree. it's seems to be used only in
> powerpc
> > > > and microblaze to print out.
> > > > However from my understanding pci_space is the phys.hi field of
> the
> > > > ranges property: it defines the properties of the address space
> > > associated
> > > > to the PCI address. if you're curious you can find a nice and
> quick
> > > to read
> > > > "guide" in http://devicetree.org/MPC5200:PCI
> > > >
> > > >>
> > > >> pci_addr: I assume this is a PCI bus address, like what you
> would
> > > see
> > > >> if
> > > >> you put an analyzer on the bus/link.  This address could go in a
> BAR.
> > > >
> > > > Yes, this is the PCI start address of the range: phys.mid +
> phys.low
> > > in the
> > > > guide mentioned above
> > > >
> > > >>
> > > >> cpu_addr: I assume this is a CPU physical address, like what you
> > > would
> > > >> see
> > > >> in /proc/iomem and what you would pass to ioremap().
> > > >
> > > > Yes correct
> > > >
> > > >>
> > > >> bus_addr: ?
> > > >>
> > > >
> > > > According to the guide above, this is the address into which the
> > > pci_address
> > > > get translated to and that is passed to the root complex. Between
> the
> > > root
> > > > complex and the CPU there can be intermediate translation layers:
> see
> > > that to
> > > > get pci_address we call "of_translate_address"; this will apply
> all
> > > the
> > > > translation layers (ranges in the DT) that it finds till it comes
> to
> > > the root
> > > > node of the DT (thus retrieving the CPU address).
> > > > Now said that, for designware we need the first translated PCI
> > > address, that we call
> > >
> > > I think you mean "translated CPU address." The flow of addresses
> looks
> > > like this:
> > >
> > > CPU -> CPU bus address -> bus fabric address decoding -> bus
> address
> > > -> DW PCI -> PCI address
> > >
> > > This is quite common that an IP block does not see the full address.
> > > It is unusual that the IP block needs to know its full address on
> the
> > > slave side. It is quite common for the master side and the kernel
> > > deals with that all the time with DMA mapping
> > >
> > > > here bus_addr after Rob Herring suggested the name...honestly I
> > > cannot think of a
> > > > different name
> > >
> > > Thinking about this some more, is this really a translation versus
> > > just a stripping of high bits? Does the DW IP have less than 32-
> bits
> > > address? If so, we could express differently and just mask the CPU
> > > address within the driver instead.
> >
> > I don’t think we should rely on [CPU] addresses...what if the
> intermediate
> > translation layer changes the lower significant bits of the "bus
> address"
> > to translate into a cpu address?
> 
> Is it really a possiblity that the lower bits could be changed?

I've checked all the current deignware users DTs except "pci-layerscape" 
that I could not find:
spear1310.dtsi
spear1340.dtsi
dra7.dtsi
imx6qdl.dtsi
imx6sx.dtsi
keystone.dtsi
exynos5440.dtsi

None of them modifies the lower bits. To be more precise the only guy
that provides another translation layer is "dra7.dtsi":
axi0
http://lxr.free-electrons.com/source/arch/arm/boot/dts/dra7.dtsi#L207

axi1
http://lxr.free-electrons.com/source/arch/arm/boot/dts/dra7.dtsi#L241

For this case masking the top 4bits (bits28 to 31) should make the job.

Speaking in general terms so far I've always seen linear mappings
that differ by bitmask offset, however linear does not mean that you 
cannot affect the lower bits: e.g. <0x0> to <0x0 + size> can map to
<0x0000cafe to 0x0000cafe + size>, but I guess that for HW design reasons
it is much easier to remap directly using a bitmask...but I was not sure
and I didn't think about the problems that can arise with ACPI.

If you think the bitmask is Ok then I can directly define it in
designware and we can drop this patch.

Thanks
Gab
 
> 
> I think we're talking about MMIO here, and a bridge has an MMIO
> window.  A window is a range of CPU physical addresses that the
> bridge forwards to PCI.  The PCI core assumes that a CPU physical
> address range is linearly mapped to PCI bus addresses, i.e., if the
> window base is X and maps to PCI address Y, then as long as X+n is
> inside the window, it must map to Y+n.
> 
> That means the low-order bits (the ones that are the offset into the
> window) cannot change.
> --
> To unsubscribe from this list: send the line "unsubscribe linux-pci" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 73+ messages in thread
* RE: [PATCH v6] PCI: Store PCIe bus address in struct of_pci_range
@ 2015-07-30 16:50                   ` Gabriele Paoloni
  0 siblings, 0 replies; 73+ messages in thread
From: Gabriele Paoloni @ 2015-07-30 16:50 UTC (permalink / raw)
  To: Bjorn Helgaas
  Cc: Rob Herring, arnd, lorenzo.pieralisi, Wangzhou (B),
	robh+dt, james.morse, Liviu.Dudau, linux-pci, linux-arm-kernel,
	devicetree, Yuanzhichang, Zhudacai, zhangjukuo, qiuzhenfa,
	Liguozhu (Kenneth)

DQoNCj4gLS0tLS1PcmlnaW5hbCBNZXNzYWdlLS0tLS0NCj4gRnJvbTogbGludXgtcGNpLW93bmVy
QHZnZXIua2VybmVsLm9yZyBbbWFpbHRvOmxpbnV4LXBjaS0NCj4gb3duZXJAdmdlci5rZXJuZWwu
b3JnXSBPbiBCZWhhbGYgT2YgQmpvcm4gSGVsZ2Fhcw0KPiBTZW50OiBUaHVyc2RheSwgSnVseSAz
MCwgMjAxNSA1OjE1IFBNDQo+IFRvOiBHYWJyaWVsZSBQYW9sb25pDQo+IENjOiBSb2IgSGVycmlu
ZzsgYXJuZEBhcm5kYi5kZTsgbG9yZW56by5waWVyYWxpc2lAYXJtLmNvbTsgV2FuZ3pob3UgKEIp
Ow0KPiByb2JoK2R0QGtlcm5lbC5vcmc7IGphbWVzLm1vcnNlQGFybS5jb207IExpdml1LkR1ZGF1
QGFybS5jb207IGxpbnV4LQ0KPiBwY2lAdmdlci5rZXJuZWwub3JnOyBsaW51eC1hcm0ta2VybmVs
QGxpc3RzLmluZnJhZGVhZC5vcmc7DQo+IGRldmljZXRyZWVAdmdlci5rZXJuZWwub3JnOyBZdWFu
emhpY2hhbmc7IFpodWRhY2FpOyB6aGFuZ2p1a3VvOw0KPiBxaXV6aGVuZmE7IExpZ3Vvemh1IChL
ZW5uZXRoKQ0KPiBTdWJqZWN0OiBSZTogW1BBVENIIHY2XSBQQ0k6IFN0b3JlIFBDSWUgYnVzIGFk
ZHJlc3MgaW4gc3RydWN0DQo+IG9mX3BjaV9yYW5nZQ0KPiANCj4gT24gVGh1LCBKdWwgMzAsIDIw
MTUgYXQgMDE6NTI6MTNQTSArMDAwMCwgR2FicmllbGUgUGFvbG9uaSB3cm90ZToNCj4gPg0KPiA+
DQo+ID4gPiAtLS0tLU9yaWdpbmFsIE1lc3NhZ2UtLS0tLQ0KPiA+ID4gRnJvbTogbGludXgtcGNp
LW93bmVyQHZnZXIua2VybmVsLm9yZyBbbWFpbHRvOmxpbnV4LXBjaS0NCj4gPiA+IG93bmVyQHZn
ZXIua2VybmVsLm9yZ10gT24gQmVoYWxmIE9mIFJvYiBIZXJyaW5nDQo+ID4gPiBTZW50OiBUaHVy
c2RheSwgSnVseSAzMCwgMjAxNSAyOjQzIFBNDQo+ID4gPiBUbzogR2FicmllbGUgUGFvbG9uaQ0K
PiA+ID4gQ2M6IEJqb3JuIEhlbGdhYXM7IGFybmRAYXJuZGIuZGU7IGxvcmVuem8ucGllcmFsaXNp
QGFybS5jb207DQo+IFdhbmd6aG91DQo+ID4gPiAoQik7IHJvYmgrZHRAa2VybmVsLm9yZzsgamFt
ZXMubW9yc2VAYXJtLmNvbTsgTGl2aXUuRHVkYXVAYXJtLmNvbTsNCj4gPiA+IGxpbnV4LXBjaUB2
Z2VyLmtlcm5lbC5vcmc7IGxpbnV4LWFybS1rZXJuZWxAbGlzdHMuaW5mcmFkZWFkLm9yZzsNCj4g
PiA+IGRldmljZXRyZWVAdmdlci5rZXJuZWwub3JnOyBZdWFuemhpY2hhbmc7IFpodWRhY2FpOyB6
aGFuZ2p1a3VvOw0KPiA+ID4gcWl1emhlbmZhOyBMaWd1b3podSAoS2VubmV0aCkNCj4gPiA+IFN1
YmplY3Q6IFJlOiBbUEFUQ0ggdjZdIFBDSTogU3RvcmUgUENJZSBidXMgYWRkcmVzcyBpbiBzdHJ1
Y3QNCj4gPiA+IG9mX3BjaV9yYW5nZQ0KPiA+ID4NCj4gPiA+IE9uIFdlZCwgSnVsIDI5LCAyMDE1
IGF0IDI6NDQgUE0sIEdhYnJpZWxlIFBhb2xvbmkNCj4gPiA+IDxnYWJyaWVsZS5wYW9sb25pQGh1
YXdlaS5jb20+IHdyb3RlOg0KPiA+ID4gPiBIaSBCam9ybg0KPiA+ID4gPg0KPiA+ID4gPiBNYW55
IFRoYW5rcyBmb3IgeW91ciByZXBseQ0KPiA+ID4gPg0KPiA+ID4gPiBJIGhhdmUgY29tbWVudGVk
IGJhY2sgaW5saW5lIHdpdGggcmVzb2x1dGlvbnMgZnJvbSBteSBzaWRlLg0KPiA+ID4gPg0KPiA+
ID4gPiBJZiB5b3UncmUgb2sgd2l0aCB0aGVtIEknbGwgc2VuZCBpdCBvdXQgYSBuZXcgdmVyc2lv
biBpbiB0aGUNCj4gPiA+IGFwcHJvcHJpYXRlIHBhdGNoc2V0DQo+ID4gPiA+DQo+ID4gPiA+IENo
ZWVycw0KPiA+ID4gPg0KPiA+ID4gPiBHYWINCj4gPiA+ID4NCj4gPiA+ID4+IC0tLS0tT3JpZ2lu
YWwgTWVzc2FnZS0tLS0tDQo+ID4gPiA+PiBGcm9tOiBCam9ybiBIZWxnYWFzIFttYWlsdG86Ymhl
bGdhYXNAZ29vZ2xlLmNvbV0NCj4gPiA+ID4+IFNlbnQ6IFdlZG5lc2RheSwgSnVseSAyOSwgMjAx
NSA2OjIxIFBNDQo+ID4gPiA+PiBUbzogR2FicmllbGUgUGFvbG9uaQ0KPiA+ID4gPj4gQ2M6IGFy
bmRAYXJuZGIuZGU7IGxvcmVuem8ucGllcmFsaXNpQGFybS5jb207IFdhbmd6aG91IChCKTsNCj4g
PiA+ID4+IHJvYmgrZHRAa2VybmVsLm9yZzsgamFtZXMubW9yc2VAYXJtLmNvbTsgTGl2aXUuRHVk
YXVAYXJtLmNvbTsNCj4gbGludXgtDQo+ID4gPiA+PiBwY2lAdmdlci5rZXJuZWwub3JnOyBsaW51
eC1hcm0ta2VybmVsQGxpc3RzLmluZnJhZGVhZC5vcmc7DQo+ID4gPiA+PiBkZXZpY2V0cmVlQHZn
ZXIua2VybmVsLm9yZzsgWXVhbnpoaWNoYW5nOyBaaHVkYWNhaTsgemhhbmdqdWt1bzsNCj4gPiA+
ID4+IHFpdXpoZW5mYTsgTGlndW96aHUgKEtlbm5ldGgpDQo+ID4gPiA+PiBTdWJqZWN0OiBSZTog
W1BBVENIIHY2XSBQQ0k6IFN0b3JlIFBDSWUgYnVzIGFkZHJlc3MgaW4gc3RydWN0DQo+ID4gPiA+
PiBvZl9wY2lfcmFuZ2UNCj4gPiA+ID4+DQo+ID4gPiA+PiBIaSBHYWJyaWVsZSwNCj4gPiA+ID4+
DQo+ID4gPiA+PiBBcyBmYXIgYXMgSSBjYW4gdGVsbCwgdGhpcyBpcyBub3Qgc3BlY2lmaWMgdG8g
UENJZSwgc28gcGxlYXNlDQo+IHVzZQ0KPiA+ID4gIlBDSSINCj4gPiA+ID4+IGluDQo+ID4gPiA+
PiB0aGUgc3ViamVjdCBhcyBhIGdlbmVyaWMgdGVybSB0aGF0IGluY2x1ZGVzIGJvdGggUENJIGFu
ZCBQQ0llLg0KPiA+ID4gPg0KPiA+ID4gPiBzdXJlIGFncmVlZA0KPiA+ID4gPg0KPiA+ID4gPj4N
Cj4gPiA+ID4+IE9uIE1vbiwgSnVsIDI3LCAyMDE1IGF0IDExOjE3OjAzUE0gKzA4MDAsIEdhYnJp
ZWxlIFBhb2xvbmkgd3JvdGU6DQo+ID4gPiA+PiA+IEZyb206IGdhYnJpZWxlIHBhb2xvbmkgPGdh
YnJpZWxlLnBhb2xvbmlAaHVhd2VpLmNvbT4NCj4gPiA+ID4+ID4NCj4gPiA+ID4+ID4gICAgIFRo
aXMgcGF0Y2ggaXMgbmVlZGVkIHBvcnQgUENJZSBkZXNpZ253YXJlIHRvIG5ldyBEVA0KPiBwYXJz
aW5nDQo+ID4gPiBBUEkNCj4gPiA+ID4+ID4gICAgIEFzIGRpc2N1c3NlZCBpbg0KPiA+ID4gPj4g
PiAgICAgaHR0cDovL2xpc3RzLmluZnJhZGVhZC5vcmcvcGlwZXJtYWlsL2xpbnV4LWFybS0NCj4g
a2VybmVsLzIwMTUtDQo+ID4gPiA+PiBKYW51YXJ5LzMxNzc0My5odG1sDQo+ID4gPiA+PiA+ICAg
ICBpbiBkZXNpZ253YXJlIHdlIGhhdmUgYSBwcm9ibGVtIGFzIHRoZSBQQ0kgYWRkcmVzc2VzIGlu
DQo+IHRoZQ0KPiA+ID4gUENJZQ0KPiA+ID4gPj4gY29udHJvbGxlcg0KPiA+ID4gPj4gPiAgICAg
YWRkcmVzcyBzcGFjZSBhcmUgcmVxdWlyZWQgaW4gb3JkZXIgdG8gcGVyZm9ybSBjb3JyZWN0IEhX
DQo+ID4gPiA+PiBvcGVyYXRpb24uDQo+ID4gPiA+PiA+DQo+ID4gPiA+PiA+ICAgICBJbiBvcmRl
ciB0byBzb2x2ZSB0aGlzIHByb2JsZW0gY29tbWl0IGY0YzU1YzVhMyAiUENJOg0KPiA+ID4gZGVz
aWdud2FyZToNCj4gPiA+ID4+ID4gICAgIFByb2dyYW0gQVRVIHdpdGggdW50cmFuc2xhdGVkIGFk
ZHJlc3MiIGFkZGVkIGNvZGUgdG8gcmVhZA0KPiB0aGUNCj4gPiA+ID4+IFBDSWUNCj4gPiA+ID4+
DQo+ID4gPiA+PiBDb252ZW50aW9uYWwgcmVmZXJlbmNlIGlzIDEyLWNoYXIgU0hBMSwgbGlrZSB0
aGlzOg0KPiA+ID4gPj4NCj4gPiA+ID4+ICAgZjRjNTVjNWEzZjdmICgiUENJOiBkZXNpZ253YXJl
OiBQcm9ncmFtIEFUVSB3aXRoIHVudHJhbnNsYXRlZA0KPiA+ID4gPj4gYWRkcmVzcyIpDQo+ID4g
PiA+DQo+ID4gPiA+IEFncmVlZCwgd2lsbCBjaGFuZ2UgdGhpcw0KPiA+ID4gPg0KPiA+ID4gPj4N
Cj4gPiA+ID4+ID4gICAgIGNvbnRyb2xsZXIgc3RhcnQgYWRkcmVzcyBkaXJlY3RseSBmcm9tIHRo
ZSBEVCByYW5nZXMuDQo+ID4gPiA+PiA+DQo+ID4gPiA+PiA+ICAgICBJbiB0aGUgbmV3IERUIHBh
cnNpbmcgQVBJDQo+IG9mX3BjaV9nZXRfaG9zdF9icmlkZ2VfcmVzb3VyY2VzKCkNCj4gPiA+ID4+
IGhpZGVzIHRoZQ0KPiA+ID4gPj4gPiAgICAgRFQgcGFyc2VyIGZyb20gdGhlIGhvc3QgY29udHJv
bGxlciBkcml2ZXJzLCBzbyBpdCBpcyBub3QNCj4gPiA+IHBvc3NpYmxlDQo+ID4gPiA+PiA+ICAg
ICBmb3IgZHJpdmVycyB0byBwYXJzZSB2YWx1ZXMgZGlyZWN0bHkgZnJvbSB0aGUgRFQuDQo+ID4g
PiA+PiA+DQo+ID4gPiA+PiA+ICAgICBJbiBodHRwOi8vd3d3LnNwaW5pY3MubmV0L2xpc3RzL2xp
bnV4LXBjaS9tc2c0MjU0MC5odG1sIHdlDQo+ID4gPiA+PiBhbHJlYWR5IHRyaWVkDQo+ID4gPiA+
PiA+ICAgICB0byB1c2UgdGhlIG5ldyBEVCBwYXJzaW5nIEFQSSBidXQgdGhlcmUgaXMgYSBidWcN
Cj4gKG9idmlvdXNseSkNCj4gPiA+IGluDQo+ID4gPiA+PiBzZXR0aW5nDQo+ID4gPiA+PiA+ICAg
ICB0aGUgPCo+X21vZF9iYXNlIGFkZHJlc3Nlcw0KPiA+ID4gPj4gPiAgICAgQXBwbHlpbmcgdGhp
cyBwYXRjaCB3ZSBjYW4gZWFzaWx5IHNldCAiPCo+X21vZF9iYXNlID0gd2luLQ0KPiA+ID4gPj4g
Pl9fcmVzLnN0YXJ0Ig0KPiA+ID4gPj4NCj4gPiA+ID4+IEJ5IGl0c2VsZiwgdGhpcyBwYXRjaCBh
ZGRzIHNvbWV0aGluZy4gIEl0IHdvdWxkIGhlbHAgbWUNCj4gdW5kZXJzdGFuZA0KPiA+ID4gaXQN
Cj4gPiA+ID4+IGlmDQo+ID4gPiA+PiB0aGUgKnVzZXIqIG9mIHRoaXMgbmV3IHNvbWV0aGluZyB3
ZXJlIGluIHRoZSBzYW1lIHBhdGNoIHNlcmllcy4NCj4gPiA+ID4NCj4gPiA+ID4gdGhlIHVzZXIg
aXM6ICJbUEFUQ0ggdjUgMi81XSBQQ0k6IGRlc2lnbndhcmU6IEFkZCBBUk02NCBzdXBwb3J0Ig0K
PiA+ID4gPiBJIHdpbGwgYXNrIFpob3UgV2FuZyB0byBpbmNsdWRlIHRoaXMgcGF0Y2ggaW4gaGlz
IHBhdGNoc2V0DQo+ID4gPiA+DQo+ID4gPiA+DQo+ID4gPiA+Pg0KPiA+ID4gPj4gPiAgICAgVGhp
cyBwYXRjaCBhZGRzIGEgbmV3IGZpZWxkIGluICJzdHJ1Y3Qgb2ZfcGNpX3JhbmdlIiB0bw0KPiBz
dG9yZQ0KPiA+ID4gdGhlDQo+ID4gPiA+PiA+ICAgICBwY2kgYnVzIHN0YXJ0IGFkZHJlc3M7IGl0
IGZpbGxzIHRoZSBmaWVsZCBpbg0KPiA+ID4gPj4gb2ZfcGNpX3JhbmdlX3BhcnNlcl9vbmUoKTsN
Cj4gPiA+ID4+ID4gICAgIGluIG9mX3BjaV9nZXRfaG9zdF9icmlkZ2VfcmVzb3VyY2VzKCkgaXQg
cmV0cmlldmVzIHRoZQ0KPiA+ID4gcmVzb3VyY2UNCj4gPiA+ID4+IGVudHJ5DQo+ID4gPiA+PiA+
ICAgICBhZnRlciBpdCBpcyBjcmVhdGVkIGFuZCBhZGRlZCB0byB0aGUgcmVzb3VyY2UgbGlzdCBh
bmQNCj4gdXNlcw0KPiA+ID4gPj4gPiAgICAgZW50cnktPl9fcmVzLnN0YXJ0IHRvIHN0b3JlIHRo
ZSBwY2kgY29udHJvbGxlciBhZGRyZXNzDQo+ID4gPiA+Pg0KPiA+ID4gPj4gc3RydWN0IG9mX3Bj
aV9yYW5nZSBpcyBzdGFydGluZyB0byBnZXQgY29uZnVzaW5nIHRvIG5vbi1PRiBmb2xrcw0KPiA+
ID4gbGlrZQ0KPiA+ID4gPj4gbWUuDQo+ID4gPiA+PiBJdCBub3cgY29udGFpbnM6DQo+ID4gPiA+
Pg0KPiA+ID4gPj4gICB1MzIgcGNpX3NwYWNlOw0KPiA+ID4gPj4gICB1NjQgcGNpX2FkZHI7DQo+
ID4gPiA+PiAgIHU2NCBjcHVfYWRkcjsNCj4gPiA+ID4+ICAgdTY0IGJ1c19hZGRyOw0KPiA+ID4g
Pj4NCj4gPiA+ID4+IENhbiB5b3UgZXhwbGFpbiB3aGF0IGFsbCB0aGVzZSB0aGluZ3MgbWVhbiwg
YW5kIG1heWJlIGV2ZW4gYWRkDQo+IG9uZS0NCj4gPiA+IGxpbmUNCj4gPiA+ID4+IGNvbW1lbnRz
IHRvIHRoZSBzdHJ1Y3R1cmU/DQo+ID4gPiA+DQo+ID4gPiA+IHN1cmUgSSBjYW4gYWRkIGNvbW1l
bnRzIGlubGluZSBpbiB0aGUgY29kZQ0KPiA+ID4gPg0KPiA+ID4gPj4NCj4gPiA+ID4+IHBjaV9z
cGFjZTogVGhlIG9ubHkgdXNlcyBJIHNlZSBhcmUgdG8gZGV0ZXJtaW5lIHdoZXRoZXIgdG8gcHJp
bnQNCj4gPiA+ID4+ICJQcmVmZXRjaCIuICBJIGRvbid0IHNlZSBhbnkgcmVhbCBmdW5jdGlvbmFs
aXR5IHRoYXQgdXNlcyB0aGlzLg0KPiA+ID4gPg0KPiA+ID4gPiBMb29raW5nIGF0IHRoZSBjb2Rl
IEkgYWdyZWUuIGl0J3Mgc2VlbXMgdG8gYmUgdXNlZCBvbmx5IGluDQo+IHBvd2VycGMNCj4gPiA+
ID4gYW5kIG1pY3JvYmxhemUgdG8gcHJpbnQgb3V0Lg0KPiA+ID4gPiBIb3dldmVyIGZyb20gbXkg
dW5kZXJzdGFuZGluZyBwY2lfc3BhY2UgaXMgdGhlIHBoeXMuaGkgZmllbGQgb2YNCj4gdGhlDQo+
ID4gPiA+IHJhbmdlcyBwcm9wZXJ0eTogaXQgZGVmaW5lcyB0aGUgcHJvcGVydGllcyBvZiB0aGUg
YWRkcmVzcyBzcGFjZQ0KPiA+ID4gYXNzb2NpYXRlZA0KPiA+ID4gPiB0byB0aGUgUENJIGFkZHJl
c3MuIGlmIHlvdSdyZSBjdXJpb3VzIHlvdSBjYW4gZmluZCBhIG5pY2UgYW5kDQo+IHF1aWNrDQo+
ID4gPiB0byByZWFkDQo+ID4gPiA+ICJndWlkZSIgaW4gaHR0cDovL2RldmljZXRyZWUub3JnL01Q
QzUyMDA6UENJDQo+ID4gPiA+DQo+ID4gPiA+Pg0KPiA+ID4gPj4gcGNpX2FkZHI6IEkgYXNzdW1l
IHRoaXMgaXMgYSBQQ0kgYnVzIGFkZHJlc3MsIGxpa2Ugd2hhdCB5b3UNCj4gd291bGQNCj4gPiA+
IHNlZQ0KPiA+ID4gPj4gaWYNCj4gPiA+ID4+IHlvdSBwdXQgYW4gYW5hbHl6ZXIgb24gdGhlIGJ1
cy9saW5rLiAgVGhpcyBhZGRyZXNzIGNvdWxkIGdvIGluIGENCj4gQkFSLg0KPiA+ID4gPg0KPiA+
ID4gPiBZZXMsIHRoaXMgaXMgdGhlIFBDSSBzdGFydCBhZGRyZXNzIG9mIHRoZSByYW5nZTogcGh5
cy5taWQgKw0KPiBwaHlzLmxvdw0KPiA+ID4gaW4gdGhlDQo+ID4gPiA+IGd1aWRlIG1lbnRpb25l
ZCBhYm92ZQ0KPiA+ID4gPg0KPiA+ID4gPj4NCj4gPiA+ID4+IGNwdV9hZGRyOiBJIGFzc3VtZSB0
aGlzIGlzIGEgQ1BVIHBoeXNpY2FsIGFkZHJlc3MsIGxpa2Ugd2hhdCB5b3UNCj4gPiA+IHdvdWxk
DQo+ID4gPiA+PiBzZWUNCj4gPiA+ID4+IGluIC9wcm9jL2lvbWVtIGFuZCB3aGF0IHlvdSB3b3Vs
ZCBwYXNzIHRvIGlvcmVtYXAoKS4NCj4gPiA+ID4NCj4gPiA+ID4gWWVzIGNvcnJlY3QNCj4gPiA+
ID4NCj4gPiA+ID4+DQo+ID4gPiA+PiBidXNfYWRkcjogPw0KPiA+ID4gPj4NCj4gPiA+ID4NCj4g
PiA+ID4gQWNjb3JkaW5nIHRvIHRoZSBndWlkZSBhYm92ZSwgdGhpcyBpcyB0aGUgYWRkcmVzcyBp
bnRvIHdoaWNoIHRoZQ0KPiA+ID4gcGNpX2FkZHJlc3MNCj4gPiA+ID4gZ2V0IHRyYW5zbGF0ZWQg
dG8gYW5kIHRoYXQgaXMgcGFzc2VkIHRvIHRoZSByb290IGNvbXBsZXguIEJldHdlZW4NCj4gdGhl
DQo+ID4gPiByb290DQo+ID4gPiA+IGNvbXBsZXggYW5kIHRoZSBDUFUgdGhlcmUgY2FuIGJlIGlu
dGVybWVkaWF0ZSB0cmFuc2xhdGlvbiBsYXllcnM6DQo+IHNlZQ0KPiA+ID4gdGhhdCB0bw0KPiA+
ID4gPiBnZXQgcGNpX2FkZHJlc3Mgd2UgY2FsbCAib2ZfdHJhbnNsYXRlX2FkZHJlc3MiOyB0aGlz
IHdpbGwgYXBwbHkNCj4gYWxsDQo+ID4gPiB0aGUNCj4gPiA+ID4gdHJhbnNsYXRpb24gbGF5ZXJz
IChyYW5nZXMgaW4gdGhlIERUKSB0aGF0IGl0IGZpbmRzIHRpbGwgaXQgY29tZXMNCj4gdG8NCj4g
PiA+IHRoZSByb290DQo+ID4gPiA+IG5vZGUgb2YgdGhlIERUICh0aHVzIHJldHJpZXZpbmcgdGhl
IENQVSBhZGRyZXNzKS4NCj4gPiA+ID4gTm93IHNhaWQgdGhhdCwgZm9yIGRlc2lnbndhcmUgd2Ug
bmVlZCB0aGUgZmlyc3QgdHJhbnNsYXRlZCBQQ0kNCj4gPiA+IGFkZHJlc3MsIHRoYXQgd2UgY2Fs
bA0KPiA+ID4NCj4gPiA+IEkgdGhpbmsgeW91IG1lYW4gInRyYW5zbGF0ZWQgQ1BVIGFkZHJlc3Mu
IiBUaGUgZmxvdyBvZiBhZGRyZXNzZXMNCj4gbG9va3MNCj4gPiA+IGxpa2UgdGhpczoNCj4gPiA+
DQo+ID4gPiBDUFUgLT4gQ1BVIGJ1cyBhZGRyZXNzIC0+IGJ1cyBmYWJyaWMgYWRkcmVzcyBkZWNv
ZGluZyAtPiBidXMNCj4gYWRkcmVzcw0KPiA+ID4gLT4gRFcgUENJIC0+IFBDSSBhZGRyZXNzDQo+
ID4gPg0KPiA+ID4gVGhpcyBpcyBxdWl0ZSBjb21tb24gdGhhdCBhbiBJUCBibG9jayBkb2VzIG5v
dCBzZWUgdGhlIGZ1bGwgYWRkcmVzcy4NCj4gPiA+IEl0IGlzIHVudXN1YWwgdGhhdCB0aGUgSVAg
YmxvY2sgbmVlZHMgdG8ga25vdyBpdHMgZnVsbCBhZGRyZXNzIG9uDQo+IHRoZQ0KPiA+ID4gc2xh
dmUgc2lkZS4gSXQgaXMgcXVpdGUgY29tbW9uIGZvciB0aGUgbWFzdGVyIHNpZGUgYW5kIHRoZSBr
ZXJuZWwNCj4gPiA+IGRlYWxzIHdpdGggdGhhdCBhbGwgdGhlIHRpbWUgd2l0aCBETUEgbWFwcGlu
Zw0KPiA+ID4NCj4gPiA+ID4gaGVyZSBidXNfYWRkciBhZnRlciBSb2IgSGVycmluZyBzdWdnZXN0
ZWQgdGhlIG5hbWUuLi5ob25lc3RseSBJDQo+ID4gPiBjYW5ub3QgdGhpbmsgb2YgYQ0KPiA+ID4g
PiBkaWZmZXJlbnQgbmFtZQ0KPiA+ID4NCj4gPiA+IFRoaW5raW5nIGFib3V0IHRoaXMgc29tZSBt
b3JlLCBpcyB0aGlzIHJlYWxseSBhIHRyYW5zbGF0aW9uIHZlcnN1cw0KPiA+ID4ganVzdCBhIHN0
cmlwcGluZyBvZiBoaWdoIGJpdHM/IERvZXMgdGhlIERXIElQIGhhdmUgbGVzcyB0aGFuIDMyLQ0K
PiBiaXRzDQo+ID4gPiBhZGRyZXNzPyBJZiBzbywgd2UgY291bGQgZXhwcmVzcyBkaWZmZXJlbnRs
eSBhbmQganVzdCBtYXNrIHRoZSBDUFUNCj4gPiA+IGFkZHJlc3Mgd2l0aGluIHRoZSBkcml2ZXIg
aW5zdGVhZC4NCj4gPg0KPiA+IEkgZG9u4oCZdCB0aGluayB3ZSBzaG91bGQgcmVseSBvbiBbQ1BV
XSBhZGRyZXNzZXMuLi53aGF0IGlmIHRoZQ0KPiBpbnRlcm1lZGlhdGUNCj4gPiB0cmFuc2xhdGlv
biBsYXllciBjaGFuZ2VzIHRoZSBsb3dlciBzaWduaWZpY2FudCBiaXRzIG9mIHRoZSAiYnVzDQo+
IGFkZHJlc3MiDQo+ID4gdG8gdHJhbnNsYXRlIGludG8gYSBjcHUgYWRkcmVzcz8NCj4gDQo+IElz
IGl0IHJlYWxseSBhIHBvc3NpYmxpdHkgdGhhdCB0aGUgbG93ZXIgYml0cyBjb3VsZCBiZSBjaGFu
Z2VkPw0KDQpJJ3ZlIGNoZWNrZWQgYWxsIHRoZSBjdXJyZW50IGRlaWdud2FyZSB1c2VycyBEVHMg
ZXhjZXB0ICJwY2ktbGF5ZXJzY2FwZSIgDQp0aGF0IEkgY291bGQgbm90IGZpbmQ6DQpzcGVhcjEz
MTAuZHRzaQ0Kc3BlYXIxMzQwLmR0c2kNCmRyYTcuZHRzaQ0KaW14NnFkbC5kdHNpDQppbXg2c3gu
ZHRzaQ0Ka2V5c3RvbmUuZHRzaQ0KZXh5bm9zNTQ0MC5kdHNpDQoNCk5vbmUgb2YgdGhlbSBtb2Rp
ZmllcyB0aGUgbG93ZXIgYml0cy4gVG8gYmUgbW9yZSBwcmVjaXNlIHRoZSBvbmx5IGd1eQ0KdGhh
dCBwcm92aWRlcyBhbm90aGVyIHRyYW5zbGF0aW9uIGxheWVyIGlzICJkcmE3LmR0c2kiOg0KYXhp
MA0KaHR0cDovL2x4ci5mcmVlLWVsZWN0cm9ucy5jb20vc291cmNlL2FyY2gvYXJtL2Jvb3QvZHRz
L2RyYTcuZHRzaSNMMjA3DQoNCmF4aTENCmh0dHA6Ly9seHIuZnJlZS1lbGVjdHJvbnMuY29tL3Nv
dXJjZS9hcmNoL2FybS9ib290L2R0cy9kcmE3LmR0c2kjTDI0MQ0KDQpGb3IgdGhpcyBjYXNlIG1h
c2tpbmcgdGhlIHRvcCA0Yml0cyAoYml0czI4IHRvIDMxKSBzaG91bGQgbWFrZSB0aGUgam9iLg0K
DQpTcGVha2luZyBpbiBnZW5lcmFsIHRlcm1zIHNvIGZhciBJJ3ZlIGFsd2F5cyBzZWVuIGxpbmVh
ciBtYXBwaW5ncw0KdGhhdCBkaWZmZXIgYnkgYml0bWFzayBvZmZzZXQsIGhvd2V2ZXIgbGluZWFy
IGRvZXMgbm90IG1lYW4gdGhhdCB5b3UgDQpjYW5ub3QgYWZmZWN0IHRoZSBsb3dlciBiaXRzOiBl
LmcuIDwweDA+IHRvIDwweDAgKyBzaXplPiBjYW4gbWFwIHRvDQo8MHgwMDAwY2FmZSB0byAweDAw
MDBjYWZlICsgc2l6ZT4sIGJ1dCBJIGd1ZXNzIHRoYXQgZm9yIEhXIGRlc2lnbiByZWFzb25zDQpp
dCBpcyBtdWNoIGVhc2llciB0byByZW1hcCBkaXJlY3RseSB1c2luZyBhIGJpdG1hc2suLi5idXQg
SSB3YXMgbm90IHN1cmUNCmFuZCBJIGRpZG4ndCB0aGluayBhYm91dCB0aGUgcHJvYmxlbXMgdGhh
dCBjYW4gYXJpc2Ugd2l0aCBBQ1BJLg0KDQpJZiB5b3UgdGhpbmsgdGhlIGJpdG1hc2sgaXMgT2sg
dGhlbiBJIGNhbiBkaXJlY3RseSBkZWZpbmUgaXQgaW4NCmRlc2lnbndhcmUgYW5kIHdlIGNhbiBk
cm9wIHRoaXMgcGF0Y2guDQoNClRoYW5rcw0KR2FiDQogDQo+IA0KPiBJIHRoaW5rIHdlJ3JlIHRh
bGtpbmcgYWJvdXQgTU1JTyBoZXJlLCBhbmQgYSBicmlkZ2UgaGFzIGFuIE1NSU8NCj4gd2luZG93
LiAgQSB3aW5kb3cgaXMgYSByYW5nZSBvZiBDUFUgcGh5c2ljYWwgYWRkcmVzc2VzIHRoYXQgdGhl
DQo+IGJyaWRnZSBmb3J3YXJkcyB0byBQQ0kuICBUaGUgUENJIGNvcmUgYXNzdW1lcyB0aGF0IGEg
Q1BVIHBoeXNpY2FsDQo+IGFkZHJlc3MgcmFuZ2UgaXMgbGluZWFybHkgbWFwcGVkIHRvIFBDSSBi
dXMgYWRkcmVzc2VzLCBpLmUuLCBpZiB0aGUNCj4gd2luZG93IGJhc2UgaXMgWCBhbmQgbWFwcyB0
byBQQ0kgYWRkcmVzcyBZLCB0aGVuIGFzIGxvbmcgYXMgWCtuIGlzDQo+IGluc2lkZSB0aGUgd2lu
ZG93LCBpdCBtdXN0IG1hcCB0byBZK24uDQo+IA0KPiBUaGF0IG1lYW5zIHRoZSBsb3ctb3JkZXIg
Yml0cyAodGhlIG9uZXMgdGhhdCBhcmUgdGhlIG9mZnNldCBpbnRvIHRoZQ0KPiB3aW5kb3cpIGNh
bm5vdCBjaGFuZ2UuDQo+IC0tDQo+IFRvIHVuc3Vic2NyaWJlIGZyb20gdGhpcyBsaXN0OiBzZW5k
IHRoZSBsaW5lICJ1bnN1YnNjcmliZSBsaW51eC1wY2kiIGluDQo+IHRoZSBib2R5IG9mIGEgbWVz
c2FnZSB0byBtYWpvcmRvbW9Admdlci5rZXJuZWwub3JnDQo+IE1vcmUgbWFqb3Jkb21vIGluZm8g
YXQgIGh0dHA6Ly92Z2VyLmtlcm5lbC5vcmcvbWFqb3Jkb21vLWluZm8uaHRtbA0K

^ permalink raw reply	[flat|nested] 73+ messages in thread
* [PATCH v6] PCI: Store PCIe bus address in struct of_pci_range
@ 2015-07-30 16:50                   ` Gabriele Paoloni
  0 siblings, 0 replies; 73+ messages in thread
From: Gabriele Paoloni @ 2015-07-30 16:50 UTC (permalink / raw)
  To: linux-arm-kernel



> -----Original Message-----
> From: linux-pci-owner at vger.kernel.org [mailto:linux-pci-
> owner at vger.kernel.org] On Behalf Of Bjorn Helgaas
> Sent: Thursday, July 30, 2015 5:15 PM
> To: Gabriele Paoloni
> Cc: Rob Herring; arnd at arndb.de; lorenzo.pieralisi at arm.com; Wangzhou (B);
> robh+dt at kernel.org; james.morse at arm.com; Liviu.Dudau at arm.com; linux-
> pci at vger.kernel.org; linux-arm-kernel at lists.infradead.org;
> devicetree at vger.kernel.org; Yuanzhichang; Zhudacai; zhangjukuo;
> qiuzhenfa; Liguozhu (Kenneth)
> Subject: Re: [PATCH v6] PCI: Store PCIe bus address in struct
> of_pci_range
> 
> On Thu, Jul 30, 2015 at 01:52:13PM +0000, Gabriele Paoloni wrote:
> >
> >
> > > -----Original Message-----
> > > From: linux-pci-owner at vger.kernel.org [mailto:linux-pci-
> > > owner at vger.kernel.org] On Behalf Of Rob Herring
> > > Sent: Thursday, July 30, 2015 2:43 PM
> > > To: Gabriele Paoloni
> > > Cc: Bjorn Helgaas; arnd at arndb.de; lorenzo.pieralisi at arm.com;
> Wangzhou
> > > (B); robh+dt at kernel.org; james.morse at arm.com; Liviu.Dudau at arm.com;
> > > linux-pci at vger.kernel.org; linux-arm-kernel at lists.infradead.org;
> > > devicetree at vger.kernel.org; Yuanzhichang; Zhudacai; zhangjukuo;
> > > qiuzhenfa; Liguozhu (Kenneth)
> > > Subject: Re: [PATCH v6] PCI: Store PCIe bus address in struct
> > > of_pci_range
> > >
> > > On Wed, Jul 29, 2015 at 2:44 PM, Gabriele Paoloni
> > > <gabriele.paoloni@huawei.com> wrote:
> > > > Hi Bjorn
> > > >
> > > > Many Thanks for your reply
> > > >
> > > > I have commented back inline with resolutions from my side.
> > > >
> > > > If you're ok with them I'll send it out a new version in the
> > > appropriate patchset
> > > >
> > > > Cheers
> > > >
> > > > Gab
> > > >
> > > >> -----Original Message-----
> > > >> From: Bjorn Helgaas [mailto:bhelgaas at google.com]
> > > >> Sent: Wednesday, July 29, 2015 6:21 PM
> > > >> To: Gabriele Paoloni
> > > >> Cc: arnd at arndb.de; lorenzo.pieralisi at arm.com; Wangzhou (B);
> > > >> robh+dt at kernel.org; james.morse at arm.com; Liviu.Dudau at arm.com;
> linux-
> > > >> pci at vger.kernel.org; linux-arm-kernel at lists.infradead.org;
> > > >> devicetree at vger.kernel.org; Yuanzhichang; Zhudacai; zhangjukuo;
> > > >> qiuzhenfa; Liguozhu (Kenneth)
> > > >> Subject: Re: [PATCH v6] PCI: Store PCIe bus address in struct
> > > >> of_pci_range
> > > >>
> > > >> Hi Gabriele,
> > > >>
> > > >> As far as I can tell, this is not specific to PCIe, so please
> use
> > > "PCI"
> > > >> in
> > > >> the subject as a generic term that includes both PCI and PCIe.
> > > >
> > > > sure agreed
> > > >
> > > >>
> > > >> On Mon, Jul 27, 2015 at 11:17:03PM +0800, Gabriele Paoloni wrote:
> > > >> > From: gabriele paoloni <gabriele.paoloni@huawei.com>
> > > >> >
> > > >> >     This patch is needed port PCIe designware to new DT
> parsing
> > > API
> > > >> >     As discussed in
> > > >> >     http://lists.infradead.org/pipermail/linux-arm-
> kernel/2015-
> > > >> January/317743.html
> > > >> >     in designware we have a problem as the PCI addresses in
> the
> > > PCIe
> > > >> controller
> > > >> >     address space are required in order to perform correct HW
> > > >> operation.
> > > >> >
> > > >> >     In order to solve this problem commit f4c55c5a3 "PCI:
> > > designware:
> > > >> >     Program ATU with untranslated address" added code to read
> the
> > > >> PCIe
> > > >>
> > > >> Conventional reference is 12-char SHA1, like this:
> > > >>
> > > >>   f4c55c5a3f7f ("PCI: designware: Program ATU with untranslated
> > > >> address")
> > > >
> > > > Agreed, will change this
> > > >
> > > >>
> > > >> >     controller start address directly from the DT ranges.
> > > >> >
> > > >> >     In the new DT parsing API
> of_pci_get_host_bridge_resources()
> > > >> hides the
> > > >> >     DT parser from the host controller drivers, so it is not
> > > possible
> > > >> >     for drivers to parse values directly from the DT.
> > > >> >
> > > >> >     In http://www.spinics.net/lists/linux-pci/msg42540.html we
> > > >> already tried
> > > >> >     to use the new DT parsing API but there is a bug
> (obviously)
> > > in
> > > >> setting
> > > >> >     the <*>_mod_base addresses
> > > >> >     Applying this patch we can easily set "<*>_mod_base = win-
> > > >> >__res.start"
> > > >>
> > > >> By itself, this patch adds something.  It would help me
> understand
> > > it
> > > >> if
> > > >> the *user* of this new something were in the same patch series.
> > > >
> > > > the user is: "[PATCH v5 2/5] PCI: designware: Add ARM64 support"
> > > > I will ask Zhou Wang to include this patch in his patchset
> > > >
> > > >
> > > >>
> > > >> >     This patch adds a new field in "struct of_pci_range" to
> store
> > > the
> > > >> >     pci bus start address; it fills the field in
> > > >> of_pci_range_parser_one();
> > > >> >     in of_pci_get_host_bridge_resources() it retrieves the
> > > resource
> > > >> entry
> > > >> >     after it is created and added to the resource list and
> uses
> > > >> >     entry->__res.start to store the pci controller address
> > > >>
> > > >> struct of_pci_range is starting to get confusing to non-OF folks
> > > like
> > > >> me.
> > > >> It now contains:
> > > >>
> > > >>   u32 pci_space;
> > > >>   u64 pci_addr;
> > > >>   u64 cpu_addr;
> > > >>   u64 bus_addr;
> > > >>
> > > >> Can you explain what all these things mean, and maybe even add
> one-
> > > line
> > > >> comments to the structure?
> > > >
> > > > sure I can add comments inline in the code
> > > >
> > > >>
> > > >> pci_space: The only uses I see are to determine whether to print
> > > >> "Prefetch".  I don't see any real functionality that uses this.
> > > >
> > > > Looking at the code I agree. it's seems to be used only in
> powerpc
> > > > and microblaze to print out.
> > > > However from my understanding pci_space is the phys.hi field of
> the
> > > > ranges property: it defines the properties of the address space
> > > associated
> > > > to the PCI address. if you're curious you can find a nice and
> quick
> > > to read
> > > > "guide" in http://devicetree.org/MPC5200:PCI
> > > >
> > > >>
> > > >> pci_addr: I assume this is a PCI bus address, like what you
> would
> > > see
> > > >> if
> > > >> you put an analyzer on the bus/link.  This address could go in a
> BAR.
> > > >
> > > > Yes, this is the PCI start address of the range: phys.mid +
> phys.low
> > > in the
> > > > guide mentioned above
> > > >
> > > >>
> > > >> cpu_addr: I assume this is a CPU physical address, like what you
> > > would
> > > >> see
> > > >> in /proc/iomem and what you would pass to ioremap().
> > > >
> > > > Yes correct
> > > >
> > > >>
> > > >> bus_addr: ?
> > > >>
> > > >
> > > > According to the guide above, this is the address into which the
> > > pci_address
> > > > get translated to and that is passed to the root complex. Between
> the
> > > root
> > > > complex and the CPU there can be intermediate translation layers:
> see
> > > that to
> > > > get pci_address we call "of_translate_address"; this will apply
> all
> > > the
> > > > translation layers (ranges in the DT) that it finds till it comes
> to
> > > the root
> > > > node of the DT (thus retrieving the CPU address).
> > > > Now said that, for designware we need the first translated PCI
> > > address, that we call
> > >
> > > I think you mean "translated CPU address." The flow of addresses
> looks
> > > like this:
> > >
> > > CPU -> CPU bus address -> bus fabric address decoding -> bus
> address
> > > -> DW PCI -> PCI address
> > >
> > > This is quite common that an IP block does not see the full address.
> > > It is unusual that the IP block needs to know its full address on
> the
> > > slave side. It is quite common for the master side and the kernel
> > > deals with that all the time with DMA mapping
> > >
> > > > here bus_addr after Rob Herring suggested the name...honestly I
> > > cannot think of a
> > > > different name
> > >
> > > Thinking about this some more, is this really a translation versus
> > > just a stripping of high bits? Does the DW IP have less than 32-
> bits
> > > address? If so, we could express differently and just mask the CPU
> > > address within the driver instead.
> >
> > I don?t think we should rely on [CPU] addresses...what if the
> intermediate
> > translation layer changes the lower significant bits of the "bus
> address"
> > to translate into a cpu address?
> 
> Is it really a possiblity that the lower bits could be changed?

I've checked all the current deignware users DTs except "pci-layerscape" 
that I could not find:
spear1310.dtsi
spear1340.dtsi
dra7.dtsi
imx6qdl.dtsi
imx6sx.dtsi
keystone.dtsi
exynos5440.dtsi

None of them modifies the lower bits. To be more precise the only guy
that provides another translation layer is "dra7.dtsi":
axi0
http://lxr.free-electrons.com/source/arch/arm/boot/dts/dra7.dtsi#L207

axi1
http://lxr.free-electrons.com/source/arch/arm/boot/dts/dra7.dtsi#L241

For this case masking the top 4bits (bits28 to 31) should make the job.

Speaking in general terms so far I've always seen linear mappings
that differ by bitmask offset, however linear does not mean that you 
cannot affect the lower bits: e.g. <0x0> to <0x0 + size> can map to
<0x0000cafe to 0x0000cafe + size>, but I guess that for HW design reasons
it is much easier to remap directly using a bitmask...but I was not sure
and I didn't think about the problems that can arise with ACPI.

If you think the bitmask is Ok then I can directly define it in
designware and we can drop this patch.

Thanks
Gab
 
> 
> I think we're talking about MMIO here, and a bridge has an MMIO
> window.  A window is a range of CPU physical addresses that the
> bridge forwards to PCI.  The PCI core assumes that a CPU physical
> address range is linearly mapped to PCI bus addresses, i.e., if the
> window base is X and maps to PCI address Y, then as long as X+n is
> inside the window, it must map to Y+n.
> 
> That means the low-order bits (the ones that are the offset into the
> window) cannot change.
> --
> To unsubscribe from this list: send the line "unsubscribe linux-pci" in
> the body of a message to majordomo at vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 73+ messages in thread
* Re: [PATCH v6] PCI: Store PCIe bus address in struct of_pci_range
  2015-07-30 16:50                   ` Gabriele Paoloni
@ 2015-07-30 17:14                     ` Bjorn Helgaas
  -1 siblings, 0 replies; 73+ messages in thread
From: Bjorn Helgaas @ 2015-07-30 17:14 UTC (permalink / raw)
  To: Gabriele Paoloni
  Cc: Rob Herring, arnd, lorenzo.pieralisi, Wangzhou (B),
	robh+dt, james.morse, Liviu.Dudau, linux-pci, linux-arm-kernel,
	devicetree, Yuanzhichang, Zhudacai, zhangjukuo, qiuzhenfa,
	Liguozhu (Kenneth),
	Jingoo Han, Pratyush Anand

[+cc Jingoo, Pratyush]

On Thu, Jul 30, 2015 at 04:50:55PM +0000, Gabriele Paoloni wrote:
> > -----Original Message-----
> > From: linux-pci-owner@vger.kernel.org [mailto:linux-pci-
> > owner@vger.kernel.org] On Behalf Of Bjorn Helgaas
> > Sent: Thursday, July 30, 2015 5:15 PM
> > To: Gabriele Paoloni
> > Cc: Rob Herring; arnd@arndb.de; lorenzo.pieralisi@arm.com; Wangzhou (B);
> > robh+dt@kernel.org; james.morse@arm.com; Liviu.Dudau@arm.com; linux-
> > pci@vger.kernel.org; linux-arm-kernel@lists.infradead.org;
> > devicetree@vger.kernel.org; Yuanzhichang; Zhudacai; zhangjukuo;
> > qiuzhenfa; Liguozhu (Kenneth)
> > Subject: Re: [PATCH v6] PCI: Store PCIe bus address in struct
> > of_pci_range
> > 
> > On Thu, Jul 30, 2015 at 01:52:13PM +0000, Gabriele Paoloni wrote:
> > >
> > >
> > > > -----Original Message-----
> > > > From: linux-pci-owner@vger.kernel.org [mailto:linux-pci-
> > > > owner@vger.kernel.org] On Behalf Of Rob Herring
> > > > Sent: Thursday, July 30, 2015 2:43 PM
> > > > To: Gabriele Paoloni
> > > > Cc: Bjorn Helgaas; arnd@arndb.de; lorenzo.pieralisi@arm.com;
> > Wangzhou
> > > > (B); robh+dt@kernel.org; james.morse@arm.com; Liviu.Dudau@arm.com;
> > > > linux-pci@vger.kernel.org; linux-arm-kernel@lists.infradead.org;
> > > > devicetree@vger.kernel.org; Yuanzhichang; Zhudacai; zhangjukuo;
> > > > qiuzhenfa; Liguozhu (Kenneth)
> > > > Subject: Re: [PATCH v6] PCI: Store PCIe bus address in struct
> > > > of_pci_range
> > > >
> > > > On Wed, Jul 29, 2015 at 2:44 PM, Gabriele Paoloni
> > > > <gabriele.paoloni@huawei.com> wrote:
> > > > > Hi Bjorn
> > > > >
> > > > > Many Thanks for your reply
> > > > >
> > > > > I have commented back inline with resolutions from my side.
> > > > >
> > > > > If you're ok with them I'll send it out a new version in the
> > > > appropriate patchset
> > > > >
> > > > > Cheers
> > > > >
> > > > > Gab
> > > > >
> > > > >> -----Original Message-----
> > > > >> From: Bjorn Helgaas [mailto:bhelgaas@google.com]
> > > > >> Sent: Wednesday, July 29, 2015 6:21 PM
> > > > >> To: Gabriele Paoloni
> > > > >> Cc: arnd@arndb.de; lorenzo.pieralisi@arm.com; Wangzhou (B);
> > > > >> robh+dt@kernel.org; james.morse@arm.com; Liviu.Dudau@arm.com;
> > linux-
> > > > >> pci@vger.kernel.org; linux-arm-kernel@lists.infradead.org;
> > > > >> devicetree@vger.kernel.org; Yuanzhichang; Zhudacai; zhangjukuo;
> > > > >> qiuzhenfa; Liguozhu (Kenneth)
> > > > >> Subject: Re: [PATCH v6] PCI: Store PCIe bus address in struct
> > > > >> of_pci_range
> > > > >>
> > > > >> Hi Gabriele,
> > > > >>
> > > > >> As far as I can tell, this is not specific to PCIe, so please
> > use
> > > > "PCI"
> > > > >> in
> > > > >> the subject as a generic term that includes both PCI and PCIe.
> > > > >
> > > > > sure agreed
> > > > >
> > > > >>
> > > > >> On Mon, Jul 27, 2015 at 11:17:03PM +0800, Gabriele Paoloni wrote:
> > > > >> > From: gabriele paoloni <gabriele.paoloni@huawei.com>
> > > > >> >
> > > > >> >     This patch is needed port PCIe designware to new DT
> > parsing
> > > > API
> > > > >> >     As discussed in
> > > > >> >     http://lists.infradead.org/pipermail/linux-arm-
> > kernel/2015-
> > > > >> January/317743.html
> > > > >> >     in designware we have a problem as the PCI addresses in
> > the
> > > > PCIe
> > > > >> controller
> > > > >> >     address space are required in order to perform correct HW
> > > > >> operation.
> > > > >> >
> > > > >> >     In order to solve this problem commit f4c55c5a3 "PCI:
> > > > designware:
> > > > >> >     Program ATU with untranslated address" added code to read
> > the
> > > > >> PCIe
> > > > >>
> > > > >> Conventional reference is 12-char SHA1, like this:
> > > > >>
> > > > >>   f4c55c5a3f7f ("PCI: designware: Program ATU with untranslated
> > > > >> address")
> > > > >
> > > > > Agreed, will change this
> > > > >
> > > > >>
> > > > >> >     controller start address directly from the DT ranges.
> > > > >> >
> > > > >> >     In the new DT parsing API
> > of_pci_get_host_bridge_resources()
> > > > >> hides the
> > > > >> >     DT parser from the host controller drivers, so it is not
> > > > possible
> > > > >> >     for drivers to parse values directly from the DT.
> > > > >> >
> > > > >> >     In http://www.spinics.net/lists/linux-pci/msg42540.html we
> > > > >> already tried
> > > > >> >     to use the new DT parsing API but there is a bug
> > (obviously)
> > > > in
> > > > >> setting
> > > > >> >     the <*>_mod_base addresses
> > > > >> >     Applying this patch we can easily set "<*>_mod_base = win-
> > > > >> >__res.start"
> > > > >>
> > > > >> By itself, this patch adds something.  It would help me
> > understand
> > > > it
> > > > >> if
> > > > >> the *user* of this new something were in the same patch series.
> > > > >
> > > > > the user is: "[PATCH v5 2/5] PCI: designware: Add ARM64 support"
> > > > > I will ask Zhou Wang to include this patch in his patchset
> > > > >
> > > > >
> > > > >>
> > > > >> >     This patch adds a new field in "struct of_pci_range" to
> > store
> > > > the
> > > > >> >     pci bus start address; it fills the field in
> > > > >> of_pci_range_parser_one();
> > > > >> >     in of_pci_get_host_bridge_resources() it retrieves the
> > > > resource
> > > > >> entry
> > > > >> >     after it is created and added to the resource list and
> > uses
> > > > >> >     entry->__res.start to store the pci controller address
> > > > >>
> > > > >> struct of_pci_range is starting to get confusing to non-OF folks
> > > > like
> > > > >> me.
> > > > >> It now contains:
> > > > >>
> > > > >>   u32 pci_space;
> > > > >>   u64 pci_addr;
> > > > >>   u64 cpu_addr;
> > > > >>   u64 bus_addr;
> > > > >>
> > > > >> Can you explain what all these things mean, and maybe even add
> > one-
> > > > line
> > > > >> comments to the structure?
> > > > >
> > > > > sure I can add comments inline in the code
> > > > >
> > > > >>
> > > > >> pci_space: The only uses I see are to determine whether to print
> > > > >> "Prefetch".  I don't see any real functionality that uses this.
> > > > >
> > > > > Looking at the code I agree. it's seems to be used only in
> > powerpc
> > > > > and microblaze to print out.
> > > > > However from my understanding pci_space is the phys.hi field of
> > the
> > > > > ranges property: it defines the properties of the address space
> > > > associated
> > > > > to the PCI address. if you're curious you can find a nice and
> > quick
> > > > to read
> > > > > "guide" in http://devicetree.org/MPC5200:PCI
> > > > >
> > > > >>
> > > > >> pci_addr: I assume this is a PCI bus address, like what you
> > would
> > > > see
> > > > >> if
> > > > >> you put an analyzer on the bus/link.  This address could go in a
> > BAR.
> > > > >
> > > > > Yes, this is the PCI start address of the range: phys.mid +
> > phys.low
> > > > in the
> > > > > guide mentioned above
> > > > >
> > > > >>
> > > > >> cpu_addr: I assume this is a CPU physical address, like what you
> > > > would
> > > > >> see
> > > > >> in /proc/iomem and what you would pass to ioremap().
> > > > >
> > > > > Yes correct
> > > > >
> > > > >>
> > > > >> bus_addr: ?
> > > > >>
> > > > >
> > > > > According to the guide above, this is the address into which the
> > > > pci_address
> > > > > get translated to and that is passed to the root complex. Between
> > the
> > > > root
> > > > > complex and the CPU there can be intermediate translation layers:
> > see
> > > > that to
> > > > > get pci_address we call "of_translate_address"; this will apply
> > all
> > > > the
> > > > > translation layers (ranges in the DT) that it finds till it comes
> > to
> > > > the root
> > > > > node of the DT (thus retrieving the CPU address).
> > > > > Now said that, for designware we need the first translated PCI
> > > > address, that we call
> > > >
> > > > I think you mean "translated CPU address." The flow of addresses
> > looks
> > > > like this:
> > > >
> > > > CPU -> CPU bus address -> bus fabric address decoding -> bus
> > address
> > > > -> DW PCI -> PCI address
> > > >
> > > > This is quite common that an IP block does not see the full address.
> > > > It is unusual that the IP block needs to know its full address on
> > the
> > > > slave side. It is quite common for the master side and the kernel
> > > > deals with that all the time with DMA mapping
> > > >
> > > > > here bus_addr after Rob Herring suggested the name...honestly I
> > > > cannot think of a
> > > > > different name
> > > >
> > > > Thinking about this some more, is this really a translation versus
> > > > just a stripping of high bits? Does the DW IP have less than 32-
> > bits
> > > > address? If so, we could express differently and just mask the CPU
> > > > address within the driver instead.
> > >
> > > I don’t think we should rely on [CPU] addresses...what if the
> > intermediate
> > > translation layer changes the lower significant bits of the "bus
> > address"
> > > to translate into a cpu address?
> > 
> > Is it really a possiblity that the lower bits could be changed?
> 
> I've checked all the current deignware users DTs except "pci-layerscape" 
> that I could not find:
> spear1310.dtsi
> spear1340.dtsi
> dra7.dtsi
> imx6qdl.dtsi
> imx6sx.dtsi
> keystone.dtsi
> exynos5440.dtsi
> 
> None of them modifies the lower bits. To be more precise the only guy
> that provides another translation layer is "dra7.dtsi":
> axi0
> http://lxr.free-electrons.com/source/arch/arm/boot/dts/dra7.dtsi#L207
> 
> axi1
> http://lxr.free-electrons.com/source/arch/arm/boot/dts/dra7.dtsi#L241
> 
> For this case masking the top 4bits (bits28 to 31) should make the job.
> 
> Speaking in general terms so far I've always seen linear mappings
> that differ by bitmask offset, however linear does not mean that you 
> cannot affect the lower bits: e.g. <0x0> to <0x0 + size> can map to
> <0x0000cafe to 0x0000cafe + size>, but I guess that for HW design reasons
> it is much easier to remap directly using a bitmask...but I was not sure
> and I didn't think about the problems that can arise with ACPI.

As I said, it wouldn't make sense for the bits that comprise the
offset into the window to change, so the mapping only has to be linear
inside the window.

For the bits outside the window offset, I don't know enough to say
whether masking is sufficient or safe.

> If you think the bitmask is Ok then I can directly define it in
> designware and we can drop this patch.

It's OK by me, but I know nothing about the actual hardware involved.
For the DesignWare question, I think you would just have to convince
Jingoo and Pratyush (cc'd).

> > I think we're talking about MMIO here, and a bridge has an MMIO
> > window.  A window is a range of CPU physical addresses that the
> > bridge forwards to PCI.  The PCI core assumes that a CPU physical
> > address range is linearly mapped to PCI bus addresses, i.e., if the
> > window base is X and maps to PCI address Y, then as long as X+n is
> > inside the window, it must map to Y+n.
> > 
> > That means the low-order bits (the ones that are the offset into the
> > window) cannot change.
> > --
> > To unsubscribe from this list: send the line "unsubscribe linux-pci" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 73+ messages in thread
* [PATCH v6] PCI: Store PCIe bus address in struct of_pci_range
@ 2015-07-30 17:14                     ` Bjorn Helgaas
  0 siblings, 0 replies; 73+ messages in thread
From: Bjorn Helgaas @ 2015-07-30 17:14 UTC (permalink / raw)
  To: linux-arm-kernel

[+cc Jingoo, Pratyush]

On Thu, Jul 30, 2015 at 04:50:55PM +0000, Gabriele Paoloni wrote:
> > -----Original Message-----
> > From: linux-pci-owner at vger.kernel.org [mailto:linux-pci-
> > owner at vger.kernel.org] On Behalf Of Bjorn Helgaas
> > Sent: Thursday, July 30, 2015 5:15 PM
> > To: Gabriele Paoloni
> > Cc: Rob Herring; arnd at arndb.de; lorenzo.pieralisi at arm.com; Wangzhou (B);
> > robh+dt at kernel.org; james.morse at arm.com; Liviu.Dudau at arm.com; linux-
> > pci at vger.kernel.org; linux-arm-kernel at lists.infradead.org;
> > devicetree at vger.kernel.org; Yuanzhichang; Zhudacai; zhangjukuo;
> > qiuzhenfa; Liguozhu (Kenneth)
> > Subject: Re: [PATCH v6] PCI: Store PCIe bus address in struct
> > of_pci_range
> > 
> > On Thu, Jul 30, 2015 at 01:52:13PM +0000, Gabriele Paoloni wrote:
> > >
> > >
> > > > -----Original Message-----
> > > > From: linux-pci-owner at vger.kernel.org [mailto:linux-pci-
> > > > owner at vger.kernel.org] On Behalf Of Rob Herring
> > > > Sent: Thursday, July 30, 2015 2:43 PM
> > > > To: Gabriele Paoloni
> > > > Cc: Bjorn Helgaas; arnd at arndb.de; lorenzo.pieralisi at arm.com;
> > Wangzhou
> > > > (B); robh+dt at kernel.org; james.morse at arm.com; Liviu.Dudau at arm.com;
> > > > linux-pci at vger.kernel.org; linux-arm-kernel at lists.infradead.org;
> > > > devicetree at vger.kernel.org; Yuanzhichang; Zhudacai; zhangjukuo;
> > > > qiuzhenfa; Liguozhu (Kenneth)
> > > > Subject: Re: [PATCH v6] PCI: Store PCIe bus address in struct
> > > > of_pci_range
> > > >
> > > > On Wed, Jul 29, 2015 at 2:44 PM, Gabriele Paoloni
> > > > <gabriele.paoloni@huawei.com> wrote:
> > > > > Hi Bjorn
> > > > >
> > > > > Many Thanks for your reply
> > > > >
> > > > > I have commented back inline with resolutions from my side.
> > > > >
> > > > > If you're ok with them I'll send it out a new version in the
> > > > appropriate patchset
> > > > >
> > > > > Cheers
> > > > >
> > > > > Gab
> > > > >
> > > > >> -----Original Message-----
> > > > >> From: Bjorn Helgaas [mailto:bhelgaas at google.com]
> > > > >> Sent: Wednesday, July 29, 2015 6:21 PM
> > > > >> To: Gabriele Paoloni
> > > > >> Cc: arnd at arndb.de; lorenzo.pieralisi at arm.com; Wangzhou (B);
> > > > >> robh+dt at kernel.org; james.morse at arm.com; Liviu.Dudau at arm.com;
> > linux-
> > > > >> pci at vger.kernel.org; linux-arm-kernel at lists.infradead.org;
> > > > >> devicetree at vger.kernel.org; Yuanzhichang; Zhudacai; zhangjukuo;
> > > > >> qiuzhenfa; Liguozhu (Kenneth)
> > > > >> Subject: Re: [PATCH v6] PCI: Store PCIe bus address in struct
> > > > >> of_pci_range
> > > > >>
> > > > >> Hi Gabriele,
> > > > >>
> > > > >> As far as I can tell, this is not specific to PCIe, so please
> > use
> > > > "PCI"
> > > > >> in
> > > > >> the subject as a generic term that includes both PCI and PCIe.
> > > > >
> > > > > sure agreed
> > > > >
> > > > >>
> > > > >> On Mon, Jul 27, 2015 at 11:17:03PM +0800, Gabriele Paoloni wrote:
> > > > >> > From: gabriele paoloni <gabriele.paoloni@huawei.com>
> > > > >> >
> > > > >> >     This patch is needed port PCIe designware to new DT
> > parsing
> > > > API
> > > > >> >     As discussed in
> > > > >> >     http://lists.infradead.org/pipermail/linux-arm-
> > kernel/2015-
> > > > >> January/317743.html
> > > > >> >     in designware we have a problem as the PCI addresses in
> > the
> > > > PCIe
> > > > >> controller
> > > > >> >     address space are required in order to perform correct HW
> > > > >> operation.
> > > > >> >
> > > > >> >     In order to solve this problem commit f4c55c5a3 "PCI:
> > > > designware:
> > > > >> >     Program ATU with untranslated address" added code to read
> > the
> > > > >> PCIe
> > > > >>
> > > > >> Conventional reference is 12-char SHA1, like this:
> > > > >>
> > > > >>   f4c55c5a3f7f ("PCI: designware: Program ATU with untranslated
> > > > >> address")
> > > > >
> > > > > Agreed, will change this
> > > > >
> > > > >>
> > > > >> >     controller start address directly from the DT ranges.
> > > > >> >
> > > > >> >     In the new DT parsing API
> > of_pci_get_host_bridge_resources()
> > > > >> hides the
> > > > >> >     DT parser from the host controller drivers, so it is not
> > > > possible
> > > > >> >     for drivers to parse values directly from the DT.
> > > > >> >
> > > > >> >     In http://www.spinics.net/lists/linux-pci/msg42540.html we
> > > > >> already tried
> > > > >> >     to use the new DT parsing API but there is a bug
> > (obviously)
> > > > in
> > > > >> setting
> > > > >> >     the <*>_mod_base addresses
> > > > >> >     Applying this patch we can easily set "<*>_mod_base = win-
> > > > >> >__res.start"
> > > > >>
> > > > >> By itself, this patch adds something.  It would help me
> > understand
> > > > it
> > > > >> if
> > > > >> the *user* of this new something were in the same patch series.
> > > > >
> > > > > the user is: "[PATCH v5 2/5] PCI: designware: Add ARM64 support"
> > > > > I will ask Zhou Wang to include this patch in his patchset
> > > > >
> > > > >
> > > > >>
> > > > >> >     This patch adds a new field in "struct of_pci_range" to
> > store
> > > > the
> > > > >> >     pci bus start address; it fills the field in
> > > > >> of_pci_range_parser_one();
> > > > >> >     in of_pci_get_host_bridge_resources() it retrieves the
> > > > resource
> > > > >> entry
> > > > >> >     after it is created and added to the resource list and
> > uses
> > > > >> >     entry->__res.start to store the pci controller address
> > > > >>
> > > > >> struct of_pci_range is starting to get confusing to non-OF folks
> > > > like
> > > > >> me.
> > > > >> It now contains:
> > > > >>
> > > > >>   u32 pci_space;
> > > > >>   u64 pci_addr;
> > > > >>   u64 cpu_addr;
> > > > >>   u64 bus_addr;
> > > > >>
> > > > >> Can you explain what all these things mean, and maybe even add
> > one-
> > > > line
> > > > >> comments to the structure?
> > > > >
> > > > > sure I can add comments inline in the code
> > > > >
> > > > >>
> > > > >> pci_space: The only uses I see are to determine whether to print
> > > > >> "Prefetch".  I don't see any real functionality that uses this.
> > > > >
> > > > > Looking at the code I agree. it's seems to be used only in
> > powerpc
> > > > > and microblaze to print out.
> > > > > However from my understanding pci_space is the phys.hi field of
> > the
> > > > > ranges property: it defines the properties of the address space
> > > > associated
> > > > > to the PCI address. if you're curious you can find a nice and
> > quick
> > > > to read
> > > > > "guide" in http://devicetree.org/MPC5200:PCI
> > > > >
> > > > >>
> > > > >> pci_addr: I assume this is a PCI bus address, like what you
> > would
> > > > see
> > > > >> if
> > > > >> you put an analyzer on the bus/link.  This address could go in a
> > BAR.
> > > > >
> > > > > Yes, this is the PCI start address of the range: phys.mid +
> > phys.low
> > > > in the
> > > > > guide mentioned above
> > > > >
> > > > >>
> > > > >> cpu_addr: I assume this is a CPU physical address, like what you
> > > > would
> > > > >> see
> > > > >> in /proc/iomem and what you would pass to ioremap().
> > > > >
> > > > > Yes correct
> > > > >
> > > > >>
> > > > >> bus_addr: ?
> > > > >>
> > > > >
> > > > > According to the guide above, this is the address into which the
> > > > pci_address
> > > > > get translated to and that is passed to the root complex. Between
> > the
> > > > root
> > > > > complex and the CPU there can be intermediate translation layers:
> > see
> > > > that to
> > > > > get pci_address we call "of_translate_address"; this will apply
> > all
> > > > the
> > > > > translation layers (ranges in the DT) that it finds till it comes
> > to
> > > > the root
> > > > > node of the DT (thus retrieving the CPU address).
> > > > > Now said that, for designware we need the first translated PCI
> > > > address, that we call
> > > >
> > > > I think you mean "translated CPU address." The flow of addresses
> > looks
> > > > like this:
> > > >
> > > > CPU -> CPU bus address -> bus fabric address decoding -> bus
> > address
> > > > -> DW PCI -> PCI address
> > > >
> > > > This is quite common that an IP block does not see the full address.
> > > > It is unusual that the IP block needs to know its full address on
> > the
> > > > slave side. It is quite common for the master side and the kernel
> > > > deals with that all the time with DMA mapping
> > > >
> > > > > here bus_addr after Rob Herring suggested the name...honestly I
> > > > cannot think of a
> > > > > different name
> > > >
> > > > Thinking about this some more, is this really a translation versus
> > > > just a stripping of high bits? Does the DW IP have less than 32-
> > bits
> > > > address? If so, we could express differently and just mask the CPU
> > > > address within the driver instead.
> > >
> > > I don?t think we should rely on [CPU] addresses...what if the
> > intermediate
> > > translation layer changes the lower significant bits of the "bus
> > address"
> > > to translate into a cpu address?
> > 
> > Is it really a possiblity that the lower bits could be changed?
> 
> I've checked all the current deignware users DTs except "pci-layerscape" 
> that I could not find:
> spear1310.dtsi
> spear1340.dtsi
> dra7.dtsi
> imx6qdl.dtsi
> imx6sx.dtsi
> keystone.dtsi
> exynos5440.dtsi
> 
> None of them modifies the lower bits. To be more precise the only guy
> that provides another translation layer is "dra7.dtsi":
> axi0
> http://lxr.free-electrons.com/source/arch/arm/boot/dts/dra7.dtsi#L207
> 
> axi1
> http://lxr.free-electrons.com/source/arch/arm/boot/dts/dra7.dtsi#L241
> 
> For this case masking the top 4bits (bits28 to 31) should make the job.
> 
> Speaking in general terms so far I've always seen linear mappings
> that differ by bitmask offset, however linear does not mean that you 
> cannot affect the lower bits: e.g. <0x0> to <0x0 + size> can map to
> <0x0000cafe to 0x0000cafe + size>, but I guess that for HW design reasons
> it is much easier to remap directly using a bitmask...but I was not sure
> and I didn't think about the problems that can arise with ACPI.

As I said, it wouldn't make sense for the bits that comprise the
offset into the window to change, so the mapping only has to be linear
inside the window.

For the bits outside the window offset, I don't know enough to say
whether masking is sufficient or safe.

> If you think the bitmask is Ok then I can directly define it in
> designware and we can drop this patch.

It's OK by me, but I know nothing about the actual hardware involved.
For the DesignWare question, I think you would just have to convince
Jingoo and Pratyush (cc'd).

> > I think we're talking about MMIO here, and a bridge has an MMIO
> > window.  A window is a range of CPU physical addresses that the
> > bridge forwards to PCI.  The PCI core assumes that a CPU physical
> > address range is linearly mapped to PCI bus addresses, i.e., if the
> > window base is X and maps to PCI address Y, then as long as X+n is
> > inside the window, it must map to Y+n.
> > 
> > That means the low-order bits (the ones that are the offset into the
> > window) cannot change.
> > --
> > To unsubscribe from this list: send the line "unsubscribe linux-pci" in
> > the body of a message to majordomo at vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 73+ messages in thread
* RE: [PATCH v6] PCI: Store PCIe bus address in struct of_pci_range
  2015-07-30 17:14                     ` Bjorn Helgaas
  (?)
@ 2015-07-30 17:34                       ` Gabriele Paoloni
  -1 siblings, 0 replies; 73+ messages in thread
From: Gabriele Paoloni @ 2015-07-30 17:34 UTC (permalink / raw)
  To: Bjorn Helgaas, Gabriele Paoloni
  Cc: Rob Herring, arnd, lorenzo.pieralisi, Wangzhou (B),
	robh+dt, james.morse, Liviu.Dudau, linux-pci, linux-arm-kernel,
	devicetree, Yuanzhichang, Zhudacai, zhangjukuo, qiuzhenfa,
	Liguozhu (Kenneth),
	Jingoo Han, Pratyush Anand



> -----Original Message-----
> From: Bjorn Helgaas [mailto:bhelgaas@google.com]
> Sent: 30 July 2015 18:15
> To: Gabriele Paoloni
> Cc: Rob Herring; arnd@arndb.de; lorenzo.pieralisi@arm.com; Wangzhou (B);
> robh+dt@kernel.org; james.morse@arm.com; Liviu.Dudau@arm.com; linux-
> pci@vger.kernel.org; linux-arm-kernel@lists.infradead.org;
> devicetree@vger.kernel.org; Yuanzhichang; Zhudacai; zhangjukuo;
> qiuzhenfa; Liguozhu (Kenneth); Jingoo Han; Pratyush Anand
> Subject: Re: [PATCH v6] PCI: Store PCIe bus address in struct
> of_pci_range
> 
> [+cc Jingoo, Pratyush]
> 
> On Thu, Jul 30, 2015 at 04:50:55PM +0000, Gabriele Paoloni wrote:
> > > -----Original Message-----
> > > From: linux-pci-owner@vger.kernel.org [mailto:linux-pci-
> > > owner@vger.kernel.org] On Behalf Of Bjorn Helgaas
> > > Sent: Thursday, July 30, 2015 5:15 PM
> > > To: Gabriele Paoloni
> > > Cc: Rob Herring; arnd@arndb.de; lorenzo.pieralisi@arm.com; Wangzhou
> (B);
> > > robh+dt@kernel.org; james.morse@arm.com; Liviu.Dudau@arm.com; linux-
> > > pci@vger.kernel.org; linux-arm-kernel@lists.infradead.org;
> > > devicetree@vger.kernel.org; Yuanzhichang; Zhudacai; zhangjukuo;
> > > qiuzhenfa; Liguozhu (Kenneth)
> > > Subject: Re: [PATCH v6] PCI: Store PCIe bus address in struct
> > > of_pci_range
> > >
> > > On Thu, Jul 30, 2015 at 01:52:13PM +0000, Gabriele Paoloni wrote:
> > > >
> > > >
> > > > > -----Original Message-----
> > > > > From: linux-pci-owner@vger.kernel.org [mailto:linux-pci-
> > > > > owner@vger.kernel.org] On Behalf Of Rob Herring
> > > > > Sent: Thursday, July 30, 2015 2:43 PM
> > > > > To: Gabriele Paoloni
> > > > > Cc: Bjorn Helgaas; arnd@arndb.de; lorenzo.pieralisi@arm.com;
> > > Wangzhou
> > > > > (B); robh+dt@kernel.org; james.morse@arm.com;
> Liviu.Dudau@arm.com;
> > > > > linux-pci@vger.kernel.org; linux-arm-kernel@lists.infradead.org;
> > > > > devicetree@vger.kernel.org; Yuanzhichang; Zhudacai; zhangjukuo;
> > > > > qiuzhenfa; Liguozhu (Kenneth)
> > > > > Subject: Re: [PATCH v6] PCI: Store PCIe bus address in struct
> > > > > of_pci_range
> > > > >
> > > > > On Wed, Jul 29, 2015 at 2:44 PM, Gabriele Paoloni
> > > > > <gabriele.paoloni@huawei.com> wrote:
> > > > > > Hi Bjorn
> > > > > >
> > > > > > Many Thanks for your reply
> > > > > >
> > > > > > I have commented back inline with resolutions from my side.
> > > > > >
> > > > > > If you're ok with them I'll send it out a new version in the
> > > > > appropriate patchset
> > > > > >
> > > > > > Cheers
> > > > > >
> > > > > > Gab
> > > > > >
> > > > > >> -----Original Message-----
> > > > > >> From: Bjorn Helgaas [mailto:bhelgaas@google.com]
> > > > > >> Sent: Wednesday, July 29, 2015 6:21 PM
> > > > > >> To: Gabriele Paoloni
> > > > > >> Cc: arnd@arndb.de; lorenzo.pieralisi@arm.com; Wangzhou (B);
> > > > > >> robh+dt@kernel.org; james.morse@arm.com; Liviu.Dudau@arm.com;
> > > linux-
> > > > > >> pci@vger.kernel.org; linux-arm-kernel@lists.infradead.org;
> > > > > >> devicetree@vger.kernel.org; Yuanzhichang; Zhudacai;
> zhangjukuo;
> > > > > >> qiuzhenfa; Liguozhu (Kenneth)
> > > > > >> Subject: Re: [PATCH v6] PCI: Store PCIe bus address in struct
> > > > > >> of_pci_range
> > > > > >>
> > > > > >> Hi Gabriele,
> > > > > >>
> > > > > >> As far as I can tell, this is not specific to PCIe, so please
> > > use
> > > > > "PCI"
> > > > > >> in
> > > > > >> the subject as a generic term that includes both PCI and PCIe.
> > > > > >
> > > > > > sure agreed
> > > > > >
> > > > > >>
> > > > > >> On Mon, Jul 27, 2015 at 11:17:03PM +0800, Gabriele Paoloni
> wrote:
> > > > > >> > From: gabriele paoloni <gabriele.paoloni@huawei.com>
> > > > > >> >
> > > > > >> >     This patch is needed port PCIe designware to new DT
> > > parsing
> > > > > API
> > > > > >> >     As discussed in
> > > > > >> >     http://lists.infradead.org/pipermail/linux-arm-
> > > kernel/2015-
> > > > > >> January/317743.html
> > > > > >> >     in designware we have a problem as the PCI addresses in
> > > the
> > > > > PCIe
> > > > > >> controller
> > > > > >> >     address space are required in order to perform correct
> HW
> > > > > >> operation.
> > > > > >> >
> > > > > >> >     In order to solve this problem commit f4c55c5a3 "PCI:
> > > > > designware:
> > > > > >> >     Program ATU with untranslated address" added code to
> read
> > > the
> > > > > >> PCIe
> > > > > >>
> > > > > >> Conventional reference is 12-char SHA1, like this:
> > > > > >>
> > > > > >>   f4c55c5a3f7f ("PCI: designware: Program ATU with
> untranslated
> > > > > >> address")
> > > > > >
> > > > > > Agreed, will change this
> > > > > >
> > > > > >>
> > > > > >> >     controller start address directly from the DT ranges.
> > > > > >> >
> > > > > >> >     In the new DT parsing API
> > > of_pci_get_host_bridge_resources()
> > > > > >> hides the
> > > > > >> >     DT parser from the host controller drivers, so it is
> not
> > > > > possible
> > > > > >> >     for drivers to parse values directly from the DT.
> > > > > >> >
> > > > > >> >     In http://www.spinics.net/lists/linux-pci/msg42540.html
> we
> > > > > >> already tried
> > > > > >> >     to use the new DT parsing API but there is a bug
> > > (obviously)
> > > > > in
> > > > > >> setting
> > > > > >> >     the <*>_mod_base addresses
> > > > > >> >     Applying this patch we can easily set "<*>_mod_base =
> win-
> > > > > >> >__res.start"
> > > > > >>
> > > > > >> By itself, this patch adds something.  It would help me
> > > understand
> > > > > it
> > > > > >> if
> > > > > >> the *user* of this new something were in the same patch
> series.
> > > > > >
> > > > > > the user is: "[PATCH v5 2/5] PCI: designware: Add ARM64
> support"
> > > > > > I will ask Zhou Wang to include this patch in his patchset
> > > > > >
> > > > > >
> > > > > >>
> > > > > >> >     This patch adds a new field in "struct of_pci_range" to
> > > store
> > > > > the
> > > > > >> >     pci bus start address; it fills the field in
> > > > > >> of_pci_range_parser_one();
> > > > > >> >     in of_pci_get_host_bridge_resources() it retrieves the
> > > > > resource
> > > > > >> entry
> > > > > >> >     after it is created and added to the resource list and
> > > uses
> > > > > >> >     entry->__res.start to store the pci controller address
> > > > > >>
> > > > > >> struct of_pci_range is starting to get confusing to non-OF
> folks
> > > > > like
> > > > > >> me.
> > > > > >> It now contains:
> > > > > >>
> > > > > >>   u32 pci_space;
> > > > > >>   u64 pci_addr;
> > > > > >>   u64 cpu_addr;
> > > > > >>   u64 bus_addr;
> > > > > >>
> > > > > >> Can you explain what all these things mean, and maybe even
> add
> > > one-
> > > > > line
> > > > > >> comments to the structure?
> > > > > >
> > > > > > sure I can add comments inline in the code
> > > > > >
> > > > > >>
> > > > > >> pci_space: The only uses I see are to determine whether to
> print
> > > > > >> "Prefetch".  I don't see any real functionality that uses
> this.
> > > > > >
> > > > > > Looking at the code I agree. it's seems to be used only in
> > > powerpc
> > > > > > and microblaze to print out.
> > > > > > However from my understanding pci_space is the phys.hi field
> of
> > > the
> > > > > > ranges property: it defines the properties of the address
> space
> > > > > associated
> > > > > > to the PCI address. if you're curious you can find a nice and
> > > quick
> > > > > to read
> > > > > > "guide" in http://devicetree.org/MPC5200:PCI
> > > > > >
> > > > > >>
> > > > > >> pci_addr: I assume this is a PCI bus address, like what you
> > > would
> > > > > see
> > > > > >> if
> > > > > >> you put an analyzer on the bus/link.  This address could go
> in a
> > > BAR.
> > > > > >
> > > > > > Yes, this is the PCI start address of the range: phys.mid +
> > > phys.low
> > > > > in the
> > > > > > guide mentioned above
> > > > > >
> > > > > >>
> > > > > >> cpu_addr: I assume this is a CPU physical address, like what
> you
> > > > > would
> > > > > >> see
> > > > > >> in /proc/iomem and what you would pass to ioremap().
> > > > > >
> > > > > > Yes correct
> > > > > >
> > > > > >>
> > > > > >> bus_addr: ?
> > > > > >>
> > > > > >
> > > > > > According to the guide above, this is the address into which
> the
> > > > > pci_address
> > > > > > get translated to and that is passed to the root complex.
> Between
> > > the
> > > > > root
> > > > > > complex and the CPU there can be intermediate translation
> layers:
> > > see
> > > > > that to
> > > > > > get pci_address we call "of_translate_address"; this will
> apply
> > > all
> > > > > the
> > > > > > translation layers (ranges in the DT) that it finds till it
> comes
> > > to
> > > > > the root
> > > > > > node of the DT (thus retrieving the CPU address).
> > > > > > Now said that, for designware we need the first translated PCI
> > > > > address, that we call
> > > > >
> > > > > I think you mean "translated CPU address." The flow of addresses
> > > looks
> > > > > like this:
> > > > >
> > > > > CPU -> CPU bus address -> bus fabric address decoding -> bus
> > > address
> > > > > -> DW PCI -> PCI address
> > > > >
> > > > > This is quite common that an IP block does not see the full
> address.
> > > > > It is unusual that the IP block needs to know its full address
> on
> > > the
> > > > > slave side. It is quite common for the master side and the
> kernel
> > > > > deals with that all the time with DMA mapping
> > > > >
> > > > > > here bus_addr after Rob Herring suggested the name...honestly
> I
> > > > > cannot think of a
> > > > > > different name
> > > > >
> > > > > Thinking about this some more, is this really a translation
> versus
> > > > > just a stripping of high bits? Does the DW IP have less than 32-
> > > bits
> > > > > address? If so, we could express differently and just mask the
> CPU
> > > > > address within the driver instead.
> > > >
> > > > I don’t think we should rely on [CPU] addresses...what if the
> > > intermediate
> > > > translation layer changes the lower significant bits of the "bus
> > > address"
> > > > to translate into a cpu address?
> > >
> > > Is it really a possiblity that the lower bits could be changed?
> >
> > I've checked all the current deignware users DTs except "pci-
> layerscape"
> > that I could not find:
> > spear1310.dtsi
> > spear1340.dtsi
> > dra7.dtsi
> > imx6qdl.dtsi
> > imx6sx.dtsi
> > keystone.dtsi
> > exynos5440.dtsi
> >
> > None of them modifies the lower bits. To be more precise the only guy
> > that provides another translation layer is "dra7.dtsi":
> > axi0
> > http://lxr.free-electrons.com/source/arch/arm/boot/dts/dra7.dtsi#L207
> >
> > axi1
> > http://lxr.free-electrons.com/source/arch/arm/boot/dts/dra7.dtsi#L241
> >
> > For this case masking the top 4bits (bits28 to 31) should make the job.
> >
> > Speaking in general terms so far I've always seen linear mappings
> > that differ by bitmask offset, however linear does not mean that you
> > cannot affect the lower bits: e.g. <0x0> to <0x0 + size> can map to
> > <0x0000cafe to 0x0000cafe + size>, but I guess that for HW design
> reasons
> > it is much easier to remap directly using a bitmask...but I was not
> sure
> > and I didn't think about the problems that can arise with ACPI.
> 
> As I said, it wouldn't make sense for the bits that comprise the
> offset into the window to change, so the mapping only has to be linear
> inside the window.
> 
> For the bits outside the window offset, I don't know enough to say
> whether masking is sufficient or safe.
> 
> > If you think the bitmask is Ok then I can directly define it in
> > designware and we can drop this patch.
> 
> It's OK by me, but I know nothing about the actual hardware involved.
> For the DesignWare question, I think you would just have to convince
> Jingoo and Pratyush (cc'd).

Yes actually looking at
http://lxr.free-electrons.com/source/arch/arm/boot/dts/spear1310.dtsi#L99
I can see that pci_addr=0 is mapped to bus_addr 0x80020000, so clearing 
the top 4 bits would alter the behaviour of the current designware 
SW framework...now I don't know if DW needs only the low 28b of that
value or not....

Jingoo, Pratyush what do you think?

> 
> > > I think we're talking about MMIO here, and a bridge has an MMIO
> > > window.  A window is a range of CPU physical addresses that the
> > > bridge forwards to PCI.  The PCI core assumes that a CPU physical
> > > address range is linearly mapped to PCI bus addresses, i.e., if the
> > > window base is X and maps to PCI address Y, then as long as X+n is
> > > inside the window, it must map to Y+n.
> > >
> > > That means the low-order bits (the ones that are the offset into the
> > > window) cannot change.
> > > --
> > > To unsubscribe from this list: send the line "unsubscribe linux-pci"
> in
> > > the body of a message to majordomo@vger.kernel.org
> > > More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 73+ messages in thread
* RE: [PATCH v6] PCI: Store PCIe bus address in struct of_pci_range
@ 2015-07-30 17:34                       ` Gabriele Paoloni
  0 siblings, 0 replies; 73+ messages in thread
From: Gabriele Paoloni @ 2015-07-30 17:34 UTC (permalink / raw)
  To: Bjorn Helgaas, Gabriele Paoloni
  Cc: Rob Herring, arnd, lorenzo.pieralisi, Wangzhou (B),
	robh+dt, james.morse, Liviu.Dudau, linux-pci, linux-arm-kernel,
	devicetree, Yuanzhichang, Zhudacai, zhangjukuo, qiuzhenfa,
	Liguozhu (Kenneth),
	Jingoo Han, Pratyush Anand

DQoNCj4gLS0tLS1PcmlnaW5hbCBNZXNzYWdlLS0tLS0NCj4gRnJvbTogQmpvcm4gSGVsZ2FhcyBb
bWFpbHRvOmJoZWxnYWFzQGdvb2dsZS5jb21dDQo+IFNlbnQ6IDMwIEp1bHkgMjAxNSAxODoxNQ0K
PiBUbzogR2FicmllbGUgUGFvbG9uaQ0KPiBDYzogUm9iIEhlcnJpbmc7IGFybmRAYXJuZGIuZGU7
IGxvcmVuem8ucGllcmFsaXNpQGFybS5jb207IFdhbmd6aG91IChCKTsNCj4gcm9iaCtkdEBrZXJu
ZWwub3JnOyBqYW1lcy5tb3JzZUBhcm0uY29tOyBMaXZpdS5EdWRhdUBhcm0uY29tOyBsaW51eC0N
Cj4gcGNpQHZnZXIua2VybmVsLm9yZzsgbGludXgtYXJtLWtlcm5lbEBsaXN0cy5pbmZyYWRlYWQu
b3JnOw0KPiBkZXZpY2V0cmVlQHZnZXIua2VybmVsLm9yZzsgWXVhbnpoaWNoYW5nOyBaaHVkYWNh
aTsgemhhbmdqdWt1bzsNCj4gcWl1emhlbmZhOyBMaWd1b3podSAoS2VubmV0aCk7IEppbmdvbyBI
YW47IFByYXR5dXNoIEFuYW5kDQo+IFN1YmplY3Q6IFJlOiBbUEFUQ0ggdjZdIFBDSTogU3RvcmUg
UENJZSBidXMgYWRkcmVzcyBpbiBzdHJ1Y3QNCj4gb2ZfcGNpX3JhbmdlDQo+IA0KPiBbK2NjIEpp
bmdvbywgUHJhdHl1c2hdDQo+IA0KPiBPbiBUaHUsIEp1bCAzMCwgMjAxNSBhdCAwNDo1MDo1NVBN
ICswMDAwLCBHYWJyaWVsZSBQYW9sb25pIHdyb3RlOg0KPiA+ID4gLS0tLS1PcmlnaW5hbCBNZXNz
YWdlLS0tLS0NCj4gPiA+IEZyb206IGxpbnV4LXBjaS1vd25lckB2Z2VyLmtlcm5lbC5vcmcgW21h
aWx0bzpsaW51eC1wY2ktDQo+ID4gPiBvd25lckB2Z2VyLmtlcm5lbC5vcmddIE9uIEJlaGFsZiBP
ZiBCam9ybiBIZWxnYWFzDQo+ID4gPiBTZW50OiBUaHVyc2RheSwgSnVseSAzMCwgMjAxNSA1OjE1
IFBNDQo+ID4gPiBUbzogR2FicmllbGUgUGFvbG9uaQ0KPiA+ID4gQ2M6IFJvYiBIZXJyaW5nOyBh
cm5kQGFybmRiLmRlOyBsb3JlbnpvLnBpZXJhbGlzaUBhcm0uY29tOyBXYW5nemhvdQ0KPiAoQik7
DQo+ID4gPiByb2JoK2R0QGtlcm5lbC5vcmc7IGphbWVzLm1vcnNlQGFybS5jb207IExpdml1LkR1
ZGF1QGFybS5jb207IGxpbnV4LQ0KPiA+ID4gcGNpQHZnZXIua2VybmVsLm9yZzsgbGludXgtYXJt
LWtlcm5lbEBsaXN0cy5pbmZyYWRlYWQub3JnOw0KPiA+ID4gZGV2aWNldHJlZUB2Z2VyLmtlcm5l
bC5vcmc7IFl1YW56aGljaGFuZzsgWmh1ZGFjYWk7IHpoYW5nanVrdW87DQo+ID4gPiBxaXV6aGVu
ZmE7IExpZ3Vvemh1IChLZW5uZXRoKQ0KPiA+ID4gU3ViamVjdDogUmU6IFtQQVRDSCB2Nl0gUENJ
OiBTdG9yZSBQQ0llIGJ1cyBhZGRyZXNzIGluIHN0cnVjdA0KPiA+ID4gb2ZfcGNpX3JhbmdlDQo+
ID4gPg0KPiA+ID4gT24gVGh1LCBKdWwgMzAsIDIwMTUgYXQgMDE6NTI6MTNQTSArMDAwMCwgR2Fi
cmllbGUgUGFvbG9uaSB3cm90ZToNCj4gPiA+ID4NCj4gPiA+ID4NCj4gPiA+ID4gPiAtLS0tLU9y
aWdpbmFsIE1lc3NhZ2UtLS0tLQ0KPiA+ID4gPiA+IEZyb206IGxpbnV4LXBjaS1vd25lckB2Z2Vy
Lmtlcm5lbC5vcmcgW21haWx0bzpsaW51eC1wY2ktDQo+ID4gPiA+ID4gb3duZXJAdmdlci5rZXJu
ZWwub3JnXSBPbiBCZWhhbGYgT2YgUm9iIEhlcnJpbmcNCj4gPiA+ID4gPiBTZW50OiBUaHVyc2Rh
eSwgSnVseSAzMCwgMjAxNSAyOjQzIFBNDQo+ID4gPiA+ID4gVG86IEdhYnJpZWxlIFBhb2xvbmkN
Cj4gPiA+ID4gPiBDYzogQmpvcm4gSGVsZ2FhczsgYXJuZEBhcm5kYi5kZTsgbG9yZW56by5waWVy
YWxpc2lAYXJtLmNvbTsNCj4gPiA+IFdhbmd6aG91DQo+ID4gPiA+ID4gKEIpOyByb2JoK2R0QGtl
cm5lbC5vcmc7IGphbWVzLm1vcnNlQGFybS5jb207DQo+IExpdml1LkR1ZGF1QGFybS5jb207DQo+
ID4gPiA+ID4gbGludXgtcGNpQHZnZXIua2VybmVsLm9yZzsgbGludXgtYXJtLWtlcm5lbEBsaXN0
cy5pbmZyYWRlYWQub3JnOw0KPiA+ID4gPiA+IGRldmljZXRyZWVAdmdlci5rZXJuZWwub3JnOyBZ
dWFuemhpY2hhbmc7IFpodWRhY2FpOyB6aGFuZ2p1a3VvOw0KPiA+ID4gPiA+IHFpdXpoZW5mYTsg
TGlndW96aHUgKEtlbm5ldGgpDQo+ID4gPiA+ID4gU3ViamVjdDogUmU6IFtQQVRDSCB2Nl0gUENJ
OiBTdG9yZSBQQ0llIGJ1cyBhZGRyZXNzIGluIHN0cnVjdA0KPiA+ID4gPiA+IG9mX3BjaV9yYW5n
ZQ0KPiA+ID4gPiA+DQo+ID4gPiA+ID4gT24gV2VkLCBKdWwgMjksIDIwMTUgYXQgMjo0NCBQTSwg
R2FicmllbGUgUGFvbG9uaQ0KPiA+ID4gPiA+IDxnYWJyaWVsZS5wYW9sb25pQGh1YXdlaS5jb20+
IHdyb3RlOg0KPiA+ID4gPiA+ID4gSGkgQmpvcm4NCj4gPiA+ID4gPiA+DQo+ID4gPiA+ID4gPiBN
YW55IFRoYW5rcyBmb3IgeW91ciByZXBseQ0KPiA+ID4gPiA+ID4NCj4gPiA+ID4gPiA+IEkgaGF2
ZSBjb21tZW50ZWQgYmFjayBpbmxpbmUgd2l0aCByZXNvbHV0aW9ucyBmcm9tIG15IHNpZGUuDQo+
ID4gPiA+ID4gPg0KPiA+ID4gPiA+ID4gSWYgeW91J3JlIG9rIHdpdGggdGhlbSBJJ2xsIHNlbmQg
aXQgb3V0IGEgbmV3IHZlcnNpb24gaW4gdGhlDQo+ID4gPiA+ID4gYXBwcm9wcmlhdGUgcGF0Y2hz
ZXQNCj4gPiA+ID4gPiA+DQo+ID4gPiA+ID4gPiBDaGVlcnMNCj4gPiA+ID4gPiA+DQo+ID4gPiA+
ID4gPiBHYWINCj4gPiA+ID4gPiA+DQo+ID4gPiA+ID4gPj4gLS0tLS1PcmlnaW5hbCBNZXNzYWdl
LS0tLS0NCj4gPiA+ID4gPiA+PiBGcm9tOiBCam9ybiBIZWxnYWFzIFttYWlsdG86YmhlbGdhYXNA
Z29vZ2xlLmNvbV0NCj4gPiA+ID4gPiA+PiBTZW50OiBXZWRuZXNkYXksIEp1bHkgMjksIDIwMTUg
NjoyMSBQTQ0KPiA+ID4gPiA+ID4+IFRvOiBHYWJyaWVsZSBQYW9sb25pDQo+ID4gPiA+ID4gPj4g
Q2M6IGFybmRAYXJuZGIuZGU7IGxvcmVuem8ucGllcmFsaXNpQGFybS5jb207IFdhbmd6aG91IChC
KTsNCj4gPiA+ID4gPiA+PiByb2JoK2R0QGtlcm5lbC5vcmc7IGphbWVzLm1vcnNlQGFybS5jb207
IExpdml1LkR1ZGF1QGFybS5jb207DQo+ID4gPiBsaW51eC0NCj4gPiA+ID4gPiA+PiBwY2lAdmdl
ci5rZXJuZWwub3JnOyBsaW51eC1hcm0ta2VybmVsQGxpc3RzLmluZnJhZGVhZC5vcmc7DQo+ID4g
PiA+ID4gPj4gZGV2aWNldHJlZUB2Z2VyLmtlcm5lbC5vcmc7IFl1YW56aGljaGFuZzsgWmh1ZGFj
YWk7DQo+IHpoYW5nanVrdW87DQo+ID4gPiA+ID4gPj4gcWl1emhlbmZhOyBMaWd1b3podSAoS2Vu
bmV0aCkNCj4gPiA+ID4gPiA+PiBTdWJqZWN0OiBSZTogW1BBVENIIHY2XSBQQ0k6IFN0b3JlIFBD
SWUgYnVzIGFkZHJlc3MgaW4gc3RydWN0DQo+ID4gPiA+ID4gPj4gb2ZfcGNpX3JhbmdlDQo+ID4g
PiA+ID4gPj4NCj4gPiA+ID4gPiA+PiBIaSBHYWJyaWVsZSwNCj4gPiA+ID4gPiA+Pg0KPiA+ID4g
PiA+ID4+IEFzIGZhciBhcyBJIGNhbiB0ZWxsLCB0aGlzIGlzIG5vdCBzcGVjaWZpYyB0byBQQ0ll
LCBzbyBwbGVhc2UNCj4gPiA+IHVzZQ0KPiA+ID4gPiA+ICJQQ0kiDQo+ID4gPiA+ID4gPj4gaW4N
Cj4gPiA+ID4gPiA+PiB0aGUgc3ViamVjdCBhcyBhIGdlbmVyaWMgdGVybSB0aGF0IGluY2x1ZGVz
IGJvdGggUENJIGFuZCBQQ0llLg0KPiA+ID4gPiA+ID4NCj4gPiA+ID4gPiA+IHN1cmUgYWdyZWVk
DQo+ID4gPiA+ID4gPg0KPiA+ID4gPiA+ID4+DQo+ID4gPiA+ID4gPj4gT24gTW9uLCBKdWwgMjcs
IDIwMTUgYXQgMTE6MTc6MDNQTSArMDgwMCwgR2FicmllbGUgUGFvbG9uaQ0KPiB3cm90ZToNCj4g
PiA+ID4gPiA+PiA+IEZyb206IGdhYnJpZWxlIHBhb2xvbmkgPGdhYnJpZWxlLnBhb2xvbmlAaHVh
d2VpLmNvbT4NCj4gPiA+ID4gPiA+PiA+DQo+ID4gPiA+ID4gPj4gPiAgICAgVGhpcyBwYXRjaCBp
cyBuZWVkZWQgcG9ydCBQQ0llIGRlc2lnbndhcmUgdG8gbmV3IERUDQo+ID4gPiBwYXJzaW5nDQo+
ID4gPiA+ID4gQVBJDQo+ID4gPiA+ID4gPj4gPiAgICAgQXMgZGlzY3Vzc2VkIGluDQo+ID4gPiA+
ID4gPj4gPiAgICAgaHR0cDovL2xpc3RzLmluZnJhZGVhZC5vcmcvcGlwZXJtYWlsL2xpbnV4LWFy
bS0NCj4gPiA+IGtlcm5lbC8yMDE1LQ0KPiA+ID4gPiA+ID4+IEphbnVhcnkvMzE3NzQzLmh0bWwN
Cj4gPiA+ID4gPiA+PiA+ICAgICBpbiBkZXNpZ253YXJlIHdlIGhhdmUgYSBwcm9ibGVtIGFzIHRo
ZSBQQ0kgYWRkcmVzc2VzIGluDQo+ID4gPiB0aGUNCj4gPiA+ID4gPiBQQ0llDQo+ID4gPiA+ID4g
Pj4gY29udHJvbGxlcg0KPiA+ID4gPiA+ID4+ID4gICAgIGFkZHJlc3Mgc3BhY2UgYXJlIHJlcXVp
cmVkIGluIG9yZGVyIHRvIHBlcmZvcm0gY29ycmVjdA0KPiBIVw0KPiA+ID4gPiA+ID4+IG9wZXJh
dGlvbi4NCj4gPiA+ID4gPiA+PiA+DQo+ID4gPiA+ID4gPj4gPiAgICAgSW4gb3JkZXIgdG8gc29s
dmUgdGhpcyBwcm9ibGVtIGNvbW1pdCBmNGM1NWM1YTMgIlBDSToNCj4gPiA+ID4gPiBkZXNpZ253
YXJlOg0KPiA+ID4gPiA+ID4+ID4gICAgIFByb2dyYW0gQVRVIHdpdGggdW50cmFuc2xhdGVkIGFk
ZHJlc3MiIGFkZGVkIGNvZGUgdG8NCj4gcmVhZA0KPiA+ID4gdGhlDQo+ID4gPiA+ID4gPj4gUENJ
ZQ0KPiA+ID4gPiA+ID4+DQo+ID4gPiA+ID4gPj4gQ29udmVudGlvbmFsIHJlZmVyZW5jZSBpcyAx
Mi1jaGFyIFNIQTEsIGxpa2UgdGhpczoNCj4gPiA+ID4gPiA+Pg0KPiA+ID4gPiA+ID4+ICAgZjRj
NTVjNWEzZjdmICgiUENJOiBkZXNpZ253YXJlOiBQcm9ncmFtIEFUVSB3aXRoDQo+IHVudHJhbnNs
YXRlZA0KPiA+ID4gPiA+ID4+IGFkZHJlc3MiKQ0KPiA+ID4gPiA+ID4NCj4gPiA+ID4gPiA+IEFn
cmVlZCwgd2lsbCBjaGFuZ2UgdGhpcw0KPiA+ID4gPiA+ID4NCj4gPiA+ID4gPiA+Pg0KPiA+ID4g
PiA+ID4+ID4gICAgIGNvbnRyb2xsZXIgc3RhcnQgYWRkcmVzcyBkaXJlY3RseSBmcm9tIHRoZSBE
VCByYW5nZXMuDQo+ID4gPiA+ID4gPj4gPg0KPiA+ID4gPiA+ID4+ID4gICAgIEluIHRoZSBuZXcg
RFQgcGFyc2luZyBBUEkNCj4gPiA+IG9mX3BjaV9nZXRfaG9zdF9icmlkZ2VfcmVzb3VyY2VzKCkN
Cj4gPiA+ID4gPiA+PiBoaWRlcyB0aGUNCj4gPiA+ID4gPiA+PiA+ICAgICBEVCBwYXJzZXIgZnJv
bSB0aGUgaG9zdCBjb250cm9sbGVyIGRyaXZlcnMsIHNvIGl0IGlzDQo+IG5vdA0KPiA+ID4gPiA+
IHBvc3NpYmxlDQo+ID4gPiA+ID4gPj4gPiAgICAgZm9yIGRyaXZlcnMgdG8gcGFyc2UgdmFsdWVz
IGRpcmVjdGx5IGZyb20gdGhlIERULg0KPiA+ID4gPiA+ID4+ID4NCj4gPiA+ID4gPiA+PiA+ICAg
ICBJbiBodHRwOi8vd3d3LnNwaW5pY3MubmV0L2xpc3RzL2xpbnV4LXBjaS9tc2c0MjU0MC5odG1s
DQo+IHdlDQo+ID4gPiA+ID4gPj4gYWxyZWFkeSB0cmllZA0KPiA+ID4gPiA+ID4+ID4gICAgIHRv
IHVzZSB0aGUgbmV3IERUIHBhcnNpbmcgQVBJIGJ1dCB0aGVyZSBpcyBhIGJ1Zw0KPiA+ID4gKG9i
dmlvdXNseSkNCj4gPiA+ID4gPiBpbg0KPiA+ID4gPiA+ID4+IHNldHRpbmcNCj4gPiA+ID4gPiA+
PiA+ICAgICB0aGUgPCo+X21vZF9iYXNlIGFkZHJlc3Nlcw0KPiA+ID4gPiA+ID4+ID4gICAgIEFw
cGx5aW5nIHRoaXMgcGF0Y2ggd2UgY2FuIGVhc2lseSBzZXQgIjwqPl9tb2RfYmFzZSA9DQo+IHdp
bi0NCj4gPiA+ID4gPiA+PiA+X19yZXMuc3RhcnQiDQo+ID4gPiA+ID4gPj4NCj4gPiA+ID4gPiA+
PiBCeSBpdHNlbGYsIHRoaXMgcGF0Y2ggYWRkcyBzb21ldGhpbmcuICBJdCB3b3VsZCBoZWxwIG1l
DQo+ID4gPiB1bmRlcnN0YW5kDQo+ID4gPiA+ID4gaXQNCj4gPiA+ID4gPiA+PiBpZg0KPiA+ID4g
PiA+ID4+IHRoZSAqdXNlciogb2YgdGhpcyBuZXcgc29tZXRoaW5nIHdlcmUgaW4gdGhlIHNhbWUg
cGF0Y2gNCj4gc2VyaWVzLg0KPiA+ID4gPiA+ID4NCj4gPiA+ID4gPiA+IHRoZSB1c2VyIGlzOiAi
W1BBVENIIHY1IDIvNV0gUENJOiBkZXNpZ253YXJlOiBBZGQgQVJNNjQNCj4gc3VwcG9ydCINCj4g
PiA+ID4gPiA+IEkgd2lsbCBhc2sgWmhvdSBXYW5nIHRvIGluY2x1ZGUgdGhpcyBwYXRjaCBpbiBo
aXMgcGF0Y2hzZXQNCj4gPiA+ID4gPiA+DQo+ID4gPiA+ID4gPg0KPiA+ID4gPiA+ID4+DQo+ID4g
PiA+ID4gPj4gPiAgICAgVGhpcyBwYXRjaCBhZGRzIGEgbmV3IGZpZWxkIGluICJzdHJ1Y3Qgb2Zf
cGNpX3JhbmdlIiB0bw0KPiA+ID4gc3RvcmUNCj4gPiA+ID4gPiB0aGUNCj4gPiA+ID4gPiA+PiA+
ICAgICBwY2kgYnVzIHN0YXJ0IGFkZHJlc3M7IGl0IGZpbGxzIHRoZSBmaWVsZCBpbg0KPiA+ID4g
PiA+ID4+IG9mX3BjaV9yYW5nZV9wYXJzZXJfb25lKCk7DQo+ID4gPiA+ID4gPj4gPiAgICAgaW4g
b2ZfcGNpX2dldF9ob3N0X2JyaWRnZV9yZXNvdXJjZXMoKSBpdCByZXRyaWV2ZXMgdGhlDQo+ID4g
PiA+ID4gcmVzb3VyY2UNCj4gPiA+ID4gPiA+PiBlbnRyeQ0KPiA+ID4gPiA+ID4+ID4gICAgIGFm
dGVyIGl0IGlzIGNyZWF0ZWQgYW5kIGFkZGVkIHRvIHRoZSByZXNvdXJjZSBsaXN0IGFuZA0KPiA+
ID4gdXNlcw0KPiA+ID4gPiA+ID4+ID4gICAgIGVudHJ5LT5fX3Jlcy5zdGFydCB0byBzdG9yZSB0
aGUgcGNpIGNvbnRyb2xsZXIgYWRkcmVzcw0KPiA+ID4gPiA+ID4+DQo+ID4gPiA+ID4gPj4gc3Ry
dWN0IG9mX3BjaV9yYW5nZSBpcyBzdGFydGluZyB0byBnZXQgY29uZnVzaW5nIHRvIG5vbi1PRg0K
PiBmb2xrcw0KPiA+ID4gPiA+IGxpa2UNCj4gPiA+ID4gPiA+PiBtZS4NCj4gPiA+ID4gPiA+PiBJ
dCBub3cgY29udGFpbnM6DQo+ID4gPiA+ID4gPj4NCj4gPiA+ID4gPiA+PiAgIHUzMiBwY2lfc3Bh
Y2U7DQo+ID4gPiA+ID4gPj4gICB1NjQgcGNpX2FkZHI7DQo+ID4gPiA+ID4gPj4gICB1NjQgY3B1
X2FkZHI7DQo+ID4gPiA+ID4gPj4gICB1NjQgYnVzX2FkZHI7DQo+ID4gPiA+ID4gPj4NCj4gPiA+
ID4gPiA+PiBDYW4geW91IGV4cGxhaW4gd2hhdCBhbGwgdGhlc2UgdGhpbmdzIG1lYW4sIGFuZCBt
YXliZSBldmVuDQo+IGFkZA0KPiA+ID4gb25lLQ0KPiA+ID4gPiA+IGxpbmUNCj4gPiA+ID4gPiA+
PiBjb21tZW50cyB0byB0aGUgc3RydWN0dXJlPw0KPiA+ID4gPiA+ID4NCj4gPiA+ID4gPiA+IHN1
cmUgSSBjYW4gYWRkIGNvbW1lbnRzIGlubGluZSBpbiB0aGUgY29kZQ0KPiA+ID4gPiA+ID4NCj4g
PiA+ID4gPiA+Pg0KPiA+ID4gPiA+ID4+IHBjaV9zcGFjZTogVGhlIG9ubHkgdXNlcyBJIHNlZSBh
cmUgdG8gZGV0ZXJtaW5lIHdoZXRoZXIgdG8NCj4gcHJpbnQNCj4gPiA+ID4gPiA+PiAiUHJlZmV0
Y2giLiAgSSBkb24ndCBzZWUgYW55IHJlYWwgZnVuY3Rpb25hbGl0eSB0aGF0IHVzZXMNCj4gdGhp
cy4NCj4gPiA+ID4gPiA+DQo+ID4gPiA+ID4gPiBMb29raW5nIGF0IHRoZSBjb2RlIEkgYWdyZWUu
IGl0J3Mgc2VlbXMgdG8gYmUgdXNlZCBvbmx5IGluDQo+ID4gPiBwb3dlcnBjDQo+ID4gPiA+ID4g
PiBhbmQgbWljcm9ibGF6ZSB0byBwcmludCBvdXQuDQo+ID4gPiA+ID4gPiBIb3dldmVyIGZyb20g
bXkgdW5kZXJzdGFuZGluZyBwY2lfc3BhY2UgaXMgdGhlIHBoeXMuaGkgZmllbGQNCj4gb2YNCj4g
PiA+IHRoZQ0KPiA+ID4gPiA+ID4gcmFuZ2VzIHByb3BlcnR5OiBpdCBkZWZpbmVzIHRoZSBwcm9w
ZXJ0aWVzIG9mIHRoZSBhZGRyZXNzDQo+IHNwYWNlDQo+ID4gPiA+ID4gYXNzb2NpYXRlZA0KPiA+
ID4gPiA+ID4gdG8gdGhlIFBDSSBhZGRyZXNzLiBpZiB5b3UncmUgY3VyaW91cyB5b3UgY2FuIGZp
bmQgYSBuaWNlIGFuZA0KPiA+ID4gcXVpY2sNCj4gPiA+ID4gPiB0byByZWFkDQo+ID4gPiA+ID4g
PiAiZ3VpZGUiIGluIGh0dHA6Ly9kZXZpY2V0cmVlLm9yZy9NUEM1MjAwOlBDSQ0KPiA+ID4gPiA+
ID4NCj4gPiA+ID4gPiA+Pg0KPiA+ID4gPiA+ID4+IHBjaV9hZGRyOiBJIGFzc3VtZSB0aGlzIGlz
IGEgUENJIGJ1cyBhZGRyZXNzLCBsaWtlIHdoYXQgeW91DQo+ID4gPiB3b3VsZA0KPiA+ID4gPiA+
IHNlZQ0KPiA+ID4gPiA+ID4+IGlmDQo+ID4gPiA+ID4gPj4geW91IHB1dCBhbiBhbmFseXplciBv
biB0aGUgYnVzL2xpbmsuICBUaGlzIGFkZHJlc3MgY291bGQgZ28NCj4gaW4gYQ0KPiA+ID4gQkFS
Lg0KPiA+ID4gPiA+ID4NCj4gPiA+ID4gPiA+IFllcywgdGhpcyBpcyB0aGUgUENJIHN0YXJ0IGFk
ZHJlc3Mgb2YgdGhlIHJhbmdlOiBwaHlzLm1pZCArDQo+ID4gPiBwaHlzLmxvdw0KPiA+ID4gPiA+
IGluIHRoZQ0KPiA+ID4gPiA+ID4gZ3VpZGUgbWVudGlvbmVkIGFib3ZlDQo+ID4gPiA+ID4gPg0K
PiA+ID4gPiA+ID4+DQo+ID4gPiA+ID4gPj4gY3B1X2FkZHI6IEkgYXNzdW1lIHRoaXMgaXMgYSBD
UFUgcGh5c2ljYWwgYWRkcmVzcywgbGlrZSB3aGF0DQo+IHlvdQ0KPiA+ID4gPiA+IHdvdWxkDQo+
ID4gPiA+ID4gPj4gc2VlDQo+ID4gPiA+ID4gPj4gaW4gL3Byb2MvaW9tZW0gYW5kIHdoYXQgeW91
IHdvdWxkIHBhc3MgdG8gaW9yZW1hcCgpLg0KPiA+ID4gPiA+ID4NCj4gPiA+ID4gPiA+IFllcyBj
b3JyZWN0DQo+ID4gPiA+ID4gPg0KPiA+ID4gPiA+ID4+DQo+ID4gPiA+ID4gPj4gYnVzX2FkZHI6
ID8NCj4gPiA+ID4gPiA+Pg0KPiA+ID4gPiA+ID4NCj4gPiA+ID4gPiA+IEFjY29yZGluZyB0byB0
aGUgZ3VpZGUgYWJvdmUsIHRoaXMgaXMgdGhlIGFkZHJlc3MgaW50byB3aGljaA0KPiB0aGUNCj4g
PiA+ID4gPiBwY2lfYWRkcmVzcw0KPiA+ID4gPiA+ID4gZ2V0IHRyYW5zbGF0ZWQgdG8gYW5kIHRo
YXQgaXMgcGFzc2VkIHRvIHRoZSByb290IGNvbXBsZXguDQo+IEJldHdlZW4NCj4gPiA+IHRoZQ0K
PiA+ID4gPiA+IHJvb3QNCj4gPiA+ID4gPiA+IGNvbXBsZXggYW5kIHRoZSBDUFUgdGhlcmUgY2Fu
IGJlIGludGVybWVkaWF0ZSB0cmFuc2xhdGlvbg0KPiBsYXllcnM6DQo+ID4gPiBzZWUNCj4gPiA+
ID4gPiB0aGF0IHRvDQo+ID4gPiA+ID4gPiBnZXQgcGNpX2FkZHJlc3Mgd2UgY2FsbCAib2ZfdHJh
bnNsYXRlX2FkZHJlc3MiOyB0aGlzIHdpbGwNCj4gYXBwbHkNCj4gPiA+IGFsbA0KPiA+ID4gPiA+
IHRoZQ0KPiA+ID4gPiA+ID4gdHJhbnNsYXRpb24gbGF5ZXJzIChyYW5nZXMgaW4gdGhlIERUKSB0
aGF0IGl0IGZpbmRzIHRpbGwgaXQNCj4gY29tZXMNCj4gPiA+IHRvDQo+ID4gPiA+ID4gdGhlIHJv
b3QNCj4gPiA+ID4gPiA+IG5vZGUgb2YgdGhlIERUICh0aHVzIHJldHJpZXZpbmcgdGhlIENQVSBh
ZGRyZXNzKS4NCj4gPiA+ID4gPiA+IE5vdyBzYWlkIHRoYXQsIGZvciBkZXNpZ253YXJlIHdlIG5l
ZWQgdGhlIGZpcnN0IHRyYW5zbGF0ZWQgUENJDQo+ID4gPiA+ID4gYWRkcmVzcywgdGhhdCB3ZSBj
YWxsDQo+ID4gPiA+ID4NCj4gPiA+ID4gPiBJIHRoaW5rIHlvdSBtZWFuICJ0cmFuc2xhdGVkIENQ
VSBhZGRyZXNzLiIgVGhlIGZsb3cgb2YgYWRkcmVzc2VzDQo+ID4gPiBsb29rcw0KPiA+ID4gPiA+
IGxpa2UgdGhpczoNCj4gPiA+ID4gPg0KPiA+ID4gPiA+IENQVSAtPiBDUFUgYnVzIGFkZHJlc3Mg
LT4gYnVzIGZhYnJpYyBhZGRyZXNzIGRlY29kaW5nIC0+IGJ1cw0KPiA+ID4gYWRkcmVzcw0KPiA+
ID4gPiA+IC0+IERXIFBDSSAtPiBQQ0kgYWRkcmVzcw0KPiA+ID4gPiA+DQo+ID4gPiA+ID4gVGhp
cyBpcyBxdWl0ZSBjb21tb24gdGhhdCBhbiBJUCBibG9jayBkb2VzIG5vdCBzZWUgdGhlIGZ1bGwN
Cj4gYWRkcmVzcy4NCj4gPiA+ID4gPiBJdCBpcyB1bnVzdWFsIHRoYXQgdGhlIElQIGJsb2NrIG5l
ZWRzIHRvIGtub3cgaXRzIGZ1bGwgYWRkcmVzcw0KPiBvbg0KPiA+ID4gdGhlDQo+ID4gPiA+ID4g
c2xhdmUgc2lkZS4gSXQgaXMgcXVpdGUgY29tbW9uIGZvciB0aGUgbWFzdGVyIHNpZGUgYW5kIHRo
ZQ0KPiBrZXJuZWwNCj4gPiA+ID4gPiBkZWFscyB3aXRoIHRoYXQgYWxsIHRoZSB0aW1lIHdpdGgg
RE1BIG1hcHBpbmcNCj4gPiA+ID4gPg0KPiA+ID4gPiA+ID4gaGVyZSBidXNfYWRkciBhZnRlciBS
b2IgSGVycmluZyBzdWdnZXN0ZWQgdGhlIG5hbWUuLi5ob25lc3RseQ0KPiBJDQo+ID4gPiA+ID4g
Y2Fubm90IHRoaW5rIG9mIGENCj4gPiA+ID4gPiA+IGRpZmZlcmVudCBuYW1lDQo+ID4gPiA+ID4N
Cj4gPiA+ID4gPiBUaGlua2luZyBhYm91dCB0aGlzIHNvbWUgbW9yZSwgaXMgdGhpcyByZWFsbHkg
YSB0cmFuc2xhdGlvbg0KPiB2ZXJzdXMNCj4gPiA+ID4gPiBqdXN0IGEgc3RyaXBwaW5nIG9mIGhp
Z2ggYml0cz8gRG9lcyB0aGUgRFcgSVAgaGF2ZSBsZXNzIHRoYW4gMzItDQo+ID4gPiBiaXRzDQo+
ID4gPiA+ID4gYWRkcmVzcz8gSWYgc28sIHdlIGNvdWxkIGV4cHJlc3MgZGlmZmVyZW50bHkgYW5k
IGp1c3QgbWFzayB0aGUNCj4gQ1BVDQo+ID4gPiA+ID4gYWRkcmVzcyB3aXRoaW4gdGhlIGRyaXZl
ciBpbnN0ZWFkLg0KPiA+ID4gPg0KPiA+ID4gPiBJIGRvbuKAmXQgdGhpbmsgd2Ugc2hvdWxkIHJl
bHkgb24gW0NQVV0gYWRkcmVzc2VzLi4ud2hhdCBpZiB0aGUNCj4gPiA+IGludGVybWVkaWF0ZQ0K
PiA+ID4gPiB0cmFuc2xhdGlvbiBsYXllciBjaGFuZ2VzIHRoZSBsb3dlciBzaWduaWZpY2FudCBi
aXRzIG9mIHRoZSAiYnVzDQo+ID4gPiBhZGRyZXNzIg0KPiA+ID4gPiB0byB0cmFuc2xhdGUgaW50
byBhIGNwdSBhZGRyZXNzPw0KPiA+ID4NCj4gPiA+IElzIGl0IHJlYWxseSBhIHBvc3NpYmxpdHkg
dGhhdCB0aGUgbG93ZXIgYml0cyBjb3VsZCBiZSBjaGFuZ2VkPw0KPiA+DQo+ID4gSSd2ZSBjaGVj
a2VkIGFsbCB0aGUgY3VycmVudCBkZWlnbndhcmUgdXNlcnMgRFRzIGV4Y2VwdCAicGNpLQ0KPiBs
YXllcnNjYXBlIg0KPiA+IHRoYXQgSSBjb3VsZCBub3QgZmluZDoNCj4gPiBzcGVhcjEzMTAuZHRz
aQ0KPiA+IHNwZWFyMTM0MC5kdHNpDQo+ID4gZHJhNy5kdHNpDQo+ID4gaW14NnFkbC5kdHNpDQo+
ID4gaW14NnN4LmR0c2kNCj4gPiBrZXlzdG9uZS5kdHNpDQo+ID4gZXh5bm9zNTQ0MC5kdHNpDQo+
ID4NCj4gPiBOb25lIG9mIHRoZW0gbW9kaWZpZXMgdGhlIGxvd2VyIGJpdHMuIFRvIGJlIG1vcmUg
cHJlY2lzZSB0aGUgb25seSBndXkNCj4gPiB0aGF0IHByb3ZpZGVzIGFub3RoZXIgdHJhbnNsYXRp
b24gbGF5ZXIgaXMgImRyYTcuZHRzaSI6DQo+ID4gYXhpMA0KPiA+IGh0dHA6Ly9seHIuZnJlZS1l
bGVjdHJvbnMuY29tL3NvdXJjZS9hcmNoL2FybS9ib290L2R0cy9kcmE3LmR0c2kjTDIwNw0KPiA+
DQo+ID4gYXhpMQ0KPiA+IGh0dHA6Ly9seHIuZnJlZS1lbGVjdHJvbnMuY29tL3NvdXJjZS9hcmNo
L2FybS9ib290L2R0cy9kcmE3LmR0c2kjTDI0MQ0KPiA+DQo+ID4gRm9yIHRoaXMgY2FzZSBtYXNr
aW5nIHRoZSB0b3AgNGJpdHMgKGJpdHMyOCB0byAzMSkgc2hvdWxkIG1ha2UgdGhlIGpvYi4NCj4g
Pg0KPiA+IFNwZWFraW5nIGluIGdlbmVyYWwgdGVybXMgc28gZmFyIEkndmUgYWx3YXlzIHNlZW4g
bGluZWFyIG1hcHBpbmdzDQo+ID4gdGhhdCBkaWZmZXIgYnkgYml0bWFzayBvZmZzZXQsIGhvd2V2
ZXIgbGluZWFyIGRvZXMgbm90IG1lYW4gdGhhdCB5b3UNCj4gPiBjYW5ub3QgYWZmZWN0IHRoZSBs
b3dlciBiaXRzOiBlLmcuIDwweDA+IHRvIDwweDAgKyBzaXplPiBjYW4gbWFwIHRvDQo+ID4gPDB4
MDAwMGNhZmUgdG8gMHgwMDAwY2FmZSArIHNpemU+LCBidXQgSSBndWVzcyB0aGF0IGZvciBIVyBk
ZXNpZ24NCj4gcmVhc29ucw0KPiA+IGl0IGlzIG11Y2ggZWFzaWVyIHRvIHJlbWFwIGRpcmVjdGx5
IHVzaW5nIGEgYml0bWFzay4uLmJ1dCBJIHdhcyBub3QNCj4gc3VyZQ0KPiA+IGFuZCBJIGRpZG4n
dCB0aGluayBhYm91dCB0aGUgcHJvYmxlbXMgdGhhdCBjYW4gYXJpc2Ugd2l0aCBBQ1BJLg0KPiAN
Cj4gQXMgSSBzYWlkLCBpdCB3b3VsZG4ndCBtYWtlIHNlbnNlIGZvciB0aGUgYml0cyB0aGF0IGNv
bXByaXNlIHRoZQ0KPiBvZmZzZXQgaW50byB0aGUgd2luZG93IHRvIGNoYW5nZSwgc28gdGhlIG1h
cHBpbmcgb25seSBoYXMgdG8gYmUgbGluZWFyDQo+IGluc2lkZSB0aGUgd2luZG93Lg0KPiANCj4g
Rm9yIHRoZSBiaXRzIG91dHNpZGUgdGhlIHdpbmRvdyBvZmZzZXQsIEkgZG9uJ3Qga25vdyBlbm91
Z2ggdG8gc2F5DQo+IHdoZXRoZXIgbWFza2luZyBpcyBzdWZmaWNpZW50IG9yIHNhZmUuDQo+IA0K
PiA+IElmIHlvdSB0aGluayB0aGUgYml0bWFzayBpcyBPayB0aGVuIEkgY2FuIGRpcmVjdGx5IGRl
ZmluZSBpdCBpbg0KPiA+IGRlc2lnbndhcmUgYW5kIHdlIGNhbiBkcm9wIHRoaXMgcGF0Y2guDQo+
IA0KPiBJdCdzIE9LIGJ5IG1lLCBidXQgSSBrbm93IG5vdGhpbmcgYWJvdXQgdGhlIGFjdHVhbCBo
YXJkd2FyZSBpbnZvbHZlZC4NCj4gRm9yIHRoZSBEZXNpZ25XYXJlIHF1ZXN0aW9uLCBJIHRoaW5r
IHlvdSB3b3VsZCBqdXN0IGhhdmUgdG8gY29udmluY2UNCj4gSmluZ29vIGFuZCBQcmF0eXVzaCAo
Y2MnZCkuDQoNClllcyBhY3R1YWxseSBsb29raW5nIGF0DQpodHRwOi8vbHhyLmZyZWUtZWxlY3Ry
b25zLmNvbS9zb3VyY2UvYXJjaC9hcm0vYm9vdC9kdHMvc3BlYXIxMzEwLmR0c2kjTDk5DQpJIGNh
biBzZWUgdGhhdCBwY2lfYWRkcj0wIGlzIG1hcHBlZCB0byBidXNfYWRkciAweDgwMDIwMDAwLCBz
byBjbGVhcmluZyANCnRoZSB0b3AgNCBiaXRzIHdvdWxkIGFsdGVyIHRoZSBiZWhhdmlvdXIgb2Yg
dGhlIGN1cnJlbnQgZGVzaWdud2FyZSANClNXIGZyYW1ld29yay4uLm5vdyBJIGRvbid0IGtub3cg
aWYgRFcgbmVlZHMgb25seSB0aGUgbG93IDI4YiBvZiB0aGF0DQp2YWx1ZSBvciBub3QuLi4uDQoN
CkppbmdvbywgUHJhdHl1c2ggd2hhdCBkbyB5b3UgdGhpbms/DQoNCj4gDQo+ID4gPiBJIHRoaW5r
IHdlJ3JlIHRhbGtpbmcgYWJvdXQgTU1JTyBoZXJlLCBhbmQgYSBicmlkZ2UgaGFzIGFuIE1NSU8N
Cj4gPiA+IHdpbmRvdy4gIEEgd2luZG93IGlzIGEgcmFuZ2Ugb2YgQ1BVIHBoeXNpY2FsIGFkZHJl
c3NlcyB0aGF0IHRoZQ0KPiA+ID4gYnJpZGdlIGZvcndhcmRzIHRvIFBDSS4gIFRoZSBQQ0kgY29y
ZSBhc3N1bWVzIHRoYXQgYSBDUFUgcGh5c2ljYWwNCj4gPiA+IGFkZHJlc3MgcmFuZ2UgaXMgbGlu
ZWFybHkgbWFwcGVkIHRvIFBDSSBidXMgYWRkcmVzc2VzLCBpLmUuLCBpZiB0aGUNCj4gPiA+IHdp
bmRvdyBiYXNlIGlzIFggYW5kIG1hcHMgdG8gUENJIGFkZHJlc3MgWSwgdGhlbiBhcyBsb25nIGFz
IFgrbiBpcw0KPiA+ID4gaW5zaWRlIHRoZSB3aW5kb3csIGl0IG11c3QgbWFwIHRvIFkrbi4NCj4g
PiA+DQo+ID4gPiBUaGF0IG1lYW5zIHRoZSBsb3ctb3JkZXIgYml0cyAodGhlIG9uZXMgdGhhdCBh
cmUgdGhlIG9mZnNldCBpbnRvIHRoZQ0KPiA+ID4gd2luZG93KSBjYW5ub3QgY2hhbmdlLg0KPiA+
ID4gLS0NCj4gPiA+IFRvIHVuc3Vic2NyaWJlIGZyb20gdGhpcyBsaXN0OiBzZW5kIHRoZSBsaW5l
ICJ1bnN1YnNjcmliZSBsaW51eC1wY2kiDQo+IGluDQo+ID4gPiB0aGUgYm9keSBvZiBhIG1lc3Nh
Z2UgdG8gbWFqb3Jkb21vQHZnZXIua2VybmVsLm9yZw0KPiA+ID4gTW9yZSBtYWpvcmRvbW8gaW5m
byBhdCAgaHR0cDovL3ZnZXIua2VybmVsLm9yZy9tYWpvcmRvbW8taW5mby5odG1sDQo=

^ permalink raw reply	[flat|nested] 73+ messages in thread
* [PATCH v6] PCI: Store PCIe bus address in struct of_pci_range
@ 2015-07-30 17:34                       ` Gabriele Paoloni
  0 siblings, 0 replies; 73+ messages in thread
From: Gabriele Paoloni @ 2015-07-30 17:34 UTC (permalink / raw)
  To: linux-arm-kernel



> -----Original Message-----
> From: Bjorn Helgaas [mailto:bhelgaas at google.com]
> Sent: 30 July 2015 18:15
> To: Gabriele Paoloni
> Cc: Rob Herring; arnd at arndb.de; lorenzo.pieralisi at arm.com; Wangzhou (B);
> robh+dt at kernel.org; james.morse at arm.com; Liviu.Dudau at arm.com; linux-
> pci at vger.kernel.org; linux-arm-kernel at lists.infradead.org;
> devicetree at vger.kernel.org; Yuanzhichang; Zhudacai; zhangjukuo;
> qiuzhenfa; Liguozhu (Kenneth); Jingoo Han; Pratyush Anand
> Subject: Re: [PATCH v6] PCI: Store PCIe bus address in struct
> of_pci_range
> 
> [+cc Jingoo, Pratyush]
> 
> On Thu, Jul 30, 2015 at 04:50:55PM +0000, Gabriele Paoloni wrote:
> > > -----Original Message-----
> > > From: linux-pci-owner at vger.kernel.org [mailto:linux-pci-
> > > owner at vger.kernel.org] On Behalf Of Bjorn Helgaas
> > > Sent: Thursday, July 30, 2015 5:15 PM
> > > To: Gabriele Paoloni
> > > Cc: Rob Herring; arnd at arndb.de; lorenzo.pieralisi at arm.com; Wangzhou
> (B);
> > > robh+dt at kernel.org; james.morse at arm.com; Liviu.Dudau at arm.com; linux-
> > > pci at vger.kernel.org; linux-arm-kernel at lists.infradead.org;
> > > devicetree at vger.kernel.org; Yuanzhichang; Zhudacai; zhangjukuo;
> > > qiuzhenfa; Liguozhu (Kenneth)
> > > Subject: Re: [PATCH v6] PCI: Store PCIe bus address in struct
> > > of_pci_range
> > >
> > > On Thu, Jul 30, 2015 at 01:52:13PM +0000, Gabriele Paoloni wrote:
> > > >
> > > >
> > > > > -----Original Message-----
> > > > > From: linux-pci-owner at vger.kernel.org [mailto:linux-pci-
> > > > > owner at vger.kernel.org] On Behalf Of Rob Herring
> > > > > Sent: Thursday, July 30, 2015 2:43 PM
> > > > > To: Gabriele Paoloni
> > > > > Cc: Bjorn Helgaas; arnd at arndb.de; lorenzo.pieralisi at arm.com;
> > > Wangzhou
> > > > > (B); robh+dt at kernel.org; james.morse at arm.com;
> Liviu.Dudau at arm.com;
> > > > > linux-pci at vger.kernel.org; linux-arm-kernel at lists.infradead.org;
> > > > > devicetree at vger.kernel.org; Yuanzhichang; Zhudacai; zhangjukuo;
> > > > > qiuzhenfa; Liguozhu (Kenneth)
> > > > > Subject: Re: [PATCH v6] PCI: Store PCIe bus address in struct
> > > > > of_pci_range
> > > > >
> > > > > On Wed, Jul 29, 2015 at 2:44 PM, Gabriele Paoloni
> > > > > <gabriele.paoloni@huawei.com> wrote:
> > > > > > Hi Bjorn
> > > > > >
> > > > > > Many Thanks for your reply
> > > > > >
> > > > > > I have commented back inline with resolutions from my side.
> > > > > >
> > > > > > If you're ok with them I'll send it out a new version in the
> > > > > appropriate patchset
> > > > > >
> > > > > > Cheers
> > > > > >
> > > > > > Gab
> > > > > >
> > > > > >> -----Original Message-----
> > > > > >> From: Bjorn Helgaas [mailto:bhelgaas at google.com]
> > > > > >> Sent: Wednesday, July 29, 2015 6:21 PM
> > > > > >> To: Gabriele Paoloni
> > > > > >> Cc: arnd at arndb.de; lorenzo.pieralisi at arm.com; Wangzhou (B);
> > > > > >> robh+dt at kernel.org; james.morse at arm.com; Liviu.Dudau at arm.com;
> > > linux-
> > > > > >> pci at vger.kernel.org; linux-arm-kernel at lists.infradead.org;
> > > > > >> devicetree at vger.kernel.org; Yuanzhichang; Zhudacai;
> zhangjukuo;
> > > > > >> qiuzhenfa; Liguozhu (Kenneth)
> > > > > >> Subject: Re: [PATCH v6] PCI: Store PCIe bus address in struct
> > > > > >> of_pci_range
> > > > > >>
> > > > > >> Hi Gabriele,
> > > > > >>
> > > > > >> As far as I can tell, this is not specific to PCIe, so please
> > > use
> > > > > "PCI"
> > > > > >> in
> > > > > >> the subject as a generic term that includes both PCI and PCIe.
> > > > > >
> > > > > > sure agreed
> > > > > >
> > > > > >>
> > > > > >> On Mon, Jul 27, 2015 at 11:17:03PM +0800, Gabriele Paoloni
> wrote:
> > > > > >> > From: gabriele paoloni <gabriele.paoloni@huawei.com>
> > > > > >> >
> > > > > >> >     This patch is needed port PCIe designware to new DT
> > > parsing
> > > > > API
> > > > > >> >     As discussed in
> > > > > >> >     http://lists.infradead.org/pipermail/linux-arm-
> > > kernel/2015-
> > > > > >> January/317743.html
> > > > > >> >     in designware we have a problem as the PCI addresses in
> > > the
> > > > > PCIe
> > > > > >> controller
> > > > > >> >     address space are required in order to perform correct
> HW
> > > > > >> operation.
> > > > > >> >
> > > > > >> >     In order to solve this problem commit f4c55c5a3 "PCI:
> > > > > designware:
> > > > > >> >     Program ATU with untranslated address" added code to
> read
> > > the
> > > > > >> PCIe
> > > > > >>
> > > > > >> Conventional reference is 12-char SHA1, like this:
> > > > > >>
> > > > > >>   f4c55c5a3f7f ("PCI: designware: Program ATU with
> untranslated
> > > > > >> address")
> > > > > >
> > > > > > Agreed, will change this
> > > > > >
> > > > > >>
> > > > > >> >     controller start address directly from the DT ranges.
> > > > > >> >
> > > > > >> >     In the new DT parsing API
> > > of_pci_get_host_bridge_resources()
> > > > > >> hides the
> > > > > >> >     DT parser from the host controller drivers, so it is
> not
> > > > > possible
> > > > > >> >     for drivers to parse values directly from the DT.
> > > > > >> >
> > > > > >> >     In http://www.spinics.net/lists/linux-pci/msg42540.html
> we
> > > > > >> already tried
> > > > > >> >     to use the new DT parsing API but there is a bug
> > > (obviously)
> > > > > in
> > > > > >> setting
> > > > > >> >     the <*>_mod_base addresses
> > > > > >> >     Applying this patch we can easily set "<*>_mod_base =
> win-
> > > > > >> >__res.start"
> > > > > >>
> > > > > >> By itself, this patch adds something.  It would help me
> > > understand
> > > > > it
> > > > > >> if
> > > > > >> the *user* of this new something were in the same patch
> series.
> > > > > >
> > > > > > the user is: "[PATCH v5 2/5] PCI: designware: Add ARM64
> support"
> > > > > > I will ask Zhou Wang to include this patch in his patchset
> > > > > >
> > > > > >
> > > > > >>
> > > > > >> >     This patch adds a new field in "struct of_pci_range" to
> > > store
> > > > > the
> > > > > >> >     pci bus start address; it fills the field in
> > > > > >> of_pci_range_parser_one();
> > > > > >> >     in of_pci_get_host_bridge_resources() it retrieves the
> > > > > resource
> > > > > >> entry
> > > > > >> >     after it is created and added to the resource list and
> > > uses
> > > > > >> >     entry->__res.start to store the pci controller address
> > > > > >>
> > > > > >> struct of_pci_range is starting to get confusing to non-OF
> folks
> > > > > like
> > > > > >> me.
> > > > > >> It now contains:
> > > > > >>
> > > > > >>   u32 pci_space;
> > > > > >>   u64 pci_addr;
> > > > > >>   u64 cpu_addr;
> > > > > >>   u64 bus_addr;
> > > > > >>
> > > > > >> Can you explain what all these things mean, and maybe even
> add
> > > one-
> > > > > line
> > > > > >> comments to the structure?
> > > > > >
> > > > > > sure I can add comments inline in the code
> > > > > >
> > > > > >>
> > > > > >> pci_space: The only uses I see are to determine whether to
> print
> > > > > >> "Prefetch".  I don't see any real functionality that uses
> this.
> > > > > >
> > > > > > Looking at the code I agree. it's seems to be used only in
> > > powerpc
> > > > > > and microblaze to print out.
> > > > > > However from my understanding pci_space is the phys.hi field
> of
> > > the
> > > > > > ranges property: it defines the properties of the address
> space
> > > > > associated
> > > > > > to the PCI address. if you're curious you can find a nice and
> > > quick
> > > > > to read
> > > > > > "guide" in http://devicetree.org/MPC5200:PCI
> > > > > >
> > > > > >>
> > > > > >> pci_addr: I assume this is a PCI bus address, like what you
> > > would
> > > > > see
> > > > > >> if
> > > > > >> you put an analyzer on the bus/link.  This address could go
> in a
> > > BAR.
> > > > > >
> > > > > > Yes, this is the PCI start address of the range: phys.mid +
> > > phys.low
> > > > > in the
> > > > > > guide mentioned above
> > > > > >
> > > > > >>
> > > > > >> cpu_addr: I assume this is a CPU physical address, like what
> you
> > > > > would
> > > > > >> see
> > > > > >> in /proc/iomem and what you would pass to ioremap().
> > > > > >
> > > > > > Yes correct
> > > > > >
> > > > > >>
> > > > > >> bus_addr: ?
> > > > > >>
> > > > > >
> > > > > > According to the guide above, this is the address into which
> the
> > > > > pci_address
> > > > > > get translated to and that is passed to the root complex.
> Between
> > > the
> > > > > root
> > > > > > complex and the CPU there can be intermediate translation
> layers:
> > > see
> > > > > that to
> > > > > > get pci_address we call "of_translate_address"; this will
> apply
> > > all
> > > > > the
> > > > > > translation layers (ranges in the DT) that it finds till it
> comes
> > > to
> > > > > the root
> > > > > > node of the DT (thus retrieving the CPU address).
> > > > > > Now said that, for designware we need the first translated PCI
> > > > > address, that we call
> > > > >
> > > > > I think you mean "translated CPU address." The flow of addresses
> > > looks
> > > > > like this:
> > > > >
> > > > > CPU -> CPU bus address -> bus fabric address decoding -> bus
> > > address
> > > > > -> DW PCI -> PCI address
> > > > >
> > > > > This is quite common that an IP block does not see the full
> address.
> > > > > It is unusual that the IP block needs to know its full address
> on
> > > the
> > > > > slave side. It is quite common for the master side and the
> kernel
> > > > > deals with that all the time with DMA mapping
> > > > >
> > > > > > here bus_addr after Rob Herring suggested the name...honestly
> I
> > > > > cannot think of a
> > > > > > different name
> > > > >
> > > > > Thinking about this some more, is this really a translation
> versus
> > > > > just a stripping of high bits? Does the DW IP have less than 32-
> > > bits
> > > > > address? If so, we could express differently and just mask the
> CPU
> > > > > address within the driver instead.
> > > >
> > > > I don?t think we should rely on [CPU] addresses...what if the
> > > intermediate
> > > > translation layer changes the lower significant bits of the "bus
> > > address"
> > > > to translate into a cpu address?
> > >
> > > Is it really a possiblity that the lower bits could be changed?
> >
> > I've checked all the current deignware users DTs except "pci-
> layerscape"
> > that I could not find:
> > spear1310.dtsi
> > spear1340.dtsi
> > dra7.dtsi
> > imx6qdl.dtsi
> > imx6sx.dtsi
> > keystone.dtsi
> > exynos5440.dtsi
> >
> > None of them modifies the lower bits. To be more precise the only guy
> > that provides another translation layer is "dra7.dtsi":
> > axi0
> > http://lxr.free-electrons.com/source/arch/arm/boot/dts/dra7.dtsi#L207
> >
> > axi1
> > http://lxr.free-electrons.com/source/arch/arm/boot/dts/dra7.dtsi#L241
> >
> > For this case masking the top 4bits (bits28 to 31) should make the job.
> >
> > Speaking in general terms so far I've always seen linear mappings
> > that differ by bitmask offset, however linear does not mean that you
> > cannot affect the lower bits: e.g. <0x0> to <0x0 + size> can map to
> > <0x0000cafe to 0x0000cafe + size>, but I guess that for HW design
> reasons
> > it is much easier to remap directly using a bitmask...but I was not
> sure
> > and I didn't think about the problems that can arise with ACPI.
> 
> As I said, it wouldn't make sense for the bits that comprise the
> offset into the window to change, so the mapping only has to be linear
> inside the window.
> 
> For the bits outside the window offset, I don't know enough to say
> whether masking is sufficient or safe.
> 
> > If you think the bitmask is Ok then I can directly define it in
> > designware and we can drop this patch.
> 
> It's OK by me, but I know nothing about the actual hardware involved.
> For the DesignWare question, I think you would just have to convince
> Jingoo and Pratyush (cc'd).

Yes actually looking at
http://lxr.free-electrons.com/source/arch/arm/boot/dts/spear1310.dtsi#L99
I can see that pci_addr=0 is mapped to bus_addr 0x80020000, so clearing 
the top 4 bits would alter the behaviour of the current designware 
SW framework...now I don't know if DW needs only the low 28b of that
value or not....

Jingoo, Pratyush what do you think?

> 
> > > I think we're talking about MMIO here, and a bridge has an MMIO
> > > window.  A window is a range of CPU physical addresses that the
> > > bridge forwards to PCI.  The PCI core assumes that a CPU physical
> > > address range is linearly mapped to PCI bus addresses, i.e., if the
> > > window base is X and maps to PCI address Y, then as long as X+n is
> > > inside the window, it must map to Y+n.
> > >
> > > That means the low-order bits (the ones that are the offset into the
> > > window) cannot change.
> > > --
> > > To unsubscribe from this list: send the line "unsubscribe linux-pci"
> in
> > > the body of a message to majordomo at vger.kernel.org
> > > More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 73+ messages in thread
* Re: [PATCH v6] PCI: Store PCIe bus address in struct of_pci_range
  2015-07-30 17:34                       ` Gabriele Paoloni
@ 2015-07-30 20:41                         ` Rob Herring
  -1 siblings, 0 replies; 73+ messages in thread
From: Rob Herring @ 2015-07-30 20:41 UTC (permalink / raw)
  To: Gabriele Paoloni
  Cc: Bjorn Helgaas, arnd, lorenzo.pieralisi, Wangzhou (B),
	robh+dt, james.morse, Liviu.Dudau, linux-pci, linux-arm-kernel,
	devicetree, Yuanzhichang, Zhudacai, zhangjukuo, qiuzhenfa,
	Liguozhu (Kenneth),
	Jingoo Han, Pratyush Anand

On Thu, Jul 30, 2015 at 12:34 PM, Gabriele Paoloni
<gabriele.paoloni@huawei.com> wrote:
>> -----Original Message-----
>> From: Bjorn Helgaas [mailto:bhelgaas@google.com]
>> Sent: 30 July 2015 18:15
>> On Thu, Jul 30, 2015 at 04:50:55PM +0000, Gabriele Paoloni wrote:
>> > > -----Original Message-----
>> > > From: linux-pci-owner@vger.kernel.org [mailto:linux-pci-
>> > > owner@vger.kernel.org] On Behalf Of Bjorn Helgaas
>> > > Sent: Thursday, July 30, 2015 5:15 PM
>> > > On Thu, Jul 30, 2015 at 01:52:13PM +0000, Gabriele Paoloni wrote:

[...]

>> > > > I don’t think we should rely on [CPU] addresses...what if the
>> > > intermediate
>> > > > translation layer changes the lower significant bits of the "bus
>> > > address"
>> > > > to translate into a cpu address?
>> > >
>> > > Is it really a possiblity that the lower bits could be changed?
>> >
>> > I've checked all the current deignware users DTs except "pci-
>> layerscape"
>> > that I could not find:
>> > spear1310.dtsi
>> > spear1340.dtsi
>> > dra7.dtsi
>> > imx6qdl.dtsi
>> > imx6sx.dtsi
>> > keystone.dtsi
>> > exynos5440.dtsi
>> >
>> > None of them modifies the lower bits. To be more precise the only guy
>> > that provides another translation layer is "dra7.dtsi":
>> > axi0
>> > http://lxr.free-electrons.com/source/arch/arm/boot/dts/dra7.dtsi#L207
>> >
>> > axi1
>> > http://lxr.free-electrons.com/source/arch/arm/boot/dts/dra7.dtsi#L241
>> >
>> > For this case masking the top 4bits (bits28 to 31) should make the job.

IMO, we should just fix this case. After further study, I don't think
this is a DW issue, but rather an SOC integration issue.

I believe you can just fixup the address in the pp->ops->host_init hook.

In looking at this, I'm curious why only 2 ATU viewports are used by
default as this causes switching on config accesses. Probably some
configuration only has 2 viewports. This would not even work on SMP
systems! Memory and i/o accesses do not have any lock. A config access
on one core will temporarily disable the i/o or memory viewport which
will cause an abort, dropped, or garbage data on an i/o or memory
access from another core. You would have to be accessing cfg space on
a bus other than the root bus, so maybe no one is doing that.

>> >
>> > Speaking in general terms so far I've always seen linear mappings
>> > that differ by bitmask offset, however linear does not mean that you
>> > cannot affect the lower bits: e.g. <0x0> to <0x0 + size> can map to
>> > <0x0000cafe to 0x0000cafe + size>, but I guess that for HW design
>> reasons
>> > it is much easier to remap directly using a bitmask...but I was not
>> sure
>> > and I didn't think about the problems that can arise with ACPI.

For higher speed buses, the h/w designs are not going to be doing
complicated translations in order to meet timing requirements. At the
top level only the highest order bits are going to be looked at. For
downstream segments, the high order bits may get dropped to simplify
routing. If an IP block has full address bits in this case, they could
either be tied to 0 or tied off to the correct address. Either is
reasonable, so I'm sure there is h/w doing both. That doesn't mean h/w
designers haven't done something crazy, too.

>> As I said, it wouldn't make sense for the bits that comprise the
>> offset into the window to change, so the mapping only has to be linear
>> inside the window.
>>
>> For the bits outside the window offset, I don't know enough to say
>> whether masking is sufficient or safe.
>>
>> > If you think the bitmask is Ok then I can directly define it in
>> > designware and we can drop this patch.
>>
>> It's OK by me, but I know nothing about the actual hardware involved.
>> For the DesignWare question, I think you would just have to convince
>> Jingoo and Pratyush (cc'd).
>
> Yes actually looking at
> http://lxr.free-electrons.com/source/arch/arm/boot/dts/spear1310.dtsi#L99
> I can see that pci_addr=0 is mapped to bus_addr 0x80020000, so clearing
> the top 4 bits would alter the behaviour of the current designware
> SW framework...now I don't know if DW needs only the low 28b of that
> value or not....

Given that most cases have same cpu and bus address, masking is not
the right solution in general.

There is also a bug here BTW. pcie0 and pcie2 have the same CPU
address for the memory range.

Rob

^ permalink raw reply	[flat|nested] 73+ messages in thread
* [PATCH v6] PCI: Store PCIe bus address in struct of_pci_range
@ 2015-07-30 20:41                         ` Rob Herring
  0 siblings, 0 replies; 73+ messages in thread
From: Rob Herring @ 2015-07-30 20:41 UTC (permalink / raw)
  To: linux-arm-kernel

On Thu, Jul 30, 2015 at 12:34 PM, Gabriele Paoloni
<gabriele.paoloni@huawei.com> wrote:
>> -----Original Message-----
>> From: Bjorn Helgaas [mailto:bhelgaas at google.com]
>> Sent: 30 July 2015 18:15
>> On Thu, Jul 30, 2015 at 04:50:55PM +0000, Gabriele Paoloni wrote:
>> > > -----Original Message-----
>> > > From: linux-pci-owner at vger.kernel.org [mailto:linux-pci-
>> > > owner at vger.kernel.org] On Behalf Of Bjorn Helgaas
>> > > Sent: Thursday, July 30, 2015 5:15 PM
>> > > On Thu, Jul 30, 2015 at 01:52:13PM +0000, Gabriele Paoloni wrote:

[...]

>> > > > I don?t think we should rely on [CPU] addresses...what if the
>> > > intermediate
>> > > > translation layer changes the lower significant bits of the "bus
>> > > address"
>> > > > to translate into a cpu address?
>> > >
>> > > Is it really a possiblity that the lower bits could be changed?
>> >
>> > I've checked all the current deignware users DTs except "pci-
>> layerscape"
>> > that I could not find:
>> > spear1310.dtsi
>> > spear1340.dtsi
>> > dra7.dtsi
>> > imx6qdl.dtsi
>> > imx6sx.dtsi
>> > keystone.dtsi
>> > exynos5440.dtsi
>> >
>> > None of them modifies the lower bits. To be more precise the only guy
>> > that provides another translation layer is "dra7.dtsi":
>> > axi0
>> > http://lxr.free-electrons.com/source/arch/arm/boot/dts/dra7.dtsi#L207
>> >
>> > axi1
>> > http://lxr.free-electrons.com/source/arch/arm/boot/dts/dra7.dtsi#L241
>> >
>> > For this case masking the top 4bits (bits28 to 31) should make the job.

IMO, we should just fix this case. After further study, I don't think
this is a DW issue, but rather an SOC integration issue.

I believe you can just fixup the address in the pp->ops->host_init hook.

In looking at this, I'm curious why only 2 ATU viewports are used by
default as this causes switching on config accesses. Probably some
configuration only has 2 viewports. This would not even work on SMP
systems! Memory and i/o accesses do not have any lock. A config access
on one core will temporarily disable the i/o or memory viewport which
will cause an abort, dropped, or garbage data on an i/o or memory
access from another core. You would have to be accessing cfg space on
a bus other than the root bus, so maybe no one is doing that.

>> >
>> > Speaking in general terms so far I've always seen linear mappings
>> > that differ by bitmask offset, however linear does not mean that you
>> > cannot affect the lower bits: e.g. <0x0> to <0x0 + size> can map to
>> > <0x0000cafe to 0x0000cafe + size>, but I guess that for HW design
>> reasons
>> > it is much easier to remap directly using a bitmask...but I was not
>> sure
>> > and I didn't think about the problems that can arise with ACPI.

For higher speed buses, the h/w designs are not going to be doing
complicated translations in order to meet timing requirements. At the
top level only the highest order bits are going to be looked at. For
downstream segments, the high order bits may get dropped to simplify
routing. If an IP block has full address bits in this case, they could
either be tied to 0 or tied off to the correct address. Either is
reasonable, so I'm sure there is h/w doing both. That doesn't mean h/w
designers haven't done something crazy, too.

>> As I said, it wouldn't make sense for the bits that comprise the
>> offset into the window to change, so the mapping only has to be linear
>> inside the window.
>>
>> For the bits outside the window offset, I don't know enough to say
>> whether masking is sufficient or safe.
>>
>> > If you think the bitmask is Ok then I can directly define it in
>> > designware and we can drop this patch.
>>
>> It's OK by me, but I know nothing about the actual hardware involved.
>> For the DesignWare question, I think you would just have to convince
>> Jingoo and Pratyush (cc'd).
>
> Yes actually looking at
> http://lxr.free-electrons.com/source/arch/arm/boot/dts/spear1310.dtsi#L99
> I can see that pci_addr=0 is mapped to bus_addr 0x80020000, so clearing
> the top 4 bits would alter the behaviour of the current designware
> SW framework...now I don't know if DW needs only the low 28b of that
> value or not....

Given that most cases have same cpu and bus address, masking is not
the right solution in general.

There is also a bug here BTW. pcie0 and pcie2 have the same CPU
address for the memory range.

Rob

^ permalink raw reply	[flat|nested] 73+ messages in thread
* RE: [PATCH v6] PCI: Store PCIe bus address in struct of_pci_range
  2015-07-30 20:41                         ` Rob Herring
  (?)
@ 2015-07-31 14:25                           ` Gabriele Paoloni
  -1 siblings, 0 replies; 73+ messages in thread
From: Gabriele Paoloni @ 2015-07-31 14:25 UTC (permalink / raw)
  To: Rob Herring
  Cc: Bjorn Helgaas, arnd, lorenzo.pieralisi, Wangzhou (B),
	robh+dt, james.morse, Liviu.Dudau, linux-pci, linux-arm-kernel,
	devicetree, Yuanzhichang, Zhudacai, zhangjukuo, qiuzhenfa,
	Liguozhu (Kenneth),
	Jingoo Han, Pratyush Anand, kishon

[+cc Kishon]

> -----Original Message-----
> From: linux-pci-owner@vger.kernel.org [mailto:linux-pci-
> owner@vger.kernel.org] On Behalf Of Rob Herring
> Sent: Thursday, July 30, 2015 9:42 PM
> To: Gabriele Paoloni
> Cc: Bjorn Helgaas; arnd@arndb.de; lorenzo.pieralisi@arm.com; Wangzhou
> (B); robh+dt@kernel.org; james.morse@arm.com; Liviu.Dudau@arm.com;
> linux-pci@vger.kernel.org; linux-arm-kernel@lists.infradead.org;
> devicetree@vger.kernel.org; Yuanzhichang; Zhudacai; zhangjukuo;
> qiuzhenfa; Liguozhu (Kenneth); Jingoo Han; Pratyush Anand
> Subject: Re: [PATCH v6] PCI: Store PCIe bus address in struct
> of_pci_range
> 
> On Thu, Jul 30, 2015 at 12:34 PM, Gabriele Paoloni
> <gabriele.paoloni@huawei.com> wrote:
> >> -----Original Message-----
> >> From: Bjorn Helgaas [mailto:bhelgaas@google.com]
> >> Sent: 30 July 2015 18:15
> >> On Thu, Jul 30, 2015 at 04:50:55PM +0000, Gabriele Paoloni wrote:
> >> > > -----Original Message-----
> >> > > From: linux-pci-owner@vger.kernel.org [mailto:linux-pci-
> >> > > owner@vger.kernel.org] On Behalf Of Bjorn Helgaas
> >> > > Sent: Thursday, July 30, 2015 5:15 PM
> >> > > On Thu, Jul 30, 2015 at 01:52:13PM +0000, Gabriele Paoloni wrote:
> 
> [...]
> 
> >> > > > I don’t think we should rely on [CPU] addresses...what if the
> >> > > intermediate
> >> > > > translation layer changes the lower significant bits of the
> "bus
> >> > > address"
> >> > > > to translate into a cpu address?
> >> > >
> >> > > Is it really a possiblity that the lower bits could be changed?
> >> >
> >> > I've checked all the current deignware users DTs except "pci-
> >> layerscape"
> >> > that I could not find:
> >> > spear1310.dtsi
> >> > spear1340.dtsi
> >> > dra7.dtsi
> >> > imx6qdl.dtsi
> >> > imx6sx.dtsi
> >> > keystone.dtsi
> >> > exynos5440.dtsi
> >> >
> >> > None of them modifies the lower bits. To be more precise the only
> guy
> >> > that provides another translation layer is "dra7.dtsi":
> >> > axi0
> >> > http://lxr.free-
> electrons.com/source/arch/arm/boot/dts/dra7.dtsi#L207
> >> >
> >> > axi1
> >> > http://lxr.free-
> electrons.com/source/arch/arm/boot/dts/dra7.dtsi#L241
> >> >
> >> > For this case masking the top 4bits (bits28 to 31) should make the
> job.
> 
> IMO, we should just fix this case. After further study, I don't think
> this is a DW issue, but rather an SOC integration issue.
> 
> I believe you can just fixup the address in the pp->ops->host_init hook.
> 

Yes I guess that I could just assign pp->(*)_mod_base to the CPU address 
in DW and mask it out in dra7xx_pcie_host_init()...

Kishon, would you be ok with that? 


> In looking at this, I'm curious why only 2 ATU viewports are used by
> default as this causes switching on config accesses. Probably some
> configuration only has 2 viewports. This would not even work on SMP
> systems! Memory and i/o accesses do not have any lock. A config access
> on one core will temporarily disable the i/o or memory viewport which
> will cause an abort, dropped, or garbage data on an i/o or memory
> access from another core. You would have to be accessing cfg space on
> a bus other than the root bus, so maybe no one is doing that.
> 
> >> >
> >> > Speaking in general terms so far I've always seen linear mappings
> >> > that differ by bitmask offset, however linear does not mean that
> you
> >> > cannot affect the lower bits: e.g. <0x0> to <0x0 + size> can map
> to
> >> > <0x0000cafe to 0x0000cafe + size>, but I guess that for HW design
> >> reasons
> >> > it is much easier to remap directly using a bitmask...but I was
> not
> >> sure
> >> > and I didn't think about the problems that can arise with ACPI.
> 
> For higher speed buses, the h/w designs are not going to be doing
> complicated translations in order to meet timing requirements. At the
> top level only the highest order bits are going to be looked at. For
> downstream segments, the high order bits may get dropped to simplify
> routing. If an IP block has full address bits in this case, they could
> either be tied to 0 or tied off to the correct address. Either is
> reasonable, so I'm sure there is h/w doing both. That doesn't mean h/w
> designers haven't done something crazy, too.
> 
> >> As I said, it wouldn't make sense for the bits that comprise the
> >> offset into the window to change, so the mapping only has to be
> linear
> >> inside the window.
> >>
> >> For the bits outside the window offset, I don't know enough to say
> >> whether masking is sufficient or safe.
> >>
> >> > If you think the bitmask is Ok then I can directly define it in
> >> > designware and we can drop this patch.
> >>
> >> It's OK by me, but I know nothing about the actual hardware involved.
> >> For the DesignWare question, I think you would just have to convince
> >> Jingoo and Pratyush (cc'd).
> >
> > Yes actually looking at
> > http://lxr.free-
> electrons.com/source/arch/arm/boot/dts/spear1310.dtsi#L99
> > I can see that pci_addr=0 is mapped to bus_addr 0x80020000, so
> clearing
> > the top 4 bits would alter the behaviour of the current designware
> > SW framework...now I don't know if DW needs only the low 28b of that
> > value or not....
> 
> Given that most cases have same cpu and bus address, masking is not
> the right solution in general.
> 
> There is also a bug here BTW. pcie0 and pcie2 have the same CPU
> address for the memory range.
> 
> Rob
> --
> To unsubscribe from this list: send the line "unsubscribe linux-pci" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 73+ messages in thread
* RE: [PATCH v6] PCI: Store PCIe bus address in struct of_pci_range
@ 2015-07-31 14:25                           ` Gabriele Paoloni
  0 siblings, 0 replies; 73+ messages in thread
From: Gabriele Paoloni @ 2015-07-31 14:25 UTC (permalink / raw)
  To: Rob Herring
  Cc: Bjorn Helgaas, arnd, lorenzo.pieralisi, Wangzhou (B),
	robh+dt, james.morse, Liviu.Dudau, linux-pci, linux-arm-kernel,
	devicetree, Yuanzhichang, Zhudacai, zhangjukuo, qiuzhenfa,
	Liguozhu (Kenneth),
	Jingoo Han, Pratyush Anand, kishon

WytjYyBLaXNob25dDQoNCj4gLS0tLS1PcmlnaW5hbCBNZXNzYWdlLS0tLS0NCj4gRnJvbTogbGlu
dXgtcGNpLW93bmVyQHZnZXIua2VybmVsLm9yZyBbbWFpbHRvOmxpbnV4LXBjaS0NCj4gb3duZXJA
dmdlci5rZXJuZWwub3JnXSBPbiBCZWhhbGYgT2YgUm9iIEhlcnJpbmcNCj4gU2VudDogVGh1cnNk
YXksIEp1bHkgMzAsIDIwMTUgOTo0MiBQTQ0KPiBUbzogR2FicmllbGUgUGFvbG9uaQ0KPiBDYzog
Qmpvcm4gSGVsZ2FhczsgYXJuZEBhcm5kYi5kZTsgbG9yZW56by5waWVyYWxpc2lAYXJtLmNvbTsg
V2FuZ3pob3UNCj4gKEIpOyByb2JoK2R0QGtlcm5lbC5vcmc7IGphbWVzLm1vcnNlQGFybS5jb207
IExpdml1LkR1ZGF1QGFybS5jb207DQo+IGxpbnV4LXBjaUB2Z2VyLmtlcm5lbC5vcmc7IGxpbnV4
LWFybS1rZXJuZWxAbGlzdHMuaW5mcmFkZWFkLm9yZzsNCj4gZGV2aWNldHJlZUB2Z2VyLmtlcm5l
bC5vcmc7IFl1YW56aGljaGFuZzsgWmh1ZGFjYWk7IHpoYW5nanVrdW87DQo+IHFpdXpoZW5mYTsg
TGlndW96aHUgKEtlbm5ldGgpOyBKaW5nb28gSGFuOyBQcmF0eXVzaCBBbmFuZA0KPiBTdWJqZWN0
OiBSZTogW1BBVENIIHY2XSBQQ0k6IFN0b3JlIFBDSWUgYnVzIGFkZHJlc3MgaW4gc3RydWN0DQo+
IG9mX3BjaV9yYW5nZQ0KPiANCj4gT24gVGh1LCBKdWwgMzAsIDIwMTUgYXQgMTI6MzQgUE0sIEdh
YnJpZWxlIFBhb2xvbmkNCj4gPGdhYnJpZWxlLnBhb2xvbmlAaHVhd2VpLmNvbT4gd3JvdGU6DQo+
ID4+IC0tLS0tT3JpZ2luYWwgTWVzc2FnZS0tLS0tDQo+ID4+IEZyb206IEJqb3JuIEhlbGdhYXMg
W21haWx0bzpiaGVsZ2Fhc0Bnb29nbGUuY29tXQ0KPiA+PiBTZW50OiAzMCBKdWx5IDIwMTUgMTg6
MTUNCj4gPj4gT24gVGh1LCBKdWwgMzAsIDIwMTUgYXQgMDQ6NTA6NTVQTSArMDAwMCwgR2Ficmll
bGUgUGFvbG9uaSB3cm90ZToNCj4gPj4gPiA+IC0tLS0tT3JpZ2luYWwgTWVzc2FnZS0tLS0tDQo+
ID4+ID4gPiBGcm9tOiBsaW51eC1wY2ktb3duZXJAdmdlci5rZXJuZWwub3JnIFttYWlsdG86bGlu
dXgtcGNpLQ0KPiA+PiA+ID4gb3duZXJAdmdlci5rZXJuZWwub3JnXSBPbiBCZWhhbGYgT2YgQmpv
cm4gSGVsZ2Fhcw0KPiA+PiA+ID4gU2VudDogVGh1cnNkYXksIEp1bHkgMzAsIDIwMTUgNToxNSBQ
TQ0KPiA+PiA+ID4gT24gVGh1LCBKdWwgMzAsIDIwMTUgYXQgMDE6NTI6MTNQTSArMDAwMCwgR2Fi
cmllbGUgUGFvbG9uaSB3cm90ZToNCj4gDQo+IFsuLi5dDQo+IA0KPiA+PiA+ID4gPiBJIGRvbuKA
mXQgdGhpbmsgd2Ugc2hvdWxkIHJlbHkgb24gW0NQVV0gYWRkcmVzc2VzLi4ud2hhdCBpZiB0aGUN
Cj4gPj4gPiA+IGludGVybWVkaWF0ZQ0KPiA+PiA+ID4gPiB0cmFuc2xhdGlvbiBsYXllciBjaGFu
Z2VzIHRoZSBsb3dlciBzaWduaWZpY2FudCBiaXRzIG9mIHRoZQ0KPiAiYnVzDQo+ID4+ID4gPiBh
ZGRyZXNzIg0KPiA+PiA+ID4gPiB0byB0cmFuc2xhdGUgaW50byBhIGNwdSBhZGRyZXNzPw0KPiA+
PiA+ID4NCj4gPj4gPiA+IElzIGl0IHJlYWxseSBhIHBvc3NpYmxpdHkgdGhhdCB0aGUgbG93ZXIg
Yml0cyBjb3VsZCBiZSBjaGFuZ2VkPw0KPiA+PiA+DQo+ID4+ID4gSSd2ZSBjaGVja2VkIGFsbCB0
aGUgY3VycmVudCBkZWlnbndhcmUgdXNlcnMgRFRzIGV4Y2VwdCAicGNpLQ0KPiA+PiBsYXllcnNj
YXBlIg0KPiA+PiA+IHRoYXQgSSBjb3VsZCBub3QgZmluZDoNCj4gPj4gPiBzcGVhcjEzMTAuZHRz
aQ0KPiA+PiA+IHNwZWFyMTM0MC5kdHNpDQo+ID4+ID4gZHJhNy5kdHNpDQo+ID4+ID4gaW14NnFk
bC5kdHNpDQo+ID4+ID4gaW14NnN4LmR0c2kNCj4gPj4gPiBrZXlzdG9uZS5kdHNpDQo+ID4+ID4g
ZXh5bm9zNTQ0MC5kdHNpDQo+ID4+ID4NCj4gPj4gPiBOb25lIG9mIHRoZW0gbW9kaWZpZXMgdGhl
IGxvd2VyIGJpdHMuIFRvIGJlIG1vcmUgcHJlY2lzZSB0aGUgb25seQ0KPiBndXkNCj4gPj4gPiB0
aGF0IHByb3ZpZGVzIGFub3RoZXIgdHJhbnNsYXRpb24gbGF5ZXIgaXMgImRyYTcuZHRzaSI6DQo+
ID4+ID4gYXhpMA0KPiA+PiA+IGh0dHA6Ly9seHIuZnJlZS0NCj4gZWxlY3Ryb25zLmNvbS9zb3Vy
Y2UvYXJjaC9hcm0vYm9vdC9kdHMvZHJhNy5kdHNpI0wyMDcNCj4gPj4gPg0KPiA+PiA+IGF4aTEN
Cj4gPj4gPiBodHRwOi8vbHhyLmZyZWUtDQo+IGVsZWN0cm9ucy5jb20vc291cmNlL2FyY2gvYXJt
L2Jvb3QvZHRzL2RyYTcuZHRzaSNMMjQxDQo+ID4+ID4NCj4gPj4gPiBGb3IgdGhpcyBjYXNlIG1h
c2tpbmcgdGhlIHRvcCA0Yml0cyAoYml0czI4IHRvIDMxKSBzaG91bGQgbWFrZSB0aGUNCj4gam9i
Lg0KPiANCj4gSU1PLCB3ZSBzaG91bGQganVzdCBmaXggdGhpcyBjYXNlLiBBZnRlciBmdXJ0aGVy
IHN0dWR5LCBJIGRvbid0IHRoaW5rDQo+IHRoaXMgaXMgYSBEVyBpc3N1ZSwgYnV0IHJhdGhlciBh
biBTT0MgaW50ZWdyYXRpb24gaXNzdWUuDQo+IA0KPiBJIGJlbGlldmUgeW91IGNhbiBqdXN0IGZp
eHVwIHRoZSBhZGRyZXNzIGluIHRoZSBwcC0+b3BzLT5ob3N0X2luaXQgaG9vay4NCj4gDQoNClll
cyBJIGd1ZXNzIHRoYXQgSSBjb3VsZCBqdXN0IGFzc2lnbiBwcC0+KCopX21vZF9iYXNlIHRvIHRo
ZSBDUFUgYWRkcmVzcyANCmluIERXIGFuZCBtYXNrIGl0IG91dCBpbiBkcmE3eHhfcGNpZV9ob3N0
X2luaXQoKS4uLg0KDQpLaXNob24sIHdvdWxkIHlvdSBiZSBvayB3aXRoIHRoYXQ/IA0KDQoNCj4g
SW4gbG9va2luZyBhdCB0aGlzLCBJJ20gY3VyaW91cyB3aHkgb25seSAyIEFUVSB2aWV3cG9ydHMg
YXJlIHVzZWQgYnkNCj4gZGVmYXVsdCBhcyB0aGlzIGNhdXNlcyBzd2l0Y2hpbmcgb24gY29uZmln
IGFjY2Vzc2VzLiBQcm9iYWJseSBzb21lDQo+IGNvbmZpZ3VyYXRpb24gb25seSBoYXMgMiB2aWV3
cG9ydHMuIFRoaXMgd291bGQgbm90IGV2ZW4gd29yayBvbiBTTVANCj4gc3lzdGVtcyEgTWVtb3J5
IGFuZCBpL28gYWNjZXNzZXMgZG8gbm90IGhhdmUgYW55IGxvY2suIEEgY29uZmlnIGFjY2Vzcw0K
PiBvbiBvbmUgY29yZSB3aWxsIHRlbXBvcmFyaWx5IGRpc2FibGUgdGhlIGkvbyBvciBtZW1vcnkg
dmlld3BvcnQgd2hpY2gNCj4gd2lsbCBjYXVzZSBhbiBhYm9ydCwgZHJvcHBlZCwgb3IgZ2FyYmFn
ZSBkYXRhIG9uIGFuIGkvbyBvciBtZW1vcnkNCj4gYWNjZXNzIGZyb20gYW5vdGhlciBjb3JlLiBZ
b3Ugd291bGQgaGF2ZSB0byBiZSBhY2Nlc3NpbmcgY2ZnIHNwYWNlIG9uDQo+IGEgYnVzIG90aGVy
IHRoYW4gdGhlIHJvb3QgYnVzLCBzbyBtYXliZSBubyBvbmUgaXMgZG9pbmcgdGhhdC4NCj4gDQo+
ID4+ID4NCj4gPj4gPiBTcGVha2luZyBpbiBnZW5lcmFsIHRlcm1zIHNvIGZhciBJJ3ZlIGFsd2F5
cyBzZWVuIGxpbmVhciBtYXBwaW5ncw0KPiA+PiA+IHRoYXQgZGlmZmVyIGJ5IGJpdG1hc2sgb2Zm
c2V0LCBob3dldmVyIGxpbmVhciBkb2VzIG5vdCBtZWFuIHRoYXQNCj4geW91DQo+ID4+ID4gY2Fu
bm90IGFmZmVjdCB0aGUgbG93ZXIgYml0czogZS5nLiA8MHgwPiB0byA8MHgwICsgc2l6ZT4gY2Fu
IG1hcA0KPiB0bw0KPiA+PiA+IDwweDAwMDBjYWZlIHRvIDB4MDAwMGNhZmUgKyBzaXplPiwgYnV0
IEkgZ3Vlc3MgdGhhdCBmb3IgSFcgZGVzaWduDQo+ID4+IHJlYXNvbnMNCj4gPj4gPiBpdCBpcyBt
dWNoIGVhc2llciB0byByZW1hcCBkaXJlY3RseSB1c2luZyBhIGJpdG1hc2suLi5idXQgSSB3YXMN
Cj4gbm90DQo+ID4+IHN1cmUNCj4gPj4gPiBhbmQgSSBkaWRuJ3QgdGhpbmsgYWJvdXQgdGhlIHBy
b2JsZW1zIHRoYXQgY2FuIGFyaXNlIHdpdGggQUNQSS4NCj4gDQo+IEZvciBoaWdoZXIgc3BlZWQg
YnVzZXMsIHRoZSBoL3cgZGVzaWducyBhcmUgbm90IGdvaW5nIHRvIGJlIGRvaW5nDQo+IGNvbXBs
aWNhdGVkIHRyYW5zbGF0aW9ucyBpbiBvcmRlciB0byBtZWV0IHRpbWluZyByZXF1aXJlbWVudHMu
IEF0IHRoZQ0KPiB0b3AgbGV2ZWwgb25seSB0aGUgaGlnaGVzdCBvcmRlciBiaXRzIGFyZSBnb2lu
ZyB0byBiZSBsb29rZWQgYXQuIEZvcg0KPiBkb3duc3RyZWFtIHNlZ21lbnRzLCB0aGUgaGlnaCBv
cmRlciBiaXRzIG1heSBnZXQgZHJvcHBlZCB0byBzaW1wbGlmeQ0KPiByb3V0aW5nLiBJZiBhbiBJ
UCBibG9jayBoYXMgZnVsbCBhZGRyZXNzIGJpdHMgaW4gdGhpcyBjYXNlLCB0aGV5IGNvdWxkDQo+
IGVpdGhlciBiZSB0aWVkIHRvIDAgb3IgdGllZCBvZmYgdG8gdGhlIGNvcnJlY3QgYWRkcmVzcy4g
RWl0aGVyIGlzDQo+IHJlYXNvbmFibGUsIHNvIEknbSBzdXJlIHRoZXJlIGlzIGgvdyBkb2luZyBi
b3RoLiBUaGF0IGRvZXNuJ3QgbWVhbiBoL3cNCj4gZGVzaWduZXJzIGhhdmVuJ3QgZG9uZSBzb21l
dGhpbmcgY3JhenksIHRvby4NCj4gDQo+ID4+IEFzIEkgc2FpZCwgaXQgd291bGRuJ3QgbWFrZSBz
ZW5zZSBmb3IgdGhlIGJpdHMgdGhhdCBjb21wcmlzZSB0aGUNCj4gPj4gb2Zmc2V0IGludG8gdGhl
IHdpbmRvdyB0byBjaGFuZ2UsIHNvIHRoZSBtYXBwaW5nIG9ubHkgaGFzIHRvIGJlDQo+IGxpbmVh
cg0KPiA+PiBpbnNpZGUgdGhlIHdpbmRvdy4NCj4gPj4NCj4gPj4gRm9yIHRoZSBiaXRzIG91dHNp
ZGUgdGhlIHdpbmRvdyBvZmZzZXQsIEkgZG9uJ3Qga25vdyBlbm91Z2ggdG8gc2F5DQo+ID4+IHdo
ZXRoZXIgbWFza2luZyBpcyBzdWZmaWNpZW50IG9yIHNhZmUuDQo+ID4+DQo+ID4+ID4gSWYgeW91
IHRoaW5rIHRoZSBiaXRtYXNrIGlzIE9rIHRoZW4gSSBjYW4gZGlyZWN0bHkgZGVmaW5lIGl0IGlu
DQo+ID4+ID4gZGVzaWdud2FyZSBhbmQgd2UgY2FuIGRyb3AgdGhpcyBwYXRjaC4NCj4gPj4NCj4g
Pj4gSXQncyBPSyBieSBtZSwgYnV0IEkga25vdyBub3RoaW5nIGFib3V0IHRoZSBhY3R1YWwgaGFy
ZHdhcmUgaW52b2x2ZWQuDQo+ID4+IEZvciB0aGUgRGVzaWduV2FyZSBxdWVzdGlvbiwgSSB0aGlu
ayB5b3Ugd291bGQganVzdCBoYXZlIHRvIGNvbnZpbmNlDQo+ID4+IEppbmdvbyBhbmQgUHJhdHl1
c2ggKGNjJ2QpLg0KPiA+DQo+ID4gWWVzIGFjdHVhbGx5IGxvb2tpbmcgYXQNCj4gPiBodHRwOi8v
bHhyLmZyZWUtDQo+IGVsZWN0cm9ucy5jb20vc291cmNlL2FyY2gvYXJtL2Jvb3QvZHRzL3NwZWFy
MTMxMC5kdHNpI0w5OQ0KPiA+IEkgY2FuIHNlZSB0aGF0IHBjaV9hZGRyPTAgaXMgbWFwcGVkIHRv
IGJ1c19hZGRyIDB4ODAwMjAwMDAsIHNvDQo+IGNsZWFyaW5nDQo+ID4gdGhlIHRvcCA0IGJpdHMg
d291bGQgYWx0ZXIgdGhlIGJlaGF2aW91ciBvZiB0aGUgY3VycmVudCBkZXNpZ253YXJlDQo+ID4g
U1cgZnJhbWV3b3JrLi4ubm93IEkgZG9uJ3Qga25vdyBpZiBEVyBuZWVkcyBvbmx5IHRoZSBsb3cg
MjhiIG9mIHRoYXQNCj4gPiB2YWx1ZSBvciBub3QuLi4uDQo+IA0KPiBHaXZlbiB0aGF0IG1vc3Qg
Y2FzZXMgaGF2ZSBzYW1lIGNwdSBhbmQgYnVzIGFkZHJlc3MsIG1hc2tpbmcgaXMgbm90DQo+IHRo
ZSByaWdodCBzb2x1dGlvbiBpbiBnZW5lcmFsLg0KPiANCj4gVGhlcmUgaXMgYWxzbyBhIGJ1ZyBo
ZXJlIEJUVy4gcGNpZTAgYW5kIHBjaWUyIGhhdmUgdGhlIHNhbWUgQ1BVDQo+IGFkZHJlc3MgZm9y
IHRoZSBtZW1vcnkgcmFuZ2UuDQo+IA0KPiBSb2INCj4gLS0NCj4gVG8gdW5zdWJzY3JpYmUgZnJv
bSB0aGlzIGxpc3Q6IHNlbmQgdGhlIGxpbmUgInVuc3Vic2NyaWJlIGxpbnV4LXBjaSIgaW4NCj4g
dGhlIGJvZHkgb2YgYSBtZXNzYWdlIHRvIG1ham9yZG9tb0B2Z2VyLmtlcm5lbC5vcmcNCj4gTW9y
ZSBtYWpvcmRvbW8gaW5mbyBhdCAgaHR0cDovL3ZnZXIua2VybmVsLm9yZy9tYWpvcmRvbW8taW5m
by5odG1sDQo=

^ permalink raw reply	[flat|nested] 73+ messages in thread
* [PATCH v6] PCI: Store PCIe bus address in struct of_pci_range
@ 2015-07-31 14:25                           ` Gabriele Paoloni
  0 siblings, 0 replies; 73+ messages in thread
From: Gabriele Paoloni @ 2015-07-31 14:25 UTC (permalink / raw)
  To: linux-arm-kernel

[+cc Kishon]

> -----Original Message-----
> From: linux-pci-owner at vger.kernel.org [mailto:linux-pci-
> owner at vger.kernel.org] On Behalf Of Rob Herring
> Sent: Thursday, July 30, 2015 9:42 PM
> To: Gabriele Paoloni
> Cc: Bjorn Helgaas; arnd at arndb.de; lorenzo.pieralisi at arm.com; Wangzhou
> (B); robh+dt at kernel.org; james.morse at arm.com; Liviu.Dudau at arm.com;
> linux-pci at vger.kernel.org; linux-arm-kernel at lists.infradead.org;
> devicetree at vger.kernel.org; Yuanzhichang; Zhudacai; zhangjukuo;
> qiuzhenfa; Liguozhu (Kenneth); Jingoo Han; Pratyush Anand
> Subject: Re: [PATCH v6] PCI: Store PCIe bus address in struct
> of_pci_range
> 
> On Thu, Jul 30, 2015 at 12:34 PM, Gabriele Paoloni
> <gabriele.paoloni@huawei.com> wrote:
> >> -----Original Message-----
> >> From: Bjorn Helgaas [mailto:bhelgaas at google.com]
> >> Sent: 30 July 2015 18:15
> >> On Thu, Jul 30, 2015 at 04:50:55PM +0000, Gabriele Paoloni wrote:
> >> > > -----Original Message-----
> >> > > From: linux-pci-owner at vger.kernel.org [mailto:linux-pci-
> >> > > owner at vger.kernel.org] On Behalf Of Bjorn Helgaas
> >> > > Sent: Thursday, July 30, 2015 5:15 PM
> >> > > On Thu, Jul 30, 2015 at 01:52:13PM +0000, Gabriele Paoloni wrote:
> 
> [...]
> 
> >> > > > I don?t think we should rely on [CPU] addresses...what if the
> >> > > intermediate
> >> > > > translation layer changes the lower significant bits of the
> "bus
> >> > > address"
> >> > > > to translate into a cpu address?
> >> > >
> >> > > Is it really a possiblity that the lower bits could be changed?
> >> >
> >> > I've checked all the current deignware users DTs except "pci-
> >> layerscape"
> >> > that I could not find:
> >> > spear1310.dtsi
> >> > spear1340.dtsi
> >> > dra7.dtsi
> >> > imx6qdl.dtsi
> >> > imx6sx.dtsi
> >> > keystone.dtsi
> >> > exynos5440.dtsi
> >> >
> >> > None of them modifies the lower bits. To be more precise the only
> guy
> >> > that provides another translation layer is "dra7.dtsi":
> >> > axi0
> >> > http://lxr.free-
> electrons.com/source/arch/arm/boot/dts/dra7.dtsi#L207
> >> >
> >> > axi1
> >> > http://lxr.free-
> electrons.com/source/arch/arm/boot/dts/dra7.dtsi#L241
> >> >
> >> > For this case masking the top 4bits (bits28 to 31) should make the
> job.
> 
> IMO, we should just fix this case. After further study, I don't think
> this is a DW issue, but rather an SOC integration issue.
> 
> I believe you can just fixup the address in the pp->ops->host_init hook.
> 

Yes I guess that I could just assign pp->(*)_mod_base to the CPU address 
in DW and mask it out in dra7xx_pcie_host_init()...

Kishon, would you be ok with that? 


> In looking at this, I'm curious why only 2 ATU viewports are used by
> default as this causes switching on config accesses. Probably some
> configuration only has 2 viewports. This would not even work on SMP
> systems! Memory and i/o accesses do not have any lock. A config access
> on one core will temporarily disable the i/o or memory viewport which
> will cause an abort, dropped, or garbage data on an i/o or memory
> access from another core. You would have to be accessing cfg space on
> a bus other than the root bus, so maybe no one is doing that.
> 
> >> >
> >> > Speaking in general terms so far I've always seen linear mappings
> >> > that differ by bitmask offset, however linear does not mean that
> you
> >> > cannot affect the lower bits: e.g. <0x0> to <0x0 + size> can map
> to
> >> > <0x0000cafe to 0x0000cafe + size>, but I guess that for HW design
> >> reasons
> >> > it is much easier to remap directly using a bitmask...but I was
> not
> >> sure
> >> > and I didn't think about the problems that can arise with ACPI.
> 
> For higher speed buses, the h/w designs are not going to be doing
> complicated translations in order to meet timing requirements. At the
> top level only the highest order bits are going to be looked at. For
> downstream segments, the high order bits may get dropped to simplify
> routing. If an IP block has full address bits in this case, they could
> either be tied to 0 or tied off to the correct address. Either is
> reasonable, so I'm sure there is h/w doing both. That doesn't mean h/w
> designers haven't done something crazy, too.
> 
> >> As I said, it wouldn't make sense for the bits that comprise the
> >> offset into the window to change, so the mapping only has to be
> linear
> >> inside the window.
> >>
> >> For the bits outside the window offset, I don't know enough to say
> >> whether masking is sufficient or safe.
> >>
> >> > If you think the bitmask is Ok then I can directly define it in
> >> > designware and we can drop this patch.
> >>
> >> It's OK by me, but I know nothing about the actual hardware involved.
> >> For the DesignWare question, I think you would just have to convince
> >> Jingoo and Pratyush (cc'd).
> >
> > Yes actually looking at
> > http://lxr.free-
> electrons.com/source/arch/arm/boot/dts/spear1310.dtsi#L99
> > I can see that pci_addr=0 is mapped to bus_addr 0x80020000, so
> clearing
> > the top 4 bits would alter the behaviour of the current designware
> > SW framework...now I don't know if DW needs only the low 28b of that
> > value or not....
> 
> Given that most cases have same cpu and bus address, masking is not
> the right solution in general.
> 
> There is also a bug here BTW. pcie0 and pcie2 have the same CPU
> address for the memory range.
> 
> Rob
> --
> To unsubscribe from this list: send the line "unsubscribe linux-pci" in
> the body of a message to majordomo at vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 73+ messages in thread
* Re: [PATCH v6] PCI: Store PCIe bus address in struct of_pci_range
  2015-07-31 14:25                           ` Gabriele Paoloni
  (?)
@ 2015-07-31 14:57                             ` Kishon Vijay Abraham I
  -1 siblings, 0 replies; 73+ messages in thread
From: Kishon Vijay Abraham I @ 2015-07-31 14:57 UTC (permalink / raw)
  To: Gabriele Paoloni, Rob Herring
  Cc: Bjorn Helgaas, arnd, lorenzo.pieralisi, Wangzhou (B),
	robh+dt, james.morse, Liviu.Dudau, linux-pci, linux-arm-kernel,
	devicetree, Yuanzhichang, Zhudacai, zhangjukuo, qiuzhenfa,
	Liguozhu (Kenneth),
	Jingoo Han, Pratyush Anand, Arnd Bergmann

+Arnd

Hi,

On Friday 31 July 2015 07:55 PM, Gabriele Paoloni wrote:
> [+cc Kishon]
> 
>> -----Original Message-----
>> From: linux-pci-owner@vger.kernel.org [mailto:linux-pci-
>> owner@vger.kernel.org] On Behalf Of Rob Herring
>> Sent: Thursday, July 30, 2015 9:42 PM
>> To: Gabriele Paoloni
>> Cc: Bjorn Helgaas; arnd@arndb.de; lorenzo.pieralisi@arm.com; Wangzhou
>> (B); robh+dt@kernel.org; james.morse@arm.com; Liviu.Dudau@arm.com;
>> linux-pci@vger.kernel.org; linux-arm-kernel@lists.infradead.org;
>> devicetree@vger.kernel.org; Yuanzhichang; Zhudacai; zhangjukuo;
>> qiuzhenfa; Liguozhu (Kenneth); Jingoo Han; Pratyush Anand
>> Subject: Re: [PATCH v6] PCI: Store PCIe bus address in struct
>> of_pci_range
>>
>> On Thu, Jul 30, 2015 at 12:34 PM, Gabriele Paoloni
>> <gabriele.paoloni@huawei.com> wrote:
>>>> -----Original Message-----
>>>> From: Bjorn Helgaas [mailto:bhelgaas@google.com]
>>>> Sent: 30 July 2015 18:15
>>>> On Thu, Jul 30, 2015 at 04:50:55PM +0000, Gabriele Paoloni wrote:
>>>>>> -----Original Message-----
>>>>>> From: linux-pci-owner@vger.kernel.org [mailto:linux-pci-
>>>>>> owner@vger.kernel.org] On Behalf Of Bjorn Helgaas
>>>>>> Sent: Thursday, July 30, 2015 5:15 PM
>>>>>> On Thu, Jul 30, 2015 at 01:52:13PM +0000, Gabriele Paoloni wrote:
>>
>> [...]
>>
>>>>>>> I don’t think we should rely on [CPU] addresses...what if the
>>>>>> intermediate
>>>>>>> translation layer changes the lower significant bits of the
>> "bus
>>>>>> address"
>>>>>>> to translate into a cpu address?
>>>>>>
>>>>>> Is it really a possiblity that the lower bits could be changed?
>>>>>
>>>>> I've checked all the current deignware users DTs except "pci-
>>>> layerscape"
>>>>> that I could not find:
>>>>> spear1310.dtsi
>>>>> spear1340.dtsi
>>>>> dra7.dtsi
>>>>> imx6qdl.dtsi
>>>>> imx6sx.dtsi
>>>>> keystone.dtsi
>>>>> exynos5440.dtsi
>>>>>
>>>>> None of them modifies the lower bits. To be more precise the only
>> guy
>>>>> that provides another translation layer is "dra7.dtsi":
>>>>> axi0
>>>>> http://lxr.free-
>> electrons.com/source/arch/arm/boot/dts/dra7.dtsi#L207
>>>>>
>>>>> axi1
>>>>> http://lxr.free-
>> electrons.com/source/arch/arm/boot/dts/dra7.dtsi#L241
>>>>>
>>>>> For this case masking the top 4bits (bits28 to 31) should make the
>> job.
>>
>> IMO, we should just fix this case. After further study, I don't think
>> this is a DW issue, but rather an SOC integration issue.
>>
>> I believe you can just fixup the address in the pp->ops->host_init hook.
>>
> 
> Yes I guess that I could just assign pp->(*)_mod_base to the CPU address 
> in DW and mask it out in dra7xx_pcie_host_init()...
> 
> Kishon, would you be ok with that? 

Initially I was using *base-mask* property from dt. Me and Arnd (cc'ed) had
this discussion [1] before we decided the current approach. It'll be good to
check with Arnd too.

[1] ->  http://lists.infradead.org/pipermail/linux-arm-kernel/2014-May/253528.html

Thanks
Kishon

^ permalink raw reply	[flat|nested] 73+ messages in thread
* Re: [PATCH v6] PCI: Store PCIe bus address in struct of_pci_range
@ 2015-07-31 14:57                             ` Kishon Vijay Abraham I
  0 siblings, 0 replies; 73+ messages in thread
From: Kishon Vijay Abraham I @ 2015-07-31 14:57 UTC (permalink / raw)
  To: Gabriele Paoloni, Rob Herring
  Cc: Bjorn Helgaas, arnd, lorenzo.pieralisi, Wangzhou (B),
	robh+dt, james.morse, Liviu.Dudau, linux-pci, linux-arm-kernel,
	devicetree, Yuanzhichang, Zhudacai, zhangjukuo, qiuzhenfa,
	Liguozhu (Kenneth),
	Jingoo Han, Pratyush Anand, Arnd Bergmann, Arnd Bergmann

+Arnd

Hi,

On Friday 31 July 2015 07:55 PM, Gabriele Paoloni wrote:
> [+cc Kishon]
> 
>> -----Original Message-----
>> From: linux-pci-owner@vger.kernel.org [mailto:linux-pci-
>> owner@vger.kernel.org] On Behalf Of Rob Herring
>> Sent: Thursday, July 30, 2015 9:42 PM
>> To: Gabriele Paoloni
>> Cc: Bjorn Helgaas; arnd@arndb.de; lorenzo.pieralisi@arm.com; Wangzhou
>> (B); robh+dt@kernel.org; james.morse@arm.com; Liviu.Dudau@arm.com;
>> linux-pci@vger.kernel.org; linux-arm-kernel@lists.infradead.org;
>> devicetree@vger.kernel.org; Yuanzhichang; Zhudacai; zhangjukuo;
>> qiuzhenfa; Liguozhu (Kenneth); Jingoo Han; Pratyush Anand
>> Subject: Re: [PATCH v6] PCI: Store PCIe bus address in struct
>> of_pci_range
>>
>> On Thu, Jul 30, 2015 at 12:34 PM, Gabriele Paoloni
>> <gabriele.paoloni@huawei.com> wrote:
>>>> -----Original Message-----
>>>> From: Bjorn Helgaas [mailto:bhelgaas@google.com]
>>>> Sent: 30 July 2015 18:15
>>>> On Thu, Jul 30, 2015 at 04:50:55PM +0000, Gabriele Paoloni wrote:
>>>>>> -----Original Message-----
>>>>>> From: linux-pci-owner@vger.kernel.org [mailto:linux-pci-
>>>>>> owner@vger.kernel.org] On Behalf Of Bjorn Helgaas
>>>>>> Sent: Thursday, July 30, 2015 5:15 PM
>>>>>> On Thu, Jul 30, 2015 at 01:52:13PM +0000, Gabriele Paoloni wrote:
>>
>> [...]
>>
>>>>>>> I don’t think we should rely on [CPU] addresses...what if the
>>>>>> intermediate
>>>>>>> translation layer changes the lower significant bits of the
>> "bus
>>>>>> address"
>>>>>>> to translate into a cpu address?
>>>>>>
>>>>>> Is it really a possiblity that the lower bits could be changed?
>>>>>
>>>>> I've checked all the current deignware users DTs except "pci-
>>>> layerscape"
>>>>> that I could not find:
>>>>> spear1310.dtsi
>>>>> spear1340.dtsi
>>>>> dra7.dtsi
>>>>> imx6qdl.dtsi
>>>>> imx6sx.dtsi
>>>>> keystone.dtsi
>>>>> exynos5440.dtsi
>>>>>
>>>>> None of them modifies the lower bits. To be more precise the only
>> guy
>>>>> that provides another translation layer is "dra7.dtsi":
>>>>> axi0
>>>>> http://lxr.free-
>> electrons.com/source/arch/arm/boot/dts/dra7.dtsi#L207
>>>>>
>>>>> axi1
>>>>> http://lxr.free-
>> electrons.com/source/arch/arm/boot/dts/dra7.dtsi#L241
>>>>>
>>>>> For this case masking the top 4bits (bits28 to 31) should make the
>> job.
>>
>> IMO, we should just fix this case. After further study, I don't think
>> this is a DW issue, but rather an SOC integration issue.
>>
>> I believe you can just fixup the address in the pp->ops->host_init hook.
>>
> 
> Yes I guess that I could just assign pp->(*)_mod_base to the CPU address 
> in DW and mask it out in dra7xx_pcie_host_init()...
> 
> Kishon, would you be ok with that? 

Initially I was using *base-mask* property from dt. Me and Arnd (cc'ed) had
this discussion [1] before we decided the current approach. It'll be good to
check with Arnd too.

[1] ->  http://lists.infradead.org/pipermail/linux-arm-kernel/2014-May/253528.html

Thanks
Kishon

^ permalink raw reply	[flat|nested] 73+ messages in thread
* [PATCH v6] PCI: Store PCIe bus address in struct of_pci_range
@ 2015-07-31 14:57                             ` Kishon Vijay Abraham I
  0 siblings, 0 replies; 73+ messages in thread
From: Kishon Vijay Abraham I @ 2015-07-31 14:57 UTC (permalink / raw)
  To: linux-arm-kernel

+Arnd

Hi,

On Friday 31 July 2015 07:55 PM, Gabriele Paoloni wrote:
> [+cc Kishon]
> 
>> -----Original Message-----
>> From: linux-pci-owner at vger.kernel.org [mailto:linux-pci-
>> owner at vger.kernel.org] On Behalf Of Rob Herring
>> Sent: Thursday, July 30, 2015 9:42 PM
>> To: Gabriele Paoloni
>> Cc: Bjorn Helgaas; arnd at arndb.de; lorenzo.pieralisi at arm.com; Wangzhou
>> (B); robh+dt at kernel.org; james.morse at arm.com; Liviu.Dudau at arm.com;
>> linux-pci at vger.kernel.org; linux-arm-kernel at lists.infradead.org;
>> devicetree at vger.kernel.org; Yuanzhichang; Zhudacai; zhangjukuo;
>> qiuzhenfa; Liguozhu (Kenneth); Jingoo Han; Pratyush Anand
>> Subject: Re: [PATCH v6] PCI: Store PCIe bus address in struct
>> of_pci_range
>>
>> On Thu, Jul 30, 2015 at 12:34 PM, Gabriele Paoloni
>> <gabriele.paoloni@huawei.com> wrote:
>>>> -----Original Message-----
>>>> From: Bjorn Helgaas [mailto:bhelgaas at google.com]
>>>> Sent: 30 July 2015 18:15
>>>> On Thu, Jul 30, 2015 at 04:50:55PM +0000, Gabriele Paoloni wrote:
>>>>>> -----Original Message-----
>>>>>> From: linux-pci-owner at vger.kernel.org [mailto:linux-pci-
>>>>>> owner at vger.kernel.org] On Behalf Of Bjorn Helgaas
>>>>>> Sent: Thursday, July 30, 2015 5:15 PM
>>>>>> On Thu, Jul 30, 2015 at 01:52:13PM +0000, Gabriele Paoloni wrote:
>>
>> [...]
>>
>>>>>>> I don?t think we should rely on [CPU] addresses...what if the
>>>>>> intermediate
>>>>>>> translation layer changes the lower significant bits of the
>> "bus
>>>>>> address"
>>>>>>> to translate into a cpu address?
>>>>>>
>>>>>> Is it really a possiblity that the lower bits could be changed?
>>>>>
>>>>> I've checked all the current deignware users DTs except "pci-
>>>> layerscape"
>>>>> that I could not find:
>>>>> spear1310.dtsi
>>>>> spear1340.dtsi
>>>>> dra7.dtsi
>>>>> imx6qdl.dtsi
>>>>> imx6sx.dtsi
>>>>> keystone.dtsi
>>>>> exynos5440.dtsi
>>>>>
>>>>> None of them modifies the lower bits. To be more precise the only
>> guy
>>>>> that provides another translation layer is "dra7.dtsi":
>>>>> axi0
>>>>> http://lxr.free-
>> electrons.com/source/arch/arm/boot/dts/dra7.dtsi#L207
>>>>>
>>>>> axi1
>>>>> http://lxr.free-
>> electrons.com/source/arch/arm/boot/dts/dra7.dtsi#L241
>>>>>
>>>>> For this case masking the top 4bits (bits28 to 31) should make the
>> job.
>>
>> IMO, we should just fix this case. After further study, I don't think
>> this is a DW issue, but rather an SOC integration issue.
>>
>> I believe you can just fixup the address in the pp->ops->host_init hook.
>>
> 
> Yes I guess that I could just assign pp->(*)_mod_base to the CPU address 
> in DW and mask it out in dra7xx_pcie_host_init()...
> 
> Kishon, would you be ok with that? 

Initially I was using *base-mask* property from dt. Me and Arnd (cc'ed) had
this discussion [1] before we decided the current approach. It'll be good to
check with Arnd too.

[1] ->  http://lists.infradead.org/pipermail/linux-arm-kernel/2014-May/253528.html

Thanks
Kishon

^ permalink raw reply	[flat|nested] 73+ messages in thread
* RE: [PATCH v6] PCI: Store PCIe bus address in struct of_pci_range
  2015-07-31 14:57                             ` Kishon Vijay Abraham I
  (?)
@ 2015-07-31 15:09                               ` Gabriele Paoloni
  -1 siblings, 0 replies; 73+ messages in thread
From: Gabriele Paoloni @ 2015-07-31 15:09 UTC (permalink / raw)
  To: Kishon Vijay Abraham I, Rob Herring
  Cc: Bjorn Helgaas, arnd, lorenzo.pieralisi, Wangzhou (B),
	robh+dt, james.morse, Liviu.Dudau, linux-pci, linux-arm-kernel,
	devicetree, Yuanzhichang, Zhudacai, zhangjukuo, qiuzhenfa,
	Liguozhu (Kenneth),
	Jingoo Han, Pratyush Anand, Arnd Bergmann



> -----Original Message-----
> From: Kishon Vijay Abraham I [mailto:kishon@ti.com]
> Sent: Friday, July 31, 2015 3:57 PM
> To: Gabriele Paoloni; Rob Herring
> Cc: Bjorn Helgaas; arnd@arndb.de; lorenzo.pieralisi@arm.com; Wangzhou
> (B); robh+dt@kernel.org; james.morse@arm.com; Liviu.Dudau@arm.com;
> linux-pci@vger.kernel.org; linux-arm-kernel@lists.infradead.org;
> devicetree@vger.kernel.org; Yuanzhichang; Zhudacai; zhangjukuo;
> qiuzhenfa; Liguozhu (Kenneth); Jingoo Han; Pratyush Anand; Arnd
> Bergmann; Arnd Bergmann
> Subject: Re: [PATCH v6] PCI: Store PCIe bus address in struct
> of_pci_range
> 
> +Arnd
> 
> Hi,
> 
> On Friday 31 July 2015 07:55 PM, Gabriele Paoloni wrote:
> > [+cc Kishon]
> >
> >> -----Original Message-----
> >> From: linux-pci-owner@vger.kernel.org [mailto:linux-pci-
> >> owner@vger.kernel.org] On Behalf Of Rob Herring
> >> Sent: Thursday, July 30, 2015 9:42 PM
> >> To: Gabriele Paoloni
> >> Cc: Bjorn Helgaas; arnd@arndb.de; lorenzo.pieralisi@arm.com;
> Wangzhou
> >> (B); robh+dt@kernel.org; james.morse@arm.com; Liviu.Dudau@arm.com;
> >> linux-pci@vger.kernel.org; linux-arm-kernel@lists.infradead.org;
> >> devicetree@vger.kernel.org; Yuanzhichang; Zhudacai; zhangjukuo;
> >> qiuzhenfa; Liguozhu (Kenneth); Jingoo Han; Pratyush Anand
> >> Subject: Re: [PATCH v6] PCI: Store PCIe bus address in struct
> >> of_pci_range
> >>
> >> On Thu, Jul 30, 2015 at 12:34 PM, Gabriele Paoloni
> >> <gabriele.paoloni@huawei.com> wrote:
> >>>> -----Original Message-----
> >>>> From: Bjorn Helgaas [mailto:bhelgaas@google.com]
> >>>> Sent: 30 July 2015 18:15
> >>>> On Thu, Jul 30, 2015 at 04:50:55PM +0000, Gabriele Paoloni wrote:
> >>>>>> -----Original Message-----
> >>>>>> From: linux-pci-owner@vger.kernel.org [mailto:linux-pci-
> >>>>>> owner@vger.kernel.org] On Behalf Of Bjorn Helgaas
> >>>>>> Sent: Thursday, July 30, 2015 5:15 PM
> >>>>>> On Thu, Jul 30, 2015 at 01:52:13PM +0000, Gabriele Paoloni wrote:
> >>
> >> [...]
> >>
> >>>>>>> I don’t think we should rely on [CPU] addresses...what if the
> >>>>>> intermediate
> >>>>>>> translation layer changes the lower significant bits of the
> >> "bus
> >>>>>> address"
> >>>>>>> to translate into a cpu address?
> >>>>>>
> >>>>>> Is it really a possiblity that the lower bits could be changed?
> >>>>>
> >>>>> I've checked all the current deignware users DTs except "pci-
> >>>> layerscape"
> >>>>> that I could not find:
> >>>>> spear1310.dtsi
> >>>>> spear1340.dtsi
> >>>>> dra7.dtsi
> >>>>> imx6qdl.dtsi
> >>>>> imx6sx.dtsi
> >>>>> keystone.dtsi
> >>>>> exynos5440.dtsi
> >>>>>
> >>>>> None of them modifies the lower bits. To be more precise the only
> >> guy
> >>>>> that provides another translation layer is "dra7.dtsi":
> >>>>> axi0
> >>>>> http://lxr.free-
> >> electrons.com/source/arch/arm/boot/dts/dra7.dtsi#L207
> >>>>>
> >>>>> axi1
> >>>>> http://lxr.free-
> >> electrons.com/source/arch/arm/boot/dts/dra7.dtsi#L241
> >>>>>
> >>>>> For this case masking the top 4bits (bits28 to 31) should make
> the
> >> job.
> >>
> >> IMO, we should just fix this case. After further study, I don't
> think
> >> this is a DW issue, but rather an SOC integration issue.
> >>
> >> I believe you can just fixup the address in the pp->ops->host_init
> hook.
> >>
> >
> > Yes I guess that I could just assign pp->(*)_mod_base to the CPU
> address
> > in DW and mask it out in dra7xx_pcie_host_init()...
> >
> > Kishon, would you be ok with that?
> 
> Initially I was using *base-mask* property from dt. Me and Arnd (cc'ed)
> had
> this discussion [1] before we decided the current approach. It'll be
> good to
> check with Arnd too.
> 
> [1] ->  http://lists.infradead.org/pipermail/linux-arm-kernel/2014-
> May/253528.html


In this patch you use the mask into designware....instead the approach 
proposed by Rob means to have the mask declared in the dra7 driver and
you modified the pp members in dra7xx_pcie_host_init by masking them...

BTW good to have Arnd opinion too..
> 
> Thanks
> Kishon

^ permalink raw reply	[flat|nested] 73+ messages in thread
* RE: [PATCH v6] PCI: Store PCIe bus address in struct of_pci_range
@ 2015-07-31 15:09                               ` Gabriele Paoloni
  0 siblings, 0 replies; 73+ messages in thread
From: Gabriele Paoloni @ 2015-07-31 15:09 UTC (permalink / raw)
  To: Kishon Vijay Abraham I, Rob Herring
  Cc: Bjorn Helgaas, arnd, lorenzo.pieralisi, Wangzhou (B),
	robh+dt, james.morse, Liviu.Dudau, linux-pci, linux-arm-kernel,
	devicetree, Yuanzhichang, Zhudacai, zhangjukuo, qiuzhenfa,
	Liguozhu (Kenneth),
	Jingoo Han, Pratyush Anand, Arnd Bergmann, Arnd Bergmann

DQoNCj4gLS0tLS1PcmlnaW5hbCBNZXNzYWdlLS0tLS0NCj4gRnJvbTogS2lzaG9uIFZpamF5IEFi
cmFoYW0gSSBbbWFpbHRvOmtpc2hvbkB0aS5jb21dDQo+IFNlbnQ6IEZyaWRheSwgSnVseSAzMSwg
MjAxNSAzOjU3IFBNDQo+IFRvOiBHYWJyaWVsZSBQYW9sb25pOyBSb2IgSGVycmluZw0KPiBDYzog
Qmpvcm4gSGVsZ2FhczsgYXJuZEBhcm5kYi5kZTsgbG9yZW56by5waWVyYWxpc2lAYXJtLmNvbTsg
V2FuZ3pob3UNCj4gKEIpOyByb2JoK2R0QGtlcm5lbC5vcmc7IGphbWVzLm1vcnNlQGFybS5jb207
IExpdml1LkR1ZGF1QGFybS5jb207DQo+IGxpbnV4LXBjaUB2Z2VyLmtlcm5lbC5vcmc7IGxpbnV4
LWFybS1rZXJuZWxAbGlzdHMuaW5mcmFkZWFkLm9yZzsNCj4gZGV2aWNldHJlZUB2Z2VyLmtlcm5l
bC5vcmc7IFl1YW56aGljaGFuZzsgWmh1ZGFjYWk7IHpoYW5nanVrdW87DQo+IHFpdXpoZW5mYTsg
TGlndW96aHUgKEtlbm5ldGgpOyBKaW5nb28gSGFuOyBQcmF0eXVzaCBBbmFuZDsgQXJuZA0KPiBC
ZXJnbWFubjsgQXJuZCBCZXJnbWFubg0KPiBTdWJqZWN0OiBSZTogW1BBVENIIHY2XSBQQ0k6IFN0
b3JlIFBDSWUgYnVzIGFkZHJlc3MgaW4gc3RydWN0DQo+IG9mX3BjaV9yYW5nZQ0KPiANCj4gK0Fy
bmQNCj4gDQo+IEhpLA0KPiANCj4gT24gRnJpZGF5IDMxIEp1bHkgMjAxNSAwNzo1NSBQTSwgR2Fi
cmllbGUgUGFvbG9uaSB3cm90ZToNCj4gPiBbK2NjIEtpc2hvbl0NCj4gPg0KPiA+PiAtLS0tLU9y
aWdpbmFsIE1lc3NhZ2UtLS0tLQ0KPiA+PiBGcm9tOiBsaW51eC1wY2ktb3duZXJAdmdlci5rZXJu
ZWwub3JnIFttYWlsdG86bGludXgtcGNpLQ0KPiA+PiBvd25lckB2Z2VyLmtlcm5lbC5vcmddIE9u
IEJlaGFsZiBPZiBSb2IgSGVycmluZw0KPiA+PiBTZW50OiBUaHVyc2RheSwgSnVseSAzMCwgMjAx
NSA5OjQyIFBNDQo+ID4+IFRvOiBHYWJyaWVsZSBQYW9sb25pDQo+ID4+IENjOiBCam9ybiBIZWxn
YWFzOyBhcm5kQGFybmRiLmRlOyBsb3JlbnpvLnBpZXJhbGlzaUBhcm0uY29tOw0KPiBXYW5nemhv
dQ0KPiA+PiAoQik7IHJvYmgrZHRAa2VybmVsLm9yZzsgamFtZXMubW9yc2VAYXJtLmNvbTsgTGl2
aXUuRHVkYXVAYXJtLmNvbTsNCj4gPj4gbGludXgtcGNpQHZnZXIua2VybmVsLm9yZzsgbGludXgt
YXJtLWtlcm5lbEBsaXN0cy5pbmZyYWRlYWQub3JnOw0KPiA+PiBkZXZpY2V0cmVlQHZnZXIua2Vy
bmVsLm9yZzsgWXVhbnpoaWNoYW5nOyBaaHVkYWNhaTsgemhhbmdqdWt1bzsNCj4gPj4gcWl1emhl
bmZhOyBMaWd1b3podSAoS2VubmV0aCk7IEppbmdvbyBIYW47IFByYXR5dXNoIEFuYW5kDQo+ID4+
IFN1YmplY3Q6IFJlOiBbUEFUQ0ggdjZdIFBDSTogU3RvcmUgUENJZSBidXMgYWRkcmVzcyBpbiBz
dHJ1Y3QNCj4gPj4gb2ZfcGNpX3JhbmdlDQo+ID4+DQo+ID4+IE9uIFRodSwgSnVsIDMwLCAyMDE1
IGF0IDEyOjM0IFBNLCBHYWJyaWVsZSBQYW9sb25pDQo+ID4+IDxnYWJyaWVsZS5wYW9sb25pQGh1
YXdlaS5jb20+IHdyb3RlOg0KPiA+Pj4+IC0tLS0tT3JpZ2luYWwgTWVzc2FnZS0tLS0tDQo+ID4+
Pj4gRnJvbTogQmpvcm4gSGVsZ2FhcyBbbWFpbHRvOmJoZWxnYWFzQGdvb2dsZS5jb21dDQo+ID4+
Pj4gU2VudDogMzAgSnVseSAyMDE1IDE4OjE1DQo+ID4+Pj4gT24gVGh1LCBKdWwgMzAsIDIwMTUg
YXQgMDQ6NTA6NTVQTSArMDAwMCwgR2FicmllbGUgUGFvbG9uaSB3cm90ZToNCj4gPj4+Pj4+IC0t
LS0tT3JpZ2luYWwgTWVzc2FnZS0tLS0tDQo+ID4+Pj4+PiBGcm9tOiBsaW51eC1wY2ktb3duZXJA
dmdlci5rZXJuZWwub3JnIFttYWlsdG86bGludXgtcGNpLQ0KPiA+Pj4+Pj4gb3duZXJAdmdlci5r
ZXJuZWwub3JnXSBPbiBCZWhhbGYgT2YgQmpvcm4gSGVsZ2Fhcw0KPiA+Pj4+Pj4gU2VudDogVGh1
cnNkYXksIEp1bHkgMzAsIDIwMTUgNToxNSBQTQ0KPiA+Pj4+Pj4gT24gVGh1LCBKdWwgMzAsIDIw
MTUgYXQgMDE6NTI6MTNQTSArMDAwMCwgR2FicmllbGUgUGFvbG9uaSB3cm90ZToNCj4gPj4NCj4g
Pj4gWy4uLl0NCj4gPj4NCj4gPj4+Pj4+PiBJIGRvbuKAmXQgdGhpbmsgd2Ugc2hvdWxkIHJlbHkg
b24gW0NQVV0gYWRkcmVzc2VzLi4ud2hhdCBpZiB0aGUNCj4gPj4+Pj4+IGludGVybWVkaWF0ZQ0K
PiA+Pj4+Pj4+IHRyYW5zbGF0aW9uIGxheWVyIGNoYW5nZXMgdGhlIGxvd2VyIHNpZ25pZmljYW50
IGJpdHMgb2YgdGhlDQo+ID4+ICJidXMNCj4gPj4+Pj4+IGFkZHJlc3MiDQo+ID4+Pj4+Pj4gdG8g
dHJhbnNsYXRlIGludG8gYSBjcHUgYWRkcmVzcz8NCj4gPj4+Pj4+DQo+ID4+Pj4+PiBJcyBpdCBy
ZWFsbHkgYSBwb3NzaWJsaXR5IHRoYXQgdGhlIGxvd2VyIGJpdHMgY291bGQgYmUgY2hhbmdlZD8N
Cj4gPj4+Pj4NCj4gPj4+Pj4gSSd2ZSBjaGVja2VkIGFsbCB0aGUgY3VycmVudCBkZWlnbndhcmUg
dXNlcnMgRFRzIGV4Y2VwdCAicGNpLQ0KPiA+Pj4+IGxheWVyc2NhcGUiDQo+ID4+Pj4+IHRoYXQg
SSBjb3VsZCBub3QgZmluZDoNCj4gPj4+Pj4gc3BlYXIxMzEwLmR0c2kNCj4gPj4+Pj4gc3BlYXIx
MzQwLmR0c2kNCj4gPj4+Pj4gZHJhNy5kdHNpDQo+ID4+Pj4+IGlteDZxZGwuZHRzaQ0KPiA+Pj4+
PiBpbXg2c3guZHRzaQ0KPiA+Pj4+PiBrZXlzdG9uZS5kdHNpDQo+ID4+Pj4+IGV4eW5vczU0NDAu
ZHRzaQ0KPiA+Pj4+Pg0KPiA+Pj4+PiBOb25lIG9mIHRoZW0gbW9kaWZpZXMgdGhlIGxvd2VyIGJp
dHMuIFRvIGJlIG1vcmUgcHJlY2lzZSB0aGUgb25seQ0KPiA+PiBndXkNCj4gPj4+Pj4gdGhhdCBw
cm92aWRlcyBhbm90aGVyIHRyYW5zbGF0aW9uIGxheWVyIGlzICJkcmE3LmR0c2kiOg0KPiA+Pj4+
PiBheGkwDQo+ID4+Pj4+IGh0dHA6Ly9seHIuZnJlZS0NCj4gPj4gZWxlY3Ryb25zLmNvbS9zb3Vy
Y2UvYXJjaC9hcm0vYm9vdC9kdHMvZHJhNy5kdHNpI0wyMDcNCj4gPj4+Pj4NCj4gPj4+Pj4gYXhp
MQ0KPiA+Pj4+PiBodHRwOi8vbHhyLmZyZWUtDQo+ID4+IGVsZWN0cm9ucy5jb20vc291cmNlL2Fy
Y2gvYXJtL2Jvb3QvZHRzL2RyYTcuZHRzaSNMMjQxDQo+ID4+Pj4+DQo+ID4+Pj4+IEZvciB0aGlz
IGNhc2UgbWFza2luZyB0aGUgdG9wIDRiaXRzIChiaXRzMjggdG8gMzEpIHNob3VsZCBtYWtlDQo+
IHRoZQ0KPiA+PiBqb2IuDQo+ID4+DQo+ID4+IElNTywgd2Ugc2hvdWxkIGp1c3QgZml4IHRoaXMg
Y2FzZS4gQWZ0ZXIgZnVydGhlciBzdHVkeSwgSSBkb24ndA0KPiB0aGluaw0KPiA+PiB0aGlzIGlz
IGEgRFcgaXNzdWUsIGJ1dCByYXRoZXIgYW4gU09DIGludGVncmF0aW9uIGlzc3VlLg0KPiA+Pg0K
PiA+PiBJIGJlbGlldmUgeW91IGNhbiBqdXN0IGZpeHVwIHRoZSBhZGRyZXNzIGluIHRoZSBwcC0+
b3BzLT5ob3N0X2luaXQNCj4gaG9vay4NCj4gPj4NCj4gPg0KPiA+IFllcyBJIGd1ZXNzIHRoYXQg
SSBjb3VsZCBqdXN0IGFzc2lnbiBwcC0+KCopX21vZF9iYXNlIHRvIHRoZSBDUFUNCj4gYWRkcmVz
cw0KPiA+IGluIERXIGFuZCBtYXNrIGl0IG91dCBpbiBkcmE3eHhfcGNpZV9ob3N0X2luaXQoKS4u
Lg0KPiA+DQo+ID4gS2lzaG9uLCB3b3VsZCB5b3UgYmUgb2sgd2l0aCB0aGF0Pw0KPiANCj4gSW5p
dGlhbGx5IEkgd2FzIHVzaW5nICpiYXNlLW1hc2sqIHByb3BlcnR5IGZyb20gZHQuIE1lIGFuZCBB
cm5kIChjYydlZCkNCj4gaGFkDQo+IHRoaXMgZGlzY3Vzc2lvbiBbMV0gYmVmb3JlIHdlIGRlY2lk
ZWQgdGhlIGN1cnJlbnQgYXBwcm9hY2guIEl0J2xsIGJlDQo+IGdvb2QgdG8NCj4gY2hlY2sgd2l0
aCBBcm5kIHRvby4NCj4gDQo+IFsxXSAtPiAgaHR0cDovL2xpc3RzLmluZnJhZGVhZC5vcmcvcGlw
ZXJtYWlsL2xpbnV4LWFybS1rZXJuZWwvMjAxNC0NCj4gTWF5LzI1MzUyOC5odG1sDQoNCg0KSW4g
dGhpcyBwYXRjaCB5b3UgdXNlIHRoZSBtYXNrIGludG8gZGVzaWdud2FyZS4uLi5pbnN0ZWFkIHRo
ZSBhcHByb2FjaCANCnByb3Bvc2VkIGJ5IFJvYiBtZWFucyB0byBoYXZlIHRoZSBtYXNrIGRlY2xh
cmVkIGluIHRoZSBkcmE3IGRyaXZlciBhbmQNCnlvdSBtb2RpZmllZCB0aGUgcHAgbWVtYmVycyBp
biBkcmE3eHhfcGNpZV9ob3N0X2luaXQgYnkgbWFza2luZyB0aGVtLi4uDQoNCkJUVyBnb29kIHRv
IGhhdmUgQXJuZCBvcGluaW9uIHRvby4uDQo+IA0KPiBUaGFua3MNCj4gS2lzaG9uDQo=

^ permalink raw reply	[flat|nested] 73+ messages in thread
* [PATCH v6] PCI: Store PCIe bus address in struct of_pci_range
@ 2015-07-31 15:09                               ` Gabriele Paoloni
  0 siblings, 0 replies; 73+ messages in thread
From: Gabriele Paoloni @ 2015-07-31 15:09 UTC (permalink / raw)
  To: linux-arm-kernel



> -----Original Message-----
> From: Kishon Vijay Abraham I [mailto:kishon at ti.com]
> Sent: Friday, July 31, 2015 3:57 PM
> To: Gabriele Paoloni; Rob Herring
> Cc: Bjorn Helgaas; arnd at arndb.de; lorenzo.pieralisi at arm.com; Wangzhou
> (B); robh+dt at kernel.org; james.morse at arm.com; Liviu.Dudau at arm.com;
> linux-pci at vger.kernel.org; linux-arm-kernel at lists.infradead.org;
> devicetree at vger.kernel.org; Yuanzhichang; Zhudacai; zhangjukuo;
> qiuzhenfa; Liguozhu (Kenneth); Jingoo Han; Pratyush Anand; Arnd
> Bergmann; Arnd Bergmann
> Subject: Re: [PATCH v6] PCI: Store PCIe bus address in struct
> of_pci_range
> 
> +Arnd
> 
> Hi,
> 
> On Friday 31 July 2015 07:55 PM, Gabriele Paoloni wrote:
> > [+cc Kishon]
> >
> >> -----Original Message-----
> >> From: linux-pci-owner at vger.kernel.org [mailto:linux-pci-
> >> owner at vger.kernel.org] On Behalf Of Rob Herring
> >> Sent: Thursday, July 30, 2015 9:42 PM
> >> To: Gabriele Paoloni
> >> Cc: Bjorn Helgaas; arnd at arndb.de; lorenzo.pieralisi at arm.com;
> Wangzhou
> >> (B); robh+dt at kernel.org; james.morse at arm.com; Liviu.Dudau at arm.com;
> >> linux-pci at vger.kernel.org; linux-arm-kernel at lists.infradead.org;
> >> devicetree at vger.kernel.org; Yuanzhichang; Zhudacai; zhangjukuo;
> >> qiuzhenfa; Liguozhu (Kenneth); Jingoo Han; Pratyush Anand
> >> Subject: Re: [PATCH v6] PCI: Store PCIe bus address in struct
> >> of_pci_range
> >>
> >> On Thu, Jul 30, 2015 at 12:34 PM, Gabriele Paoloni
> >> <gabriele.paoloni@huawei.com> wrote:
> >>>> -----Original Message-----
> >>>> From: Bjorn Helgaas [mailto:bhelgaas at google.com]
> >>>> Sent: 30 July 2015 18:15
> >>>> On Thu, Jul 30, 2015 at 04:50:55PM +0000, Gabriele Paoloni wrote:
> >>>>>> -----Original Message-----
> >>>>>> From: linux-pci-owner at vger.kernel.org [mailto:linux-pci-
> >>>>>> owner at vger.kernel.org] On Behalf Of Bjorn Helgaas
> >>>>>> Sent: Thursday, July 30, 2015 5:15 PM
> >>>>>> On Thu, Jul 30, 2015 at 01:52:13PM +0000, Gabriele Paoloni wrote:
> >>
> >> [...]
> >>
> >>>>>>> I don?t think we should rely on [CPU] addresses...what if the
> >>>>>> intermediate
> >>>>>>> translation layer changes the lower significant bits of the
> >> "bus
> >>>>>> address"
> >>>>>>> to translate into a cpu address?
> >>>>>>
> >>>>>> Is it really a possiblity that the lower bits could be changed?
> >>>>>
> >>>>> I've checked all the current deignware users DTs except "pci-
> >>>> layerscape"
> >>>>> that I could not find:
> >>>>> spear1310.dtsi
> >>>>> spear1340.dtsi
> >>>>> dra7.dtsi
> >>>>> imx6qdl.dtsi
> >>>>> imx6sx.dtsi
> >>>>> keystone.dtsi
> >>>>> exynos5440.dtsi
> >>>>>
> >>>>> None of them modifies the lower bits. To be more precise the only
> >> guy
> >>>>> that provides another translation layer is "dra7.dtsi":
> >>>>> axi0
> >>>>> http://lxr.free-
> >> electrons.com/source/arch/arm/boot/dts/dra7.dtsi#L207
> >>>>>
> >>>>> axi1
> >>>>> http://lxr.free-
> >> electrons.com/source/arch/arm/boot/dts/dra7.dtsi#L241
> >>>>>
> >>>>> For this case masking the top 4bits (bits28 to 31) should make
> the
> >> job.
> >>
> >> IMO, we should just fix this case. After further study, I don't
> think
> >> this is a DW issue, but rather an SOC integration issue.
> >>
> >> I believe you can just fixup the address in the pp->ops->host_init
> hook.
> >>
> >
> > Yes I guess that I could just assign pp->(*)_mod_base to the CPU
> address
> > in DW and mask it out in dra7xx_pcie_host_init()...
> >
> > Kishon, would you be ok with that?
> 
> Initially I was using *base-mask* property from dt. Me and Arnd (cc'ed)
> had
> this discussion [1] before we decided the current approach. It'll be
> good to
> check with Arnd too.
> 
> [1] ->  http://lists.infradead.org/pipermail/linux-arm-kernel/2014-
> May/253528.html


In this patch you use the mask into designware....instead the approach 
proposed by Rob means to have the mask declared in the dra7 driver and
you modified the pp members in dra7xx_pcie_host_init by masking them...

BTW good to have Arnd opinion too..
> 
> Thanks
> Kishon

^ permalink raw reply	[flat|nested] 73+ messages in thread
* Re: [PATCH v6] PCI: Store PCIe bus address in struct of_pci_range
  2015-07-31 14:57                             ` Kishon Vijay Abraham I
@ 2015-07-31 16:53                               ` Rob Herring
  -1 siblings, 0 replies; 73+ messages in thread
From: Rob Herring @ 2015-07-31 16:53 UTC (permalink / raw)
  To: Kishon Vijay Abraham I
  Cc: Gabriele Paoloni, Bjorn Helgaas, arnd, lorenzo.pieralisi,
	Wangzhou (B),
	robh+dt, james.morse, Liviu.Dudau, linux-pci, linux-arm-kernel,
	devicetree, Yuanzhichang, Zhudacai, zhangjukuo, qiuzhenfa,
	Liguozhu (Kenneth),
	Jingoo Han, Pratyush Anand, Arnd Bergmann

On Fri, Jul 31, 2015 at 9:57 AM, Kishon Vijay Abraham I <kishon@ti.com> wrote:
> +Arnd
>
> Hi,
>
> On Friday 31 July 2015 07:55 PM, Gabriele Paoloni wrote:
>> [+cc Kishon]
>>
>>> -----Original Message-----
>>> From: linux-pci-owner@vger.kernel.org [mailto:linux-pci-
>>> owner@vger.kernel.org] On Behalf Of Rob Herring
>>> Sent: Thursday, July 30, 2015 9:42 PM
>>> To: Gabriele Paoloni
>>> Cc: Bjorn Helgaas; arnd@arndb.de; lorenzo.pieralisi@arm.com; Wangzhou
>>> (B); robh+dt@kernel.org; james.morse@arm.com; Liviu.Dudau@arm.com;
>>> linux-pci@vger.kernel.org; linux-arm-kernel@lists.infradead.org;
>>> devicetree@vger.kernel.org; Yuanzhichang; Zhudacai; zhangjukuo;
>>> qiuzhenfa; Liguozhu (Kenneth); Jingoo Han; Pratyush Anand
>>> Subject: Re: [PATCH v6] PCI: Store PCIe bus address in struct
>>> of_pci_range
>>>
>>> On Thu, Jul 30, 2015 at 12:34 PM, Gabriele Paoloni
>>> <gabriele.paoloni@huawei.com> wrote:
>>>>> -----Original Message-----
>>>>> From: Bjorn Helgaas [mailto:bhelgaas@google.com]
>>>>> Sent: 30 July 2015 18:15
>>>>> On Thu, Jul 30, 2015 at 04:50:55PM +0000, Gabriele Paoloni wrote:
>>>>>>> -----Original Message-----
>>>>>>> From: linux-pci-owner@vger.kernel.org [mailto:linux-pci-
>>>>>>> owner@vger.kernel.org] On Behalf Of Bjorn Helgaas
>>>>>>> Sent: Thursday, July 30, 2015 5:15 PM
>>>>>>> On Thu, Jul 30, 2015 at 01:52:13PM +0000, Gabriele Paoloni wrote:
>>>
>>> [...]
>>>
>>>>>>>> I don’t think we should rely on [CPU] addresses...what if the
>>>>>>> intermediate
>>>>>>>> translation layer changes the lower significant bits of the
>>> "bus
>>>>>>> address"
>>>>>>>> to translate into a cpu address?
>>>>>>>
>>>>>>> Is it really a possiblity that the lower bits could be changed?
>>>>>>
>>>>>> I've checked all the current deignware users DTs except "pci-
>>>>> layerscape"
>>>>>> that I could not find:
>>>>>> spear1310.dtsi
>>>>>> spear1340.dtsi
>>>>>> dra7.dtsi
>>>>>> imx6qdl.dtsi
>>>>>> imx6sx.dtsi
>>>>>> keystone.dtsi
>>>>>> exynos5440.dtsi
>>>>>>
>>>>>> None of them modifies the lower bits. To be more precise the only
>>> guy
>>>>>> that provides another translation layer is "dra7.dtsi":
>>>>>> axi0
>>>>>> http://lxr.free-
>>> electrons.com/source/arch/arm/boot/dts/dra7.dtsi#L207
>>>>>>
>>>>>> axi1
>>>>>> http://lxr.free-
>>> electrons.com/source/arch/arm/boot/dts/dra7.dtsi#L241
>>>>>>
>>>>>> For this case masking the top 4bits (bits28 to 31) should make the
>>> job.
>>>
>>> IMO, we should just fix this case. After further study, I don't think
>>> this is a DW issue, but rather an SOC integration issue.
>>>
>>> I believe you can just fixup the address in the pp->ops->host_init hook.
>>>
>>
>> Yes I guess that I could just assign pp->(*)_mod_base to the CPU address
>> in DW and mask it out in dra7xx_pcie_host_init()...
>>
>> Kishon, would you be ok with that?
>
> Initially I was using *base-mask* property from dt. Me and Arnd (cc'ed) had
> this discussion [1] before we decided the current approach. It'll be good to
> check with Arnd too.
>
> [1] ->  http://lists.infradead.org/pipermail/linux-arm-kernel/2014-May/253528.html

The problem I have here is the use of ranges does not necessarily mean
fewer address bits are available. It can be used just for convenience
of not putting the full address into every node's reg property. And
vice versa, there are probably plenty of cases where we have the full
address in the nodes, but really only some of the address bits are
decoded at the IP block. Whether the address bits are present is
rarely cared about or known by s/w folks until you hit a problem like
this. Given this is an isolated case ATM, I would fix it in an
isolated way. It does not affect the binding and could be changed in
the kernel later if this becomes a common need.

Rob

^ permalink raw reply	[flat|nested] 73+ messages in thread
* [PATCH v6] PCI: Store PCIe bus address in struct of_pci_range
@ 2015-07-31 16:53                               ` Rob Herring
  0 siblings, 0 replies; 73+ messages in thread
From: Rob Herring @ 2015-07-31 16:53 UTC (permalink / raw)
  To: linux-arm-kernel

On Fri, Jul 31, 2015 at 9:57 AM, Kishon Vijay Abraham I <kishon@ti.com> wrote:
> +Arnd
>
> Hi,
>
> On Friday 31 July 2015 07:55 PM, Gabriele Paoloni wrote:
>> [+cc Kishon]
>>
>>> -----Original Message-----
>>> From: linux-pci-owner at vger.kernel.org [mailto:linux-pci-
>>> owner at vger.kernel.org] On Behalf Of Rob Herring
>>> Sent: Thursday, July 30, 2015 9:42 PM
>>> To: Gabriele Paoloni
>>> Cc: Bjorn Helgaas; arnd at arndb.de; lorenzo.pieralisi at arm.com; Wangzhou
>>> (B); robh+dt at kernel.org; james.morse at arm.com; Liviu.Dudau at arm.com;
>>> linux-pci at vger.kernel.org; linux-arm-kernel at lists.infradead.org;
>>> devicetree at vger.kernel.org; Yuanzhichang; Zhudacai; zhangjukuo;
>>> qiuzhenfa; Liguozhu (Kenneth); Jingoo Han; Pratyush Anand
>>> Subject: Re: [PATCH v6] PCI: Store PCIe bus address in struct
>>> of_pci_range
>>>
>>> On Thu, Jul 30, 2015 at 12:34 PM, Gabriele Paoloni
>>> <gabriele.paoloni@huawei.com> wrote:
>>>>> -----Original Message-----
>>>>> From: Bjorn Helgaas [mailto:bhelgaas at google.com]
>>>>> Sent: 30 July 2015 18:15
>>>>> On Thu, Jul 30, 2015 at 04:50:55PM +0000, Gabriele Paoloni wrote:
>>>>>>> -----Original Message-----
>>>>>>> From: linux-pci-owner at vger.kernel.org [mailto:linux-pci-
>>>>>>> owner at vger.kernel.org] On Behalf Of Bjorn Helgaas
>>>>>>> Sent: Thursday, July 30, 2015 5:15 PM
>>>>>>> On Thu, Jul 30, 2015 at 01:52:13PM +0000, Gabriele Paoloni wrote:
>>>
>>> [...]
>>>
>>>>>>>> I don?t think we should rely on [CPU] addresses...what if the
>>>>>>> intermediate
>>>>>>>> translation layer changes the lower significant bits of the
>>> "bus
>>>>>>> address"
>>>>>>>> to translate into a cpu address?
>>>>>>>
>>>>>>> Is it really a possiblity that the lower bits could be changed?
>>>>>>
>>>>>> I've checked all the current deignware users DTs except "pci-
>>>>> layerscape"
>>>>>> that I could not find:
>>>>>> spear1310.dtsi
>>>>>> spear1340.dtsi
>>>>>> dra7.dtsi
>>>>>> imx6qdl.dtsi
>>>>>> imx6sx.dtsi
>>>>>> keystone.dtsi
>>>>>> exynos5440.dtsi
>>>>>>
>>>>>> None of them modifies the lower bits. To be more precise the only
>>> guy
>>>>>> that provides another translation layer is "dra7.dtsi":
>>>>>> axi0
>>>>>> http://lxr.free-
>>> electrons.com/source/arch/arm/boot/dts/dra7.dtsi#L207
>>>>>>
>>>>>> axi1
>>>>>> http://lxr.free-
>>> electrons.com/source/arch/arm/boot/dts/dra7.dtsi#L241
>>>>>>
>>>>>> For this case masking the top 4bits (bits28 to 31) should make the
>>> job.
>>>
>>> IMO, we should just fix this case. After further study, I don't think
>>> this is a DW issue, but rather an SOC integration issue.
>>>
>>> I believe you can just fixup the address in the pp->ops->host_init hook.
>>>
>>
>> Yes I guess that I could just assign pp->(*)_mod_base to the CPU address
>> in DW and mask it out in dra7xx_pcie_host_init()...
>>
>> Kishon, would you be ok with that?
>
> Initially I was using *base-mask* property from dt. Me and Arnd (cc'ed) had
> this discussion [1] before we decided the current approach. It'll be good to
> check with Arnd too.
>
> [1] ->  http://lists.infradead.org/pipermail/linux-arm-kernel/2014-May/253528.html

The problem I have here is the use of ranges does not necessarily mean
fewer address bits are available. It can be used just for convenience
of not putting the full address into every node's reg property. And
vice versa, there are probably plenty of cases where we have the full
address in the nodes, but really only some of the address bits are
decoded at the IP block. Whether the address bits are present is
rarely cared about or known by s/w folks until you hit a problem like
this. Given this is an isolated case ATM, I would fix it in an
isolated way. It does not affect the binding and could be changed in
the kernel later if this becomes a common need.

Rob

^ permalink raw reply	[flat|nested] 73+ messages in thread
* RE: [PATCH v6] PCI: Store PCIe bus address in struct of_pci_range
  2015-07-31 16:53                               ` Rob Herring
  (?)
@ 2015-08-03 11:18                                 ` Gabriele Paoloni
  -1 siblings, 0 replies; 73+ messages in thread
From: Gabriele Paoloni @ 2015-08-03 11:18 UTC (permalink / raw)
  To: Rob Herring, Kishon Vijay Abraham I
  Cc: Bjorn Helgaas, arnd, lorenzo.pieralisi, Wangzhou (B),
	robh+dt, james.morse, Liviu.Dudau, linux-pci, linux-arm-kernel,
	devicetree, Yuanzhichang, Zhudacai, zhangjukuo, qiuzhenfa,
	Liguozhu (Kenneth),
	Jingoo Han, Pratyush Anand, Arnd Bergmann

Rob, Kishon what about the following solution?....

---
 drivers/pci/host/pci-dra7xx.c      | 12 ++++++++++++
 drivers/pci/host/pcie-designware.c |  9 +++------
 2 files changed, 15 insertions(+), 6 deletions(-)

diff --git a/drivers/pci/host/pci-dra7xx.c b/drivers/pci/host/pci-dra7xx.c
index 80db09e..bb2635f 100644
--- a/drivers/pci/host/pci-dra7xx.c
+++ b/drivers/pci/host/pci-dra7xx.c
@@ -61,6 +61,7 @@
 
 #define	PCIECTRL_DRA7XX_CONF_PHY_CS			0x010C
 #define	LINK_UP						BIT(16)
+#define CPU_TO_BUS_ADDR             0x0FFFFFFF
 
 struct dra7xx_pcie {
 	void __iomem		*base;
@@ -138,6 +139,17 @@ static void dra7xx_pcie_enable_interrupts(struct pcie_port *pp)
 
 static void dra7xx_pcie_host_init(struct pcie_port *pp)
 {
+	if (pp->io_mod_base)
+		pp->io_mod_base &= CPU_TO_BUS_ADDR;
+
+	if (pp->mem_mod_base)
+		pp->mem_mod_base &= CPU_TO_BUS_ADDR;
+
+	if (pp->cfg0_mod_base) {
+		pp->cfg0_mod_base &= CPU_TO_BUS_ADDR;
+		pp->cfg1_mod_base &= CPU_TO_BUS_ADDR;
+	}
+
 	dw_pcie_setup_rc(pp);
 	dra7xx_pcie_establish_link(pp);
 	if (IS_ENABLED(CONFIG_PCI_MSI))
diff --git a/drivers/pci/host/pcie-designware.c b/drivers/pci/host/pcie-designware.c
index 69486be..06c682b 100644
--- a/drivers/pci/host/pcie-designware.c
+++ b/drivers/pci/host/pcie-designware.c
@@ -416,8 +416,7 @@ int dw_pcie_host_init(struct pcie_port *pp)
 			pp->io_base = range.cpu_addr;
 
 			/* Find the untranslated IO space address */
-			pp->io_mod_base = of_read_number(parser.range -
-							 parser.np + na, ns);
+			pp->io_mod_base = range.cpu_addr;;
 		}
 		if (restype == IORESOURCE_MEM) {
 			of_pci_range_to_resource(&range, np, &pp->mem);
@@ -426,8 +425,7 @@ int dw_pcie_host_init(struct pcie_port *pp)
 			pp->mem_bus_addr = range.pci_addr;
 
 			/* Find the untranslated MEM space address */
-			pp->mem_mod_base = of_read_number(parser.range -
-							  parser.np + na, ns);
+			pp->mem_mod_base = range.cpu_addr;
 		}
 		if (restype == 0) {
 			of_pci_range_to_resource(&range, np, &pp->cfg);
@@ -437,8 +435,7 @@ int dw_pcie_host_init(struct pcie_port *pp)
 			pp->cfg1_base = pp->cfg.start + pp->cfg0_size;
 
 			/* Find the untranslated configuration space address */
-			pp->cfg0_mod_base = of_read_number(parser.range -
-							   parser.np + na, ns);
+			pp->cfg0_mod_base = range.cpu_addr;
 			pp->cfg1_mod_base = pp->cfg0_mod_base +
 					    pp->cfg0_size;
 		}
-- 
1.9.1


> -----Original Message-----
> From: linux-pci-owner@vger.kernel.org [mailto:linux-pci-
> owner@vger.kernel.org] On Behalf Of Rob Herring
> Sent: Friday, July 31, 2015 5:53 PM
> To: Kishon Vijay Abraham I
> Cc: Gabriele Paoloni; Bjorn Helgaas; arnd@arndb.de;
> lorenzo.pieralisi@arm.com; Wangzhou (B); robh+dt@kernel.org;
> james.morse@arm.com; Liviu.Dudau@arm.com; linux-pci@vger.kernel.org;
> linux-arm-kernel@lists.infradead.org; devicetree@vger.kernel.org;
> Yuanzhichang; Zhudacai; zhangjukuo; qiuzhenfa; Liguozhu (Kenneth);
> Jingoo Han; Pratyush Anand; Arnd Bergmann
> Subject: Re: [PATCH v6] PCI: Store PCIe bus address in struct
> of_pci_range
> 
> On Fri, Jul 31, 2015 at 9:57 AM, Kishon Vijay Abraham I <kishon@ti.com>
> wrote:
> > +Arnd
> >
> > Hi,
> >
> > On Friday 31 July 2015 07:55 PM, Gabriele Paoloni wrote:
> >> [+cc Kishon]
> >>
> >>> -----Original Message-----
> >>> From: linux-pci-owner@vger.kernel.org [mailto:linux-pci-
> >>> owner@vger.kernel.org] On Behalf Of Rob Herring
> >>> Sent: Thursday, July 30, 2015 9:42 PM
> >>> To: Gabriele Paoloni
> >>> Cc: Bjorn Helgaas; arnd@arndb.de; lorenzo.pieralisi@arm.com;
> Wangzhou
> >>> (B); robh+dt@kernel.org; james.morse@arm.com; Liviu.Dudau@arm.com;
> >>> linux-pci@vger.kernel.org; linux-arm-kernel@lists.infradead.org;
> >>> devicetree@vger.kernel.org; Yuanzhichang; Zhudacai; zhangjukuo;
> >>> qiuzhenfa; Liguozhu (Kenneth); Jingoo Han; Pratyush Anand
> >>> Subject: Re: [PATCH v6] PCI: Store PCIe bus address in struct
> >>> of_pci_range
> >>>
> >>> On Thu, Jul 30, 2015 at 12:34 PM, Gabriele Paoloni
> >>> <gabriele.paoloni@huawei.com> wrote:
> >>>>> -----Original Message-----
> >>>>> From: Bjorn Helgaas [mailto:bhelgaas@google.com]
> >>>>> Sent: 30 July 2015 18:15
> >>>>> On Thu, Jul 30, 2015 at 04:50:55PM +0000, Gabriele Paoloni wrote:
> >>>>>>> -----Original Message-----
> >>>>>>> From: linux-pci-owner@vger.kernel.org [mailto:linux-pci-
> >>>>>>> owner@vger.kernel.org] On Behalf Of Bjorn Helgaas
> >>>>>>> Sent: Thursday, July 30, 2015 5:15 PM
> >>>>>>> On Thu, Jul 30, 2015 at 01:52:13PM +0000, Gabriele Paoloni
> wrote:
> >>>
> >>> [...]
> >>>
> >>>>>>>> I don’t think we should rely on [CPU] addresses...what if the
> >>>>>>> intermediate
> >>>>>>>> translation layer changes the lower significant bits of the
> >>> "bus
> >>>>>>> address"
> >>>>>>>> to translate into a cpu address?
> >>>>>>>
> >>>>>>> Is it really a possiblity that the lower bits could be changed?
> >>>>>>
> >>>>>> I've checked all the current deignware users DTs except "pci-
> >>>>> layerscape"
> >>>>>> that I could not find:
> >>>>>> spear1310.dtsi
> >>>>>> spear1340.dtsi
> >>>>>> dra7.dtsi
> >>>>>> imx6qdl.dtsi
> >>>>>> imx6sx.dtsi
> >>>>>> keystone.dtsi
> >>>>>> exynos5440.dtsi
> >>>>>>
> >>>>>> None of them modifies the lower bits. To be more precise the
> only
> >>> guy
> >>>>>> that provides another translation layer is "dra7.dtsi":
> >>>>>> axi0
> >>>>>> http://lxr.free-
> >>> electrons.com/source/arch/arm/boot/dts/dra7.dtsi#L207
> >>>>>>
> >>>>>> axi1
> >>>>>> http://lxr.free-
> >>> electrons.com/source/arch/arm/boot/dts/dra7.dtsi#L241
> >>>>>>
> >>>>>> For this case masking the top 4bits (bits28 to 31) should make
> the
> >>> job.
> >>>
> >>> IMO, we should just fix this case. After further study, I don't
> think
> >>> this is a DW issue, but rather an SOC integration issue.
> >>>
> >>> I believe you can just fixup the address in the pp->ops->host_init
> hook.
> >>>
> >>
> >> Yes I guess that I could just assign pp->(*)_mod_base to the CPU
> address
> >> in DW and mask it out in dra7xx_pcie_host_init()...
> >>
> >> Kishon, would you be ok with that?
> >
> > Initially I was using *base-mask* property from dt. Me and Arnd
> (cc'ed) had
> > this discussion [1] before we decided the current approach. It'll be
> good to
> > check with Arnd too.
> >
> > [1] ->  http://lists.infradead.org/pipermail/linux-arm-kernel/2014-
> May/253528.html
> 
> The problem I have here is the use of ranges does not necessarily mean
> fewer address bits are available. It can be used just for convenience
> of not putting the full address into every node's reg property. And
> vice versa, there are probably plenty of cases where we have the full
> address in the nodes, but really only some of the address bits are
> decoded at the IP block. Whether the address bits are present is
> rarely cared about or known by s/w folks until you hit a problem like
> this. Given this is an isolated case ATM, I would fix it in an
> isolated way. It does not affect the binding and could be changed in
> the kernel later if this becomes a common need.
> 
> Rob
> --
> To unsubscribe from this list: send the line "unsubscribe linux-pci" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply related	[flat|nested] 73+ messages in thread
* RE: [PATCH v6] PCI: Store PCIe bus address in struct of_pci_range
@ 2015-08-03 11:18                                 ` Gabriele Paoloni
  0 siblings, 0 replies; 73+ messages in thread
From: Gabriele Paoloni @ 2015-08-03 11:18 UTC (permalink / raw)
  To: Rob Herring, Kishon Vijay Abraham I
  Cc: Bjorn Helgaas, arnd, lorenzo.pieralisi, Wangzhou (B),
	robh+dt, james.morse, Liviu.Dudau, linux-pci, linux-arm-kernel,
	devicetree, Yuanzhichang, Zhudacai, zhangjukuo, qiuzhenfa,
	Liguozhu (Kenneth),
	Jingoo Han, Pratyush Anand, Arnd Bergmann

Um9iLCBLaXNob24gd2hhdCBhYm91dCB0aGUgZm9sbG93aW5nIHNvbHV0aW9uPy4uLi4NCg0KLS0t
DQogZHJpdmVycy9wY2kvaG9zdC9wY2ktZHJhN3h4LmMgICAgICB8IDEyICsrKysrKysrKysrKw0K
IGRyaXZlcnMvcGNpL2hvc3QvcGNpZS1kZXNpZ253YXJlLmMgfCAgOSArKystLS0tLS0NCiAyIGZp
bGVzIGNoYW5nZWQsIDE1IGluc2VydGlvbnMoKyksIDYgZGVsZXRpb25zKC0pDQoNCmRpZmYgLS1n
aXQgYS9kcml2ZXJzL3BjaS9ob3N0L3BjaS1kcmE3eHguYyBiL2RyaXZlcnMvcGNpL2hvc3QvcGNp
LWRyYTd4eC5jDQppbmRleCA4MGRiMDllLi5iYjI2MzVmIDEwMDY0NA0KLS0tIGEvZHJpdmVycy9w
Y2kvaG9zdC9wY2ktZHJhN3h4LmMNCisrKyBiL2RyaXZlcnMvcGNpL2hvc3QvcGNpLWRyYTd4eC5j
DQpAQCAtNjEsNiArNjEsNyBAQA0KIA0KICNkZWZpbmUJUENJRUNUUkxfRFJBN1hYX0NPTkZfUEhZ
X0NTCQkJMHgwMTBDDQogI2RlZmluZQlMSU5LX1VQCQkJCQkJQklUKDE2KQ0KKyNkZWZpbmUgQ1BV
X1RPX0JVU19BRERSICAgICAgICAgICAgIDB4MEZGRkZGRkYNCiANCiBzdHJ1Y3QgZHJhN3h4X3Bj
aWUgew0KIAl2b2lkIF9faW9tZW0JCSpiYXNlOw0KQEAgLTEzOCw2ICsxMzksMTcgQEAgc3RhdGlj
IHZvaWQgZHJhN3h4X3BjaWVfZW5hYmxlX2ludGVycnVwdHMoc3RydWN0IHBjaWVfcG9ydCAqcHAp
DQogDQogc3RhdGljIHZvaWQgZHJhN3h4X3BjaWVfaG9zdF9pbml0KHN0cnVjdCBwY2llX3BvcnQg
KnBwKQ0KIHsNCisJaWYgKHBwLT5pb19tb2RfYmFzZSkNCisJCXBwLT5pb19tb2RfYmFzZSAmPSBD
UFVfVE9fQlVTX0FERFI7DQorDQorCWlmIChwcC0+bWVtX21vZF9iYXNlKQ0KKwkJcHAtPm1lbV9t
b2RfYmFzZSAmPSBDUFVfVE9fQlVTX0FERFI7DQorDQorCWlmIChwcC0+Y2ZnMF9tb2RfYmFzZSkg
ew0KKwkJcHAtPmNmZzBfbW9kX2Jhc2UgJj0gQ1BVX1RPX0JVU19BRERSOw0KKwkJcHAtPmNmZzFf
bW9kX2Jhc2UgJj0gQ1BVX1RPX0JVU19BRERSOw0KKwl9DQorDQogCWR3X3BjaWVfc2V0dXBfcmMo
cHApOw0KIAlkcmE3eHhfcGNpZV9lc3RhYmxpc2hfbGluayhwcCk7DQogCWlmIChJU19FTkFCTEVE
KENPTkZJR19QQ0lfTVNJKSkNCmRpZmYgLS1naXQgYS9kcml2ZXJzL3BjaS9ob3N0L3BjaWUtZGVz
aWdud2FyZS5jIGIvZHJpdmVycy9wY2kvaG9zdC9wY2llLWRlc2lnbndhcmUuYw0KaW5kZXggNjk0
ODZiZS4uMDZjNjgyYiAxMDA2NDQNCi0tLSBhL2RyaXZlcnMvcGNpL2hvc3QvcGNpZS1kZXNpZ253
YXJlLmMNCisrKyBiL2RyaXZlcnMvcGNpL2hvc3QvcGNpZS1kZXNpZ253YXJlLmMNCkBAIC00MTYs
OCArNDE2LDcgQEAgaW50IGR3X3BjaWVfaG9zdF9pbml0KHN0cnVjdCBwY2llX3BvcnQgKnBwKQ0K
IAkJCXBwLT5pb19iYXNlID0gcmFuZ2UuY3B1X2FkZHI7DQogDQogCQkJLyogRmluZCB0aGUgdW50
cmFuc2xhdGVkIElPIHNwYWNlIGFkZHJlc3MgKi8NCi0JCQlwcC0+aW9fbW9kX2Jhc2UgPSBvZl9y
ZWFkX251bWJlcihwYXJzZXIucmFuZ2UgLQ0KLQkJCQkJCQkgcGFyc2VyLm5wICsgbmEsIG5zKTsN
CisJCQlwcC0+aW9fbW9kX2Jhc2UgPSByYW5nZS5jcHVfYWRkcjs7DQogCQl9DQogCQlpZiAocmVz
dHlwZSA9PSBJT1JFU09VUkNFX01FTSkgew0KIAkJCW9mX3BjaV9yYW5nZV90b19yZXNvdXJjZSgm
cmFuZ2UsIG5wLCAmcHAtPm1lbSk7DQpAQCAtNDI2LDggKzQyNSw3IEBAIGludCBkd19wY2llX2hv
c3RfaW5pdChzdHJ1Y3QgcGNpZV9wb3J0ICpwcCkNCiAJCQlwcC0+bWVtX2J1c19hZGRyID0gcmFu
Z2UucGNpX2FkZHI7DQogDQogCQkJLyogRmluZCB0aGUgdW50cmFuc2xhdGVkIE1FTSBzcGFjZSBh
ZGRyZXNzICovDQotCQkJcHAtPm1lbV9tb2RfYmFzZSA9IG9mX3JlYWRfbnVtYmVyKHBhcnNlci5y
YW5nZSAtDQotCQkJCQkJCSAgcGFyc2VyLm5wICsgbmEsIG5zKTsNCisJCQlwcC0+bWVtX21vZF9i
YXNlID0gcmFuZ2UuY3B1X2FkZHI7DQogCQl9DQogCQlpZiAocmVzdHlwZSA9PSAwKSB7DQogCQkJ
b2ZfcGNpX3JhbmdlX3RvX3Jlc291cmNlKCZyYW5nZSwgbnAsICZwcC0+Y2ZnKTsNCkBAIC00Mzcs
OCArNDM1LDcgQEAgaW50IGR3X3BjaWVfaG9zdF9pbml0KHN0cnVjdCBwY2llX3BvcnQgKnBwKQ0K
IAkJCXBwLT5jZmcxX2Jhc2UgPSBwcC0+Y2ZnLnN0YXJ0ICsgcHAtPmNmZzBfc2l6ZTsNCiANCiAJ
CQkvKiBGaW5kIHRoZSB1bnRyYW5zbGF0ZWQgY29uZmlndXJhdGlvbiBzcGFjZSBhZGRyZXNzICov
DQotCQkJcHAtPmNmZzBfbW9kX2Jhc2UgPSBvZl9yZWFkX251bWJlcihwYXJzZXIucmFuZ2UgLQ0K
LQkJCQkJCQkgICBwYXJzZXIubnAgKyBuYSwgbnMpOw0KKwkJCXBwLT5jZmcwX21vZF9iYXNlID0g
cmFuZ2UuY3B1X2FkZHI7DQogCQkJcHAtPmNmZzFfbW9kX2Jhc2UgPSBwcC0+Y2ZnMF9tb2RfYmFz
ZSArDQogCQkJCQkgICAgcHAtPmNmZzBfc2l6ZTsNCiAJCX0NCi0tIA0KMS45LjENCg0KDQo+IC0t
LS0tT3JpZ2luYWwgTWVzc2FnZS0tLS0tDQo+IEZyb206IGxpbnV4LXBjaS1vd25lckB2Z2VyLmtl
cm5lbC5vcmcgW21haWx0bzpsaW51eC1wY2ktDQo+IG93bmVyQHZnZXIua2VybmVsLm9yZ10gT24g
QmVoYWxmIE9mIFJvYiBIZXJyaW5nDQo+IFNlbnQ6IEZyaWRheSwgSnVseSAzMSwgMjAxNSA1OjUz
IFBNDQo+IFRvOiBLaXNob24gVmlqYXkgQWJyYWhhbSBJDQo+IENjOiBHYWJyaWVsZSBQYW9sb25p
OyBCam9ybiBIZWxnYWFzOyBhcm5kQGFybmRiLmRlOw0KPiBsb3JlbnpvLnBpZXJhbGlzaUBhcm0u
Y29tOyBXYW5nemhvdSAoQik7IHJvYmgrZHRAa2VybmVsLm9yZzsNCj4gamFtZXMubW9yc2VAYXJt
LmNvbTsgTGl2aXUuRHVkYXVAYXJtLmNvbTsgbGludXgtcGNpQHZnZXIua2VybmVsLm9yZzsNCj4g
bGludXgtYXJtLWtlcm5lbEBsaXN0cy5pbmZyYWRlYWQub3JnOyBkZXZpY2V0cmVlQHZnZXIua2Vy
bmVsLm9yZzsNCj4gWXVhbnpoaWNoYW5nOyBaaHVkYWNhaTsgemhhbmdqdWt1bzsgcWl1emhlbmZh
OyBMaWd1b3podSAoS2VubmV0aCk7DQo+IEppbmdvbyBIYW47IFByYXR5dXNoIEFuYW5kOyBBcm5k
IEJlcmdtYW5uDQo+IFN1YmplY3Q6IFJlOiBbUEFUQ0ggdjZdIFBDSTogU3RvcmUgUENJZSBidXMg
YWRkcmVzcyBpbiBzdHJ1Y3QNCj4gb2ZfcGNpX3JhbmdlDQo+IA0KPiBPbiBGcmksIEp1bCAzMSwg
MjAxNSBhdCA5OjU3IEFNLCBLaXNob24gVmlqYXkgQWJyYWhhbSBJIDxraXNob25AdGkuY29tPg0K
PiB3cm90ZToNCj4gPiArQXJuZA0KPiA+DQo+ID4gSGksDQo+ID4NCj4gPiBPbiBGcmlkYXkgMzEg
SnVseSAyMDE1IDA3OjU1IFBNLCBHYWJyaWVsZSBQYW9sb25pIHdyb3RlOg0KPiA+PiBbK2NjIEtp
c2hvbl0NCj4gPj4NCj4gPj4+IC0tLS0tT3JpZ2luYWwgTWVzc2FnZS0tLS0tDQo+ID4+PiBGcm9t
OiBsaW51eC1wY2ktb3duZXJAdmdlci5rZXJuZWwub3JnIFttYWlsdG86bGludXgtcGNpLQ0KPiA+
Pj4gb3duZXJAdmdlci5rZXJuZWwub3JnXSBPbiBCZWhhbGYgT2YgUm9iIEhlcnJpbmcNCj4gPj4+
IFNlbnQ6IFRodXJzZGF5LCBKdWx5IDMwLCAyMDE1IDk6NDIgUE0NCj4gPj4+IFRvOiBHYWJyaWVs
ZSBQYW9sb25pDQo+ID4+PiBDYzogQmpvcm4gSGVsZ2FhczsgYXJuZEBhcm5kYi5kZTsgbG9yZW56
by5waWVyYWxpc2lAYXJtLmNvbTsNCj4gV2FuZ3pob3UNCj4gPj4+IChCKTsgcm9iaCtkdEBrZXJu
ZWwub3JnOyBqYW1lcy5tb3JzZUBhcm0uY29tOyBMaXZpdS5EdWRhdUBhcm0uY29tOw0KPiA+Pj4g
bGludXgtcGNpQHZnZXIua2VybmVsLm9yZzsgbGludXgtYXJtLWtlcm5lbEBsaXN0cy5pbmZyYWRl
YWQub3JnOw0KPiA+Pj4gZGV2aWNldHJlZUB2Z2VyLmtlcm5lbC5vcmc7IFl1YW56aGljaGFuZzsg
Wmh1ZGFjYWk7IHpoYW5nanVrdW87DQo+ID4+PiBxaXV6aGVuZmE7IExpZ3Vvemh1IChLZW5uZXRo
KTsgSmluZ29vIEhhbjsgUHJhdHl1c2ggQW5hbmQNCj4gPj4+IFN1YmplY3Q6IFJlOiBbUEFUQ0gg
djZdIFBDSTogU3RvcmUgUENJZSBidXMgYWRkcmVzcyBpbiBzdHJ1Y3QNCj4gPj4+IG9mX3BjaV9y
YW5nZQ0KPiA+Pj4NCj4gPj4+IE9uIFRodSwgSnVsIDMwLCAyMDE1IGF0IDEyOjM0IFBNLCBHYWJy
aWVsZSBQYW9sb25pDQo+ID4+PiA8Z2FicmllbGUucGFvbG9uaUBodWF3ZWkuY29tPiB3cm90ZToN
Cj4gPj4+Pj4gLS0tLS1PcmlnaW5hbCBNZXNzYWdlLS0tLS0NCj4gPj4+Pj4gRnJvbTogQmpvcm4g
SGVsZ2FhcyBbbWFpbHRvOmJoZWxnYWFzQGdvb2dsZS5jb21dDQo+ID4+Pj4+IFNlbnQ6IDMwIEp1
bHkgMjAxNSAxODoxNQ0KPiA+Pj4+PiBPbiBUaHUsIEp1bCAzMCwgMjAxNSBhdCAwNDo1MDo1NVBN
ICswMDAwLCBHYWJyaWVsZSBQYW9sb25pIHdyb3RlOg0KPiA+Pj4+Pj4+IC0tLS0tT3JpZ2luYWwg
TWVzc2FnZS0tLS0tDQo+ID4+Pj4+Pj4gRnJvbTogbGludXgtcGNpLW93bmVyQHZnZXIua2VybmVs
Lm9yZyBbbWFpbHRvOmxpbnV4LXBjaS0NCj4gPj4+Pj4+PiBvd25lckB2Z2VyLmtlcm5lbC5vcmdd
IE9uIEJlaGFsZiBPZiBCam9ybiBIZWxnYWFzDQo+ID4+Pj4+Pj4gU2VudDogVGh1cnNkYXksIEp1
bHkgMzAsIDIwMTUgNToxNSBQTQ0KPiA+Pj4+Pj4+IE9uIFRodSwgSnVsIDMwLCAyMDE1IGF0IDAx
OjUyOjEzUE0gKzAwMDAsIEdhYnJpZWxlIFBhb2xvbmkNCj4gd3JvdGU6DQo+ID4+Pg0KPiA+Pj4g
Wy4uLl0NCj4gPj4+DQo+ID4+Pj4+Pj4+IEkgZG9u4oCZdCB0aGluayB3ZSBzaG91bGQgcmVseSBv
biBbQ1BVXSBhZGRyZXNzZXMuLi53aGF0IGlmIHRoZQ0KPiA+Pj4+Pj4+IGludGVybWVkaWF0ZQ0K
PiA+Pj4+Pj4+PiB0cmFuc2xhdGlvbiBsYXllciBjaGFuZ2VzIHRoZSBsb3dlciBzaWduaWZpY2Fu
dCBiaXRzIG9mIHRoZQ0KPiA+Pj4gImJ1cw0KPiA+Pj4+Pj4+IGFkZHJlc3MiDQo+ID4+Pj4+Pj4+
IHRvIHRyYW5zbGF0ZSBpbnRvIGEgY3B1IGFkZHJlc3M/DQo+ID4+Pj4+Pj4NCj4gPj4+Pj4+PiBJ
cyBpdCByZWFsbHkgYSBwb3NzaWJsaXR5IHRoYXQgdGhlIGxvd2VyIGJpdHMgY291bGQgYmUgY2hh
bmdlZD8NCj4gPj4+Pj4+DQo+ID4+Pj4+PiBJJ3ZlIGNoZWNrZWQgYWxsIHRoZSBjdXJyZW50IGRl
aWdud2FyZSB1c2VycyBEVHMgZXhjZXB0ICJwY2ktDQo+ID4+Pj4+IGxheWVyc2NhcGUiDQo+ID4+
Pj4+PiB0aGF0IEkgY291bGQgbm90IGZpbmQ6DQo+ID4+Pj4+PiBzcGVhcjEzMTAuZHRzaQ0KPiA+
Pj4+Pj4gc3BlYXIxMzQwLmR0c2kNCj4gPj4+Pj4+IGRyYTcuZHRzaQ0KPiA+Pj4+Pj4gaW14NnFk
bC5kdHNpDQo+ID4+Pj4+PiBpbXg2c3guZHRzaQ0KPiA+Pj4+Pj4ga2V5c3RvbmUuZHRzaQ0KPiA+
Pj4+Pj4gZXh5bm9zNTQ0MC5kdHNpDQo+ID4+Pj4+Pg0KPiA+Pj4+Pj4gTm9uZSBvZiB0aGVtIG1v
ZGlmaWVzIHRoZSBsb3dlciBiaXRzLiBUbyBiZSBtb3JlIHByZWNpc2UgdGhlDQo+IG9ubHkNCj4g
Pj4+IGd1eQ0KPiA+Pj4+Pj4gdGhhdCBwcm92aWRlcyBhbm90aGVyIHRyYW5zbGF0aW9uIGxheWVy
IGlzICJkcmE3LmR0c2kiOg0KPiA+Pj4+Pj4gYXhpMA0KPiA+Pj4+Pj4gaHR0cDovL2x4ci5mcmVl
LQ0KPiA+Pj4gZWxlY3Ryb25zLmNvbS9zb3VyY2UvYXJjaC9hcm0vYm9vdC9kdHMvZHJhNy5kdHNp
I0wyMDcNCj4gPj4+Pj4+DQo+ID4+Pj4+PiBheGkxDQo+ID4+Pj4+PiBodHRwOi8vbHhyLmZyZWUt
DQo+ID4+PiBlbGVjdHJvbnMuY29tL3NvdXJjZS9hcmNoL2FybS9ib290L2R0cy9kcmE3LmR0c2kj
TDI0MQ0KPiA+Pj4+Pj4NCj4gPj4+Pj4+IEZvciB0aGlzIGNhc2UgbWFza2luZyB0aGUgdG9wIDRi
aXRzIChiaXRzMjggdG8gMzEpIHNob3VsZCBtYWtlDQo+IHRoZQ0KPiA+Pj4gam9iLg0KPiA+Pj4N
Cj4gPj4+IElNTywgd2Ugc2hvdWxkIGp1c3QgZml4IHRoaXMgY2FzZS4gQWZ0ZXIgZnVydGhlciBz
dHVkeSwgSSBkb24ndA0KPiB0aGluaw0KPiA+Pj4gdGhpcyBpcyBhIERXIGlzc3VlLCBidXQgcmF0
aGVyIGFuIFNPQyBpbnRlZ3JhdGlvbiBpc3N1ZS4NCj4gPj4+DQo+ID4+PiBJIGJlbGlldmUgeW91
IGNhbiBqdXN0IGZpeHVwIHRoZSBhZGRyZXNzIGluIHRoZSBwcC0+b3BzLT5ob3N0X2luaXQNCj4g
aG9vay4NCj4gPj4+DQo+ID4+DQo+ID4+IFllcyBJIGd1ZXNzIHRoYXQgSSBjb3VsZCBqdXN0IGFz
c2lnbiBwcC0+KCopX21vZF9iYXNlIHRvIHRoZSBDUFUNCj4gYWRkcmVzcw0KPiA+PiBpbiBEVyBh
bmQgbWFzayBpdCBvdXQgaW4gZHJhN3h4X3BjaWVfaG9zdF9pbml0KCkuLi4NCj4gPj4NCj4gPj4g
S2lzaG9uLCB3b3VsZCB5b3UgYmUgb2sgd2l0aCB0aGF0Pw0KPiA+DQo+ID4gSW5pdGlhbGx5IEkg
d2FzIHVzaW5nICpiYXNlLW1hc2sqIHByb3BlcnR5IGZyb20gZHQuIE1lIGFuZCBBcm5kDQo+IChj
YydlZCkgaGFkDQo+ID4gdGhpcyBkaXNjdXNzaW9uIFsxXSBiZWZvcmUgd2UgZGVjaWRlZCB0aGUg
Y3VycmVudCBhcHByb2FjaC4gSXQnbGwgYmUNCj4gZ29vZCB0bw0KPiA+IGNoZWNrIHdpdGggQXJu
ZCB0b28uDQo+ID4NCj4gPiBbMV0gLT4gIGh0dHA6Ly9saXN0cy5pbmZyYWRlYWQub3JnL3BpcGVy
bWFpbC9saW51eC1hcm0ta2VybmVsLzIwMTQtDQo+IE1heS8yNTM1MjguaHRtbA0KPiANCj4gVGhl
IHByb2JsZW0gSSBoYXZlIGhlcmUgaXMgdGhlIHVzZSBvZiByYW5nZXMgZG9lcyBub3QgbmVjZXNz
YXJpbHkgbWVhbg0KPiBmZXdlciBhZGRyZXNzIGJpdHMgYXJlIGF2YWlsYWJsZS4gSXQgY2FuIGJl
IHVzZWQganVzdCBmb3IgY29udmVuaWVuY2UNCj4gb2Ygbm90IHB1dHRpbmcgdGhlIGZ1bGwgYWRk
cmVzcyBpbnRvIGV2ZXJ5IG5vZGUncyByZWcgcHJvcGVydHkuIEFuZA0KPiB2aWNlIHZlcnNhLCB0
aGVyZSBhcmUgcHJvYmFibHkgcGxlbnR5IG9mIGNhc2VzIHdoZXJlIHdlIGhhdmUgdGhlIGZ1bGwN
Cj4gYWRkcmVzcyBpbiB0aGUgbm9kZXMsIGJ1dCByZWFsbHkgb25seSBzb21lIG9mIHRoZSBhZGRy
ZXNzIGJpdHMgYXJlDQo+IGRlY29kZWQgYXQgdGhlIElQIGJsb2NrLiBXaGV0aGVyIHRoZSBhZGRy
ZXNzIGJpdHMgYXJlIHByZXNlbnQgaXMNCj4gcmFyZWx5IGNhcmVkIGFib3V0IG9yIGtub3duIGJ5
IHMvdyBmb2xrcyB1bnRpbCB5b3UgaGl0IGEgcHJvYmxlbSBsaWtlDQo+IHRoaXMuIEdpdmVuIHRo
aXMgaXMgYW4gaXNvbGF0ZWQgY2FzZSBBVE0sIEkgd291bGQgZml4IGl0IGluIGFuDQo+IGlzb2xh
dGVkIHdheS4gSXQgZG9lcyBub3QgYWZmZWN0IHRoZSBiaW5kaW5nIGFuZCBjb3VsZCBiZSBjaGFu
Z2VkIGluDQo+IHRoZSBrZXJuZWwgbGF0ZXIgaWYgdGhpcyBiZWNvbWVzIGEgY29tbW9uIG5lZWQu
DQo+IA0KPiBSb2INCj4gLS0NCj4gVG8gdW5zdWJzY3JpYmUgZnJvbSB0aGlzIGxpc3Q6IHNlbmQg
dGhlIGxpbmUgInVuc3Vic2NyaWJlIGxpbnV4LXBjaSIgaW4NCj4gdGhlIGJvZHkgb2YgYSBtZXNz
YWdlIHRvIG1ham9yZG9tb0B2Z2VyLmtlcm5lbC5vcmcNCj4gTW9yZSBtYWpvcmRvbW8gaW5mbyBh
dCAgaHR0cDovL3ZnZXIua2VybmVsLm9yZy9tYWpvcmRvbW8taW5mby5odG1sDQo=

^ permalink raw reply	[flat|nested] 73+ messages in thread
* [PATCH v6] PCI: Store PCIe bus address in struct of_pci_range
@ 2015-08-03 11:18                                 ` Gabriele Paoloni
  0 siblings, 0 replies; 73+ messages in thread
From: Gabriele Paoloni @ 2015-08-03 11:18 UTC (permalink / raw)
  To: linux-arm-kernel

Rob, Kishon what about the following solution?....

---
 drivers/pci/host/pci-dra7xx.c      | 12 ++++++++++++
 drivers/pci/host/pcie-designware.c |  9 +++------
 2 files changed, 15 insertions(+), 6 deletions(-)

diff --git a/drivers/pci/host/pci-dra7xx.c b/drivers/pci/host/pci-dra7xx.c
index 80db09e..bb2635f 100644
--- a/drivers/pci/host/pci-dra7xx.c
+++ b/drivers/pci/host/pci-dra7xx.c
@@ -61,6 +61,7 @@
 
 #define	PCIECTRL_DRA7XX_CONF_PHY_CS			0x010C
 #define	LINK_UP						BIT(16)
+#define CPU_TO_BUS_ADDR             0x0FFFFFFF
 
 struct dra7xx_pcie {
 	void __iomem		*base;
@@ -138,6 +139,17 @@ static void dra7xx_pcie_enable_interrupts(struct pcie_port *pp)
 
 static void dra7xx_pcie_host_init(struct pcie_port *pp)
 {
+	if (pp->io_mod_base)
+		pp->io_mod_base &= CPU_TO_BUS_ADDR;
+
+	if (pp->mem_mod_base)
+		pp->mem_mod_base &= CPU_TO_BUS_ADDR;
+
+	if (pp->cfg0_mod_base) {
+		pp->cfg0_mod_base &= CPU_TO_BUS_ADDR;
+		pp->cfg1_mod_base &= CPU_TO_BUS_ADDR;
+	}
+
 	dw_pcie_setup_rc(pp);
 	dra7xx_pcie_establish_link(pp);
 	if (IS_ENABLED(CONFIG_PCI_MSI))
diff --git a/drivers/pci/host/pcie-designware.c b/drivers/pci/host/pcie-designware.c
index 69486be..06c682b 100644
--- a/drivers/pci/host/pcie-designware.c
+++ b/drivers/pci/host/pcie-designware.c
@@ -416,8 +416,7 @@ int dw_pcie_host_init(struct pcie_port *pp)
 			pp->io_base = range.cpu_addr;
 
 			/* Find the untranslated IO space address */
-			pp->io_mod_base = of_read_number(parser.range -
-							 parser.np + na, ns);
+			pp->io_mod_base = range.cpu_addr;;
 		}
 		if (restype == IORESOURCE_MEM) {
 			of_pci_range_to_resource(&range, np, &pp->mem);
@@ -426,8 +425,7 @@ int dw_pcie_host_init(struct pcie_port *pp)
 			pp->mem_bus_addr = range.pci_addr;
 
 			/* Find the untranslated MEM space address */
-			pp->mem_mod_base = of_read_number(parser.range -
-							  parser.np + na, ns);
+			pp->mem_mod_base = range.cpu_addr;
 		}
 		if (restype == 0) {
 			of_pci_range_to_resource(&range, np, &pp->cfg);
@@ -437,8 +435,7 @@ int dw_pcie_host_init(struct pcie_port *pp)
 			pp->cfg1_base = pp->cfg.start + pp->cfg0_size;
 
 			/* Find the untranslated configuration space address */
-			pp->cfg0_mod_base = of_read_number(parser.range -
-							   parser.np + na, ns);
+			pp->cfg0_mod_base = range.cpu_addr;
 			pp->cfg1_mod_base = pp->cfg0_mod_base +
 					    pp->cfg0_size;
 		}
-- 
1.9.1


> -----Original Message-----
> From: linux-pci-owner at vger.kernel.org [mailto:linux-pci-
> owner at vger.kernel.org] On Behalf Of Rob Herring
> Sent: Friday, July 31, 2015 5:53 PM
> To: Kishon Vijay Abraham I
> Cc: Gabriele Paoloni; Bjorn Helgaas; arnd at arndb.de;
> lorenzo.pieralisi at arm.com; Wangzhou (B); robh+dt at kernel.org;
> james.morse at arm.com; Liviu.Dudau at arm.com; linux-pci at vger.kernel.org;
> linux-arm-kernel at lists.infradead.org; devicetree at vger.kernel.org;
> Yuanzhichang; Zhudacai; zhangjukuo; qiuzhenfa; Liguozhu (Kenneth);
> Jingoo Han; Pratyush Anand; Arnd Bergmann
> Subject: Re: [PATCH v6] PCI: Store PCIe bus address in struct
> of_pci_range
> 
> On Fri, Jul 31, 2015 at 9:57 AM, Kishon Vijay Abraham I <kishon@ti.com>
> wrote:
> > +Arnd
> >
> > Hi,
> >
> > On Friday 31 July 2015 07:55 PM, Gabriele Paoloni wrote:
> >> [+cc Kishon]
> >>
> >>> -----Original Message-----
> >>> From: linux-pci-owner at vger.kernel.org [mailto:linux-pci-
> >>> owner at vger.kernel.org] On Behalf Of Rob Herring
> >>> Sent: Thursday, July 30, 2015 9:42 PM
> >>> To: Gabriele Paoloni
> >>> Cc: Bjorn Helgaas; arnd at arndb.de; lorenzo.pieralisi at arm.com;
> Wangzhou
> >>> (B); robh+dt at kernel.org; james.morse at arm.com; Liviu.Dudau at arm.com;
> >>> linux-pci at vger.kernel.org; linux-arm-kernel at lists.infradead.org;
> >>> devicetree at vger.kernel.org; Yuanzhichang; Zhudacai; zhangjukuo;
> >>> qiuzhenfa; Liguozhu (Kenneth); Jingoo Han; Pratyush Anand
> >>> Subject: Re: [PATCH v6] PCI: Store PCIe bus address in struct
> >>> of_pci_range
> >>>
> >>> On Thu, Jul 30, 2015 at 12:34 PM, Gabriele Paoloni
> >>> <gabriele.paoloni@huawei.com> wrote:
> >>>>> -----Original Message-----
> >>>>> From: Bjorn Helgaas [mailto:bhelgaas at google.com]
> >>>>> Sent: 30 July 2015 18:15
> >>>>> On Thu, Jul 30, 2015 at 04:50:55PM +0000, Gabriele Paoloni wrote:
> >>>>>>> -----Original Message-----
> >>>>>>> From: linux-pci-owner at vger.kernel.org [mailto:linux-pci-
> >>>>>>> owner at vger.kernel.org] On Behalf Of Bjorn Helgaas
> >>>>>>> Sent: Thursday, July 30, 2015 5:15 PM
> >>>>>>> On Thu, Jul 30, 2015 at 01:52:13PM +0000, Gabriele Paoloni
> wrote:
> >>>
> >>> [...]
> >>>
> >>>>>>>> I don?t think we should rely on [CPU] addresses...what if the
> >>>>>>> intermediate
> >>>>>>>> translation layer changes the lower significant bits of the
> >>> "bus
> >>>>>>> address"
> >>>>>>>> to translate into a cpu address?
> >>>>>>>
> >>>>>>> Is it really a possiblity that the lower bits could be changed?
> >>>>>>
> >>>>>> I've checked all the current deignware users DTs except "pci-
> >>>>> layerscape"
> >>>>>> that I could not find:
> >>>>>> spear1310.dtsi
> >>>>>> spear1340.dtsi
> >>>>>> dra7.dtsi
> >>>>>> imx6qdl.dtsi
> >>>>>> imx6sx.dtsi
> >>>>>> keystone.dtsi
> >>>>>> exynos5440.dtsi
> >>>>>>
> >>>>>> None of them modifies the lower bits. To be more precise the
> only
> >>> guy
> >>>>>> that provides another translation layer is "dra7.dtsi":
> >>>>>> axi0
> >>>>>> http://lxr.free-
> >>> electrons.com/source/arch/arm/boot/dts/dra7.dtsi#L207
> >>>>>>
> >>>>>> axi1
> >>>>>> http://lxr.free-
> >>> electrons.com/source/arch/arm/boot/dts/dra7.dtsi#L241
> >>>>>>
> >>>>>> For this case masking the top 4bits (bits28 to 31) should make
> the
> >>> job.
> >>>
> >>> IMO, we should just fix this case. After further study, I don't
> think
> >>> this is a DW issue, but rather an SOC integration issue.
> >>>
> >>> I believe you can just fixup the address in the pp->ops->host_init
> hook.
> >>>
> >>
> >> Yes I guess that I could just assign pp->(*)_mod_base to the CPU
> address
> >> in DW and mask it out in dra7xx_pcie_host_init()...
> >>
> >> Kishon, would you be ok with that?
> >
> > Initially I was using *base-mask* property from dt. Me and Arnd
> (cc'ed) had
> > this discussion [1] before we decided the current approach. It'll be
> good to
> > check with Arnd too.
> >
> > [1] ->  http://lists.infradead.org/pipermail/linux-arm-kernel/2014-
> May/253528.html
> 
> The problem I have here is the use of ranges does not necessarily mean
> fewer address bits are available. It can be used just for convenience
> of not putting the full address into every node's reg property. And
> vice versa, there are probably plenty of cases where we have the full
> address in the nodes, but really only some of the address bits are
> decoded at the IP block. Whether the address bits are present is
> rarely cared about or known by s/w folks until you hit a problem like
> this. Given this is an isolated case ATM, I would fix it in an
> isolated way. It does not affect the binding and could be changed in
> the kernel later if this becomes a common need.
> 
> Rob
> --
> To unsubscribe from this list: send the line "unsubscribe linux-pci" in
> the body of a message to majordomo at vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply related	[flat|nested] 73+ messages in thread
* Re: [PATCH v6] PCI: Store PCIe bus address in struct of_pci_range
  2015-07-31 15:09                               ` Gabriele Paoloni
  (?)
@ 2015-08-03 14:41                                 ` Jingoo Han
  -1 siblings, 0 replies; 73+ messages in thread
From: Jingoo Han @ 2015-08-03 14:41 UTC (permalink / raw)
  To: Gabriele Paoloni
  Cc: Kishon Vijay Abraham I, Rob Herring, Bjorn Helgaas, arnd,
	lorenzo.pieralisi, Wangzhou (B),
	robh+dt, james.morse, Liviu.Dudau, linux-pci, linux-arm-kernel,
	devicetree, Yuanzhichang, Zhudacai, zhangjukuo, qiuzhenfa,
	Liguozhu (Kenneth),
	Pratyush Anand, Arnd Bergmann, jingoohan1@gmail.com

On 2015. 8. 1., at AM 12:09, Gabriele Paoloni <gabriele.paoloni@huawei.com> wrote:
> 
>> -----Original Message-----
>> From: Kishon Vijay Abraham I [mailto:kishon@ti.com]
>> Sent: Friday, July 31, 2015 3:57 PM
>> To: Gabriele Paoloni; Rob Herring
>> Cc: Bjorn Helgaas; arnd@arndb.de; lorenzo.pieralisi@arm.com; Wangzhou
>> (B); robh+dt@kernel.org; james.morse@arm.com; Liviu.Dudau@arm.com;
>> linux-pci@vger.kernel.org; linux-arm-kernel@lists.infradead.org;
>> devicetree@vger.kernel.org; Yuanzhichang; Zhudacai; zhangjukuo;
>> qiuzhenfa; Liguozhu (Kenneth); Jingoo Han; Pratyush Anand; Arnd
>> Bergmann; Arnd Bergmann
>> Subject: Re: [PATCH v6] PCI: Store PCIe bus address in struct
>> of_pci_range
>> 
>> +Arnd
>> 
>> Hi,
>> 
>>> On Friday 31 July 2015 07:55 PM, Gabriele Paoloni wrote:
>>> [+cc Kishon]
>>> 
>>>> -----Original Message-----
>>>> From: linux-pci-owner@vger.kernel.org [mailto:linux-pci-
>>>> owner@vger.kernel.org] On Behalf Of Rob Herring
>>>> Sent: Thursday, July 30, 2015 9:42 PM
>>>> To: Gabriele Paoloni
>>>> Cc: Bjorn Helgaas; arnd@arndb.de; lorenzo.pieralisi@arm.com;
>> Wangzhou
>>>> (B); robh+dt@kernel.org; james.morse@arm.com; Liviu.Dudau@arm.com;
>>>> linux-pci@vger.kernel.org; linux-arm-kernel@lists.infradead.org;
>>>> devicetree@vger.kernel.org; Yuanzhichang; Zhudacai; zhangjukuo;
>>>> qiuzhenfa; Liguozhu (Kenneth); Jingoo Han; Pratyush Anand
>>>> Subject: Re: [PATCH v6] PCI: Store PCIe bus address in struct
>>>> of_pci_range
>>>> 
>>>> On Thu, Jul 30, 2015 at 12:34 PM, Gabriele Paoloni
>>>> <gabriele.paoloni@huawei.com> wrote:
>>>>>> -----Original Message-----
>>>>>> From: Bjorn Helgaas [mailto:bhelgaas@google.com]
>>>>>> Sent: 30 July 2015 18:15
>>>>>> On Thu, Jul 30, 2015 at 04:50:55PM +0000, Gabriele Paoloni wrote:
>>>>>>>> -----Original Message-----
>>>>>>>> From: linux-pci-owner@vger.kernel.org [mailto:linux-pci-
>>>>>>>> owner@vger.kernel.org] On Behalf Of Bjorn Helgaas
>>>>>>>> Sent: Thursday, July 30, 2015 5:15 PM
>>>>>>>> On Thu, Jul 30, 2015 at 01:52:13PM +0000, Gabriele Paoloni wrote:
>>>> 
>>>> [...]
>>>> 
>>>>>>>>> I don’t think we should rely on [CPU] addresses...what if the
>>>>>>>> intermediate
>>>>>>>>> translation layer changes the lower significant bits of the
>>>> "bus
>>>>>>>> address"
>>>>>>>>> to translate into a cpu address?
>>>>>>>> 
>>>>>>>> Is it really a possiblity that the lower bits could be changed?
>>>>>>> 
>>>>>>> I've checked all the current deignware users DTs except "pci-
>>>>>> layerscape"
>>>>>>> that I could not find:
>>>>>>> spear1310.dtsi
>>>>>>> spear1340.dtsi
>>>>>>> dra7.dtsi
>>>>>>> imx6qdl.dtsi
>>>>>>> imx6sx.dtsi
>>>>>>> keystone.dtsi
>>>>>>> exynos5440.dtsi
>>>>>>> 
>>>>>>> None of them modifies the lower bits. To be more precise the only
>>>> guy
>>>>>>> that provides another translation layer is "dra7.dtsi":
>>>>>>> axi0
>>>>>>> http://lxr.free-
>>>> electrons.com/source/arch/arm/boot/dts/dra7.dtsi#L207
>>>>>>> 
>>>>>>> axi1
>>>>>>> http://lxr.free-
>>>> electrons.com/source/arch/arm/boot/dts/dra7.dtsi#L241
>>>>>>> 
>>>>>>> For this case masking the top 4bits (bits28 to 31) should make
>> the
>>>> job.
>>>> 
>>>> IMO, we should just fix this case. After further study, I don't
>> think
>>>> this is a DW issue, but rather an SOC integration issue.
>>>> 
>>>> I believe you can just fixup the address in the pp->ops->host_init
>> hook.

Yep, it is SOC specific code for dra7.
This is NOT a DW issue.

>>> 
>>> Yes I guess that I could just assign pp->(*)_mod_base to the CPU
>> address
>>> in DW and mask it out in dra7xx_pcie_host_init()...
>>> 
>>> Kishon, would you be ok with that?
>> 
>> Initially I was using *base-mask* property from dt. Me and Arnd (cc'ed)
>> had
>> this discussion [1] before we decided the current approach. It'll be
>> good to
>> check with Arnd too.
>> 
>> [1] ->  http://lists.infradead.org/pipermail/linux-arm-kernel/2014-
>> May/253528.html
> 
> 
> In this patch you use the mask into designware....instead the approach 
> proposed by Rob means to have the mask declared in the dra7 driver and
> you modified the pp members in dra7xx_pcie_host_init by masking them...

I want to move that code to dra7 driver,
because that code is dra7-specific.

Best regards,
Jingoo Han

> BTW good to have Arnd opinion too..
>> 
>> Thanks
>> Kishon

^ permalink raw reply	[flat|nested] 73+ messages in thread
* Re: [PATCH v6] PCI: Store PCIe bus address in struct of_pci_range
@ 2015-08-03 14:41                                 ` Jingoo Han
  0 siblings, 0 replies; 73+ messages in thread
From: Jingoo Han @ 2015-08-03 14:41 UTC (permalink / raw)
  To: Gabriele Paoloni
  Cc: Kishon Vijay Abraham I, Rob Herring, Bjorn Helgaas, arnd,
	lorenzo.pieralisi, Wangzhou (B),
	robh+dt, james.morse, Liviu.Dudau, linux-pci, linux-arm-kernel,
	devicetree, Yuanzhichang, Zhudacai, zhangjukuo, qiuzhenfa,
	Liguozhu (Kenneth),
	Pratyush Anand, Arnd Bergmann, jingoohan1

On 2015. 8. 1., at AM 12:09, Gabriele Paoloni <gabriele.paoloni@huawei.com> wrote:
> 
>> -----Original Message-----
>> From: Kishon Vijay Abraham I [mailto:kishon@ti.com]
>> Sent: Friday, July 31, 2015 3:57 PM
>> To: Gabriele Paoloni; Rob Herring
>> Cc: Bjorn Helgaas; arnd@arndb.de; lorenzo.pieralisi@arm.com; Wangzhou
>> (B); robh+dt@kernel.org; james.morse@arm.com; Liviu.Dudau@arm.com;
>> linux-pci@vger.kernel.org; linux-arm-kernel@lists.infradead.org;
>> devicetree@vger.kernel.org; Yuanzhichang; Zhudacai; zhangjukuo;
>> qiuzhenfa; Liguozhu (Kenneth); Jingoo Han; Pratyush Anand; Arnd
>> Bergmann; Arnd Bergmann
>> Subject: Re: [PATCH v6] PCI: Store PCIe bus address in struct
>> of_pci_range
>> 
>> +Arnd
>> 
>> Hi,
>> 
>>> On Friday 31 July 2015 07:55 PM, Gabriele Paoloni wrote:
>>> [+cc Kishon]
>>> 
>>>> -----Original Message-----
>>>> From: linux-pci-owner@vger.kernel.org [mailto:linux-pci-
>>>> owner@vger.kernel.org] On Behalf Of Rob Herring
>>>> Sent: Thursday, July 30, 2015 9:42 PM
>>>> To: Gabriele Paoloni
>>>> Cc: Bjorn Helgaas; arnd@arndb.de; lorenzo.pieralisi@arm.com;
>> Wangzhou
>>>> (B); robh+dt@kernel.org; james.morse@arm.com; Liviu.Dudau@arm.com;
>>>> linux-pci@vger.kernel.org; linux-arm-kernel@lists.infradead.org;
>>>> devicetree@vger.kernel.org; Yuanzhichang; Zhudacai; zhangjukuo;
>>>> qiuzhenfa; Liguozhu (Kenneth); Jingoo Han; Pratyush Anand
>>>> Subject: Re: [PATCH v6] PCI: Store PCIe bus address in struct
>>>> of_pci_range
>>>> 
>>>> On Thu, Jul 30, 2015 at 12:34 PM, Gabriele Paoloni
>>>> <gabriele.paoloni@huawei.com> wrote:
>>>>>> -----Original Message-----
>>>>>> From: Bjorn Helgaas [mailto:bhelgaas@google.com]
>>>>>> Sent: 30 July 2015 18:15
>>>>>> On Thu, Jul 30, 2015 at 04:50:55PM +0000, Gabriele Paoloni wrote:
>>>>>>>> -----Original Message-----
>>>>>>>> From: linux-pci-owner@vger.kernel.org [mailto:linux-pci-
>>>>>>>> owner@vger.kernel.org] On Behalf Of Bjorn Helgaas
>>>>>>>> Sent: Thursday, July 30, 2015 5:15 PM
>>>>>>>> On Thu, Jul 30, 2015 at 01:52:13PM +0000, Gabriele Paoloni wrote:
>>>> 
>>>> [...]
>>>> 
>>>>>>>>> I don’t think we should rely on [CPU] addresses...what if the
>>>>>>>> intermediate
>>>>>>>>> translation layer changes the lower significant bits of the
>>>> "bus
>>>>>>>> address"
>>>>>>>>> to translate into a cpu address?
>>>>>>>> 
>>>>>>>> Is it really a possiblity that the lower bits could be changed?
>>>>>>> 
>>>>>>> I've checked all the current deignware users DTs except "pci-
>>>>>> layerscape"
>>>>>>> that I could not find:
>>>>>>> spear1310.dtsi
>>>>>>> spear1340.dtsi
>>>>>>> dra7.dtsi
>>>>>>> imx6qdl.dtsi
>>>>>>> imx6sx.dtsi
>>>>>>> keystone.dtsi
>>>>>>> exynos5440.dtsi
>>>>>>> 
>>>>>>> None of them modifies the lower bits. To be more precise the only
>>>> guy
>>>>>>> that provides another translation layer is "dra7.dtsi":
>>>>>>> axi0
>>>>>>> http://lxr.free-
>>>> electrons.com/source/arch/arm/boot/dts/dra7.dtsi#L207
>>>>>>> 
>>>>>>> axi1
>>>>>>> http://lxr.free-
>>>> electrons.com/source/arch/arm/boot/dts/dra7.dtsi#L241
>>>>>>> 
>>>>>>> For this case masking the top 4bits (bits28 to 31) should make
>> the
>>>> job.
>>>> 
>>>> IMO, we should just fix this case. After further study, I don't
>> think
>>>> this is a DW issue, but rather an SOC integration issue.
>>>> 
>>>> I believe you can just fixup the address in the pp->ops->host_init
>> hook.

Yep, it is SOC specific code for dra7.
This is NOT a DW issue.

>>> 
>>> Yes I guess that I could just assign pp->(*)_mod_base to the CPU
>> address
>>> in DW and mask it out in dra7xx_pcie_host_init()...
>>> 
>>> Kishon, would you be ok with that?
>> 
>> Initially I was using *base-mask* property from dt. Me and Arnd (cc'ed)
>> had
>> this discussion [1] before we decided the current approach. It'll be
>> good to
>> check with Arnd too.
>> 
>> [1] ->  http://lists.infradead.org/pipermail/linux-arm-kernel/2014-
>> May/253528.html
> 
> 
> In this patch you use the mask into designware....instead the approach 
> proposed by Rob means to have the mask declared in the dra7 driver and
> you modified the pp members in dra7xx_pcie_host_init by masking them...

I want to move that code to dra7 driver,
because that code is dra7-specific.

Best regards,
Jingoo Han

> BTW good to have Arnd opinion too..
>> 
>> Thanks
>> Kishon

^ permalink raw reply	[flat|nested] 73+ messages in thread
* [PATCH v6] PCI: Store PCIe bus address in struct of_pci_range
@ 2015-08-03 14:41                                 ` Jingoo Han
  0 siblings, 0 replies; 73+ messages in thread
From: Jingoo Han @ 2015-08-03 14:41 UTC (permalink / raw)
  To: linux-arm-kernel

On 2015. 8. 1., at AM 12:09, Gabriele Paoloni <gabriele.paoloni@huawei.com> wrote:
> 
>> -----Original Message-----
>> From: Kishon Vijay Abraham I [mailto:kishon at ti.com]
>> Sent: Friday, July 31, 2015 3:57 PM
>> To: Gabriele Paoloni; Rob Herring
>> Cc: Bjorn Helgaas; arnd at arndb.de; lorenzo.pieralisi at arm.com; Wangzhou
>> (B); robh+dt at kernel.org; james.morse at arm.com; Liviu.Dudau at arm.com;
>> linux-pci at vger.kernel.org; linux-arm-kernel at lists.infradead.org;
>> devicetree at vger.kernel.org; Yuanzhichang; Zhudacai; zhangjukuo;
>> qiuzhenfa; Liguozhu (Kenneth); Jingoo Han; Pratyush Anand; Arnd
>> Bergmann; Arnd Bergmann
>> Subject: Re: [PATCH v6] PCI: Store PCIe bus address in struct
>> of_pci_range
>> 
>> +Arnd
>> 
>> Hi,
>> 
>>> On Friday 31 July 2015 07:55 PM, Gabriele Paoloni wrote:
>>> [+cc Kishon]
>>> 
>>>> -----Original Message-----
>>>> From: linux-pci-owner at vger.kernel.org [mailto:linux-pci-
>>>> owner at vger.kernel.org] On Behalf Of Rob Herring
>>>> Sent: Thursday, July 30, 2015 9:42 PM
>>>> To: Gabriele Paoloni
>>>> Cc: Bjorn Helgaas; arnd at arndb.de; lorenzo.pieralisi at arm.com;
>> Wangzhou
>>>> (B); robh+dt at kernel.org; james.morse at arm.com; Liviu.Dudau at arm.com;
>>>> linux-pci at vger.kernel.org; linux-arm-kernel at lists.infradead.org;
>>>> devicetree at vger.kernel.org; Yuanzhichang; Zhudacai; zhangjukuo;
>>>> qiuzhenfa; Liguozhu (Kenneth); Jingoo Han; Pratyush Anand
>>>> Subject: Re: [PATCH v6] PCI: Store PCIe bus address in struct
>>>> of_pci_range
>>>> 
>>>> On Thu, Jul 30, 2015 at 12:34 PM, Gabriele Paoloni
>>>> <gabriele.paoloni@huawei.com> wrote:
>>>>>> -----Original Message-----
>>>>>> From: Bjorn Helgaas [mailto:bhelgaas at google.com]
>>>>>> Sent: 30 July 2015 18:15
>>>>>> On Thu, Jul 30, 2015 at 04:50:55PM +0000, Gabriele Paoloni wrote:
>>>>>>>> -----Original Message-----
>>>>>>>> From: linux-pci-owner at vger.kernel.org [mailto:linux-pci-
>>>>>>>> owner at vger.kernel.org] On Behalf Of Bjorn Helgaas
>>>>>>>> Sent: Thursday, July 30, 2015 5:15 PM
>>>>>>>> On Thu, Jul 30, 2015 at 01:52:13PM +0000, Gabriele Paoloni wrote:
>>>> 
>>>> [...]
>>>> 
>>>>>>>>> I don?t think we should rely on [CPU] addresses...what if the
>>>>>>>> intermediate
>>>>>>>>> translation layer changes the lower significant bits of the
>>>> "bus
>>>>>>>> address"
>>>>>>>>> to translate into a cpu address?
>>>>>>>> 
>>>>>>>> Is it really a possiblity that the lower bits could be changed?
>>>>>>> 
>>>>>>> I've checked all the current deignware users DTs except "pci-
>>>>>> layerscape"
>>>>>>> that I could not find:
>>>>>>> spear1310.dtsi
>>>>>>> spear1340.dtsi
>>>>>>> dra7.dtsi
>>>>>>> imx6qdl.dtsi
>>>>>>> imx6sx.dtsi
>>>>>>> keystone.dtsi
>>>>>>> exynos5440.dtsi
>>>>>>> 
>>>>>>> None of them modifies the lower bits. To be more precise the only
>>>> guy
>>>>>>> that provides another translation layer is "dra7.dtsi":
>>>>>>> axi0
>>>>>>> http://lxr.free-
>>>> electrons.com/source/arch/arm/boot/dts/dra7.dtsi#L207
>>>>>>> 
>>>>>>> axi1
>>>>>>> http://lxr.free-
>>>> electrons.com/source/arch/arm/boot/dts/dra7.dtsi#L241
>>>>>>> 
>>>>>>> For this case masking the top 4bits (bits28 to 31) should make
>> the
>>>> job.
>>>> 
>>>> IMO, we should just fix this case. After further study, I don't
>> think
>>>> this is a DW issue, but rather an SOC integration issue.
>>>> 
>>>> I believe you can just fixup the address in the pp->ops->host_init
>> hook.

Yep, it is SOC specific code for dra7.
This is NOT a DW issue.

>>> 
>>> Yes I guess that I could just assign pp->(*)_mod_base to the CPU
>> address
>>> in DW and mask it out in dra7xx_pcie_host_init()...
>>> 
>>> Kishon, would you be ok with that?
>> 
>> Initially I was using *base-mask* property from dt. Me and Arnd (cc'ed)
>> had
>> this discussion [1] before we decided the current approach. It'll be
>> good to
>> check with Arnd too.
>> 
>> [1] ->  http://lists.infradead.org/pipermail/linux-arm-kernel/2014-
>> May/253528.html
> 
> 
> In this patch you use the mask into designware....instead the approach 
> proposed by Rob means to have the mask declared in the dra7 driver and
> you modified the pp members in dra7xx_pcie_host_init by masking them...

I want to move that code to dra7 driver,
because that code is dra7-specific.

Best regards,
Jingoo Han

> BTW good to have Arnd opinion too..
>> 
>> Thanks
>> Kishon

^ permalink raw reply	[flat|nested] 73+ messages in thread
* Re: [PATCH v6] PCI: Store PCIe bus address in struct of_pci_range
  2015-08-03 11:18                                 ` Gabriele Paoloni
@ 2015-08-04  4:19                                   ` Jingoo Han
  -1 siblings, 0 replies; 73+ messages in thread
From: Jingoo Han @ 2015-08-04  4:19 UTC (permalink / raw)
  To: Gabriele Paoloni
  Cc: Rob Herring, Kishon Vijay Abraham I, Bjorn Helgaas, arnd,
	lorenzo.pieralisi, Wangzhou (B),
	robh+dt, james.morse, Liviu.Dudau, linux-pci, linux-arm-kernel,
	devicetree, Yuanzhichang, Zhudacai, zhangjukuo, qiuzhenfa,
	Liguozhu (Kenneth),
	Pratyush Anand, Arnd Bergmann, jingoohan1

On 2015. 8. 3., at PM 8:18, Gabriele Paoloni <gabriele.paoloni@huawei.com> wrote:
> 
> Rob, Kishon what about the following solution?....
> 
> ---
> drivers/pci/host/pci-dra7xx.c      | 12 ++++++++++++
> drivers/pci/host/pcie-designware.c |  9 +++------

Hi Paoloni,

OK, it looks good.
I want you to revert the following patch. 

commit "f4c55c5a3f7f (PCI: designware: Program ATU with untranslated address)"

Would you remove all '*_mode_base's?

Best regards,
Jingoo Han

> 2 files changed, 15 insertions(+), 6 deletions(-)
> 
> diff --git a/drivers/pci/host/pci-dra7xx.c b/drivers/pci/host/pci-dra7xx.c
> index 80db09e..bb2635f 100644
> --- a/drivers/pci/host/pci-dra7xx.c
> +++ b/drivers/pci/host/pci-dra7xx.c
> @@ -61,6 +61,7 @@
> 
> #define    PCIECTRL_DRA7XX_CONF_PHY_CS            0x010C
> #define    LINK_UP                        BIT(16)
> +#define CPU_TO_BUS_ADDR             0x0FFFFFFF
> 
> struct dra7xx_pcie {
>    void __iomem        *base;
> @@ -138,6 +139,17 @@ static void dra7xx_pcie_enable_interrupts(struct pcie_port *pp)
> 
> static void dra7xx_pcie_host_init(struct pcie_port *pp)
> {
> +    if (pp->io_mod_base)
> +        pp->io_mod_base &= CPU_TO_BUS_ADDR;
> +
> +    if (pp->mem_mod_base)
> +        pp->mem_mod_base &= CPU_TO_BUS_ADDR;
> +
> +    if (pp->cfg0_mod_base) {
> +        pp->cfg0_mod_base &= CPU_TO_BUS_ADDR;
> +        pp->cfg1_mod_base &= CPU_TO_BUS_ADDR;
> +    }
> +
>    dw_pcie_setup_rc(pp);
>    dra7xx_pcie_establish_link(pp);
>    if (IS_ENABLED(CONFIG_PCI_MSI))
> diff --git a/drivers/pci/host/pcie-designware.c b/drivers/pci/host/pcie-designware.c
> index 69486be..06c682b 100644
> --- a/drivers/pci/host/pcie-designware.c
> +++ b/drivers/pci/host/pcie-designware.c
> @@ -416,8 +416,7 @@ int dw_pcie_host_init(struct pcie_port *pp)
>            pp->io_base = range.cpu_addr;
> 
>            /* Find the untranslated IO space address */
> -            pp->io_mod_base = of_read_number(parser.range -
> -                             parser.np + na, ns);
> +            pp->io_mod_base = range.cpu_addr;;
>        }
>        if (restype == IORESOURCE_MEM) {
>            of_pci_range_to_resource(&range, np, &pp->mem);
> @@ -426,8 +425,7 @@ int dw_pcie_host_init(struct pcie_port *pp)
>            pp->mem_bus_addr = range.pci_addr;
> 
>            /* Find the untranslated MEM space address */
> -            pp->mem_mod_base = of_read_number(parser.range -
> -                              parser.np + na, ns);
> +            pp->mem_mod_base = range.cpu_addr;
>        }
>        if (restype == 0) {
>            of_pci_range_to_resource(&range, np, &pp->cfg);
> @@ -437,8 +435,7 @@ int dw_pcie_host_init(struct pcie_port *pp)
>            pp->cfg1_base = pp->cfg.start + pp->cfg0_size;
> 
>            /* Find the untranslated configuration space address */
> -            pp->cfg0_mod_base = of_read_number(parser.range -
> -                               parser.np + na, ns);
> +            pp->cfg0_mod_base = range.cpu_addr;
>            pp->cfg1_mod_base = pp->cfg0_mod_base +
>                        pp->cfg0_size;
>        }
> -- 
> 1.9.1
> 
> 
>> -----Original Message-----
>> From: linux-pci-owner@vger.kernel.org [mailto:linux-pci-
>> owner@vger.kernel.org] On Behalf Of Rob Herring
>> Sent: Friday, July 31, 2015 5:53 PM
>> To: Kishon Vijay Abraham I
>> Cc: Gabriele Paoloni; Bjorn Helgaas; arnd@arndb.de;
>> lorenzo.pieralisi@arm.com; Wangzhou (B); robh+dt@kernel.org;
>> james.morse@arm.com; Liviu.Dudau@arm.com; linux-pci@vger.kernel.org;
>> linux-arm-kernel@lists.infradead.org; devicetree@vger.kernel.org;
>> Yuanzhichang; Zhudacai; zhangjukuo; qiuzhenfa; Liguozhu (Kenneth);
>> Jingoo Han; Pratyush Anand; Arnd Bergmann
>> Subject: Re: [PATCH v6] PCI: Store PCIe bus address in struct
>> of_pci_range
>> 
>> On Fri, Jul 31, 2015 at 9:57 AM, Kishon Vijay Abraham I <kishon@ti.com>
>> wrote:
>>> +Arnd
>>> 
>>> Hi,
>>> 
>>>> On Friday 31 July 2015 07:55 PM, Gabriele Paoloni wrote:
>>>> [+cc Kishon]
>>>> 
>>>>> -----Original Message-----
>>>>> From: linux-pci-owner@vger.kernel.org [mailto:linux-pci-
>>>>> owner@vger.kernel.org] On Behalf Of Rob Herring
>>>>> Sent: Thursday, July 30, 2015 9:42 PM
>>>>> To: Gabriele Paoloni
>>>>> Cc: Bjorn Helgaas; arnd@arndb.de; lorenzo.pieralisi@arm.com;
>> Wangzhou
>>>>> (B); robh+dt@kernel.org; james.morse@arm.com; Liviu.Dudau@arm.com;
>>>>> linux-pci@vger.kernel.org; linux-arm-kernel@lists.infradead.org;
>>>>> devicetree@vger.kernel.org; Yuanzhichang; Zhudacai; zhangjukuo;
>>>>> qiuzhenfa; Liguozhu (Kenneth); Jingoo Han; Pratyush Anand
>>>>> Subject: Re: [PATCH v6] PCI: Store PCIe bus address in struct
>>>>> of_pci_range
>>>>> 
>>>>> On Thu, Jul 30, 2015 at 12:34 PM, Gabriele Paoloni
>>>>> <gabriele.paoloni@huawei.com> wrote:
>>>>>>> -----Original Message-----
>>>>>>> From: Bjorn Helgaas [mailto:bhelgaas@google.com]
>>>>>>> Sent: 30 July 2015 18:15
>>>>>>> On Thu, Jul 30, 2015 at 04:50:55PM +0000, Gabriele Paoloni wrote:
>>>>>>>>> -----Original Message-----
>>>>>>>>> From: linux-pci-owner@vger.kernel.org [mailto:linux-pci-
>>>>>>>>> owner@vger.kernel.org] On Behalf Of Bjorn Helgaas
>>>>>>>>> Sent: Thursday, July 30, 2015 5:15 PM
>>>>>>>>> On Thu, Jul 30, 2015 at 01:52:13PM +0000, Gabriele Paoloni
>> wrote:
>>>>> 
>>>>> [...]
>>>>> 
>>>>>>>>>> I don’t think we should rely on [CPU] addresses...what if the
>>>>>>>>> intermediate
>>>>>>>>>> translation layer changes the lower significant bits of the
>>>>> "bus
>>>>>>>>> address"
>>>>>>>>>> to translate into a cpu address?
>>>>>>>>> 
>>>>>>>>> Is it really a possiblity that the lower bits could be changed?
>>>>>>>> 
>>>>>>>> I've checked all the current deignware users DTs except "pci-
>>>>>>> layerscape"
>>>>>>>> that I could not find:
>>>>>>>> spear1310.dtsi
>>>>>>>> spear1340.dtsi
>>>>>>>> dra7.dtsi
>>>>>>>> imx6qdl.dtsi
>>>>>>>> imx6sx.dtsi
>>>>>>>> keystone.dtsi
>>>>>>>> exynos5440.dtsi
>>>>>>>> 
>>>>>>>> None of them modifies the lower bits. To be more precise the
>> only
>>>>> guy
>>>>>>>> that provides another translation layer is "dra7.dtsi":
>>>>>>>> axi0
>>>>>>>> http://lxr.free-
>>>>> electrons.com/source/arch/arm/boot/dts/dra7.dtsi#L207
>>>>>>>> 
>>>>>>>> axi1
>>>>>>>> http://lxr.free-
>>>>> electrons.com/source/arch/arm/boot/dts/dra7.dtsi#L241
>>>>>>>> 
>>>>>>>> For this case masking the top 4bits (bits28 to 31) should make
>> the
>>>>> job.
>>>>> 
>>>>> IMO, we should just fix this case. After further study, I don't
>> think
>>>>> this is a DW issue, but rather an SOC integration issue.
>>>>> 
>>>>> I believe you can just fixup the address in the pp->ops->host_init
>> hook.
>>>> 
>>>> Yes I guess that I could just assign pp->(*)_mod_base to the CPU
>> address
>>>> in DW and mask it out in dra7xx_pcie_host_init()...
>>>> 
>>>> Kishon, would you be ok with that?
>>> 
>>> Initially I was using *base-mask* property from dt. Me and Arnd
>> (cc'ed) had
>>> this discussion [1] before we decided the current approach. It'll be
>> good to
>>> check with Arnd too.
>>> 
>>> [1] ->  http://lists.infradead.org/pipermail/linux-arm-kernel/2014-
>> May/253528.html
>> 
>> The problem I have here is the use of ranges does not necessarily mean
>> fewer address bits are available. It can be used just for convenience
>> of not putting the full address into every node's reg property. And
>> vice versa, there are probably plenty of cases where we have the full
>> address in the nodes, but really only some of the address bits are
>> decoded at the IP block. Whether the address bits are present is
>> rarely cared about or known by s/w folks until you hit a problem like
>> this. Given this is an isolated case ATM, I would fix it in an
>> isolated way. It does not affect the binding and could be changed in
>> the kernel later if this becomes a common need.
>> 
>> Rob
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-pci" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 73+ messages in thread
* [PATCH v6] PCI: Store PCIe bus address in struct of_pci_range
@ 2015-08-04  4:19                                   ` Jingoo Han
  0 siblings, 0 replies; 73+ messages in thread
From: Jingoo Han @ 2015-08-04  4:19 UTC (permalink / raw)
  To: linux-arm-kernel

On 2015. 8. 3., at PM 8:18, Gabriele Paoloni <gabriele.paoloni@huawei.com> wrote:
> 
> Rob, Kishon what about the following solution?....
> 
> ---
> drivers/pci/host/pci-dra7xx.c      | 12 ++++++++++++
> drivers/pci/host/pcie-designware.c |  9 +++------

Hi Paoloni,

OK, it looks good.
I want you to revert the following patch. 

commit "f4c55c5a3f7f (PCI: designware: Program ATU with untranslated address)"

Would you remove all '*_mode_base's?

Best regards,
Jingoo Han

> 2 files changed, 15 insertions(+), 6 deletions(-)
> 
> diff --git a/drivers/pci/host/pci-dra7xx.c b/drivers/pci/host/pci-dra7xx.c
> index 80db09e..bb2635f 100644
> --- a/drivers/pci/host/pci-dra7xx.c
> +++ b/drivers/pci/host/pci-dra7xx.c
> @@ -61,6 +61,7 @@
> 
> #define    PCIECTRL_DRA7XX_CONF_PHY_CS            0x010C
> #define    LINK_UP                        BIT(16)
> +#define CPU_TO_BUS_ADDR             0x0FFFFFFF
> 
> struct dra7xx_pcie {
>    void __iomem        *base;
> @@ -138,6 +139,17 @@ static void dra7xx_pcie_enable_interrupts(struct pcie_port *pp)
> 
> static void dra7xx_pcie_host_init(struct pcie_port *pp)
> {
> +    if (pp->io_mod_base)
> +        pp->io_mod_base &= CPU_TO_BUS_ADDR;
> +
> +    if (pp->mem_mod_base)
> +        pp->mem_mod_base &= CPU_TO_BUS_ADDR;
> +
> +    if (pp->cfg0_mod_base) {
> +        pp->cfg0_mod_base &= CPU_TO_BUS_ADDR;
> +        pp->cfg1_mod_base &= CPU_TO_BUS_ADDR;
> +    }
> +
>    dw_pcie_setup_rc(pp);
>    dra7xx_pcie_establish_link(pp);
>    if (IS_ENABLED(CONFIG_PCI_MSI))
> diff --git a/drivers/pci/host/pcie-designware.c b/drivers/pci/host/pcie-designware.c
> index 69486be..06c682b 100644
> --- a/drivers/pci/host/pcie-designware.c
> +++ b/drivers/pci/host/pcie-designware.c
> @@ -416,8 +416,7 @@ int dw_pcie_host_init(struct pcie_port *pp)
>            pp->io_base = range.cpu_addr;
> 
>            /* Find the untranslated IO space address */
> -            pp->io_mod_base = of_read_number(parser.range -
> -                             parser.np + na, ns);
> +            pp->io_mod_base = range.cpu_addr;;
>        }
>        if (restype == IORESOURCE_MEM) {
>            of_pci_range_to_resource(&range, np, &pp->mem);
> @@ -426,8 +425,7 @@ int dw_pcie_host_init(struct pcie_port *pp)
>            pp->mem_bus_addr = range.pci_addr;
> 
>            /* Find the untranslated MEM space address */
> -            pp->mem_mod_base = of_read_number(parser.range -
> -                              parser.np + na, ns);
> +            pp->mem_mod_base = range.cpu_addr;
>        }
>        if (restype == 0) {
>            of_pci_range_to_resource(&range, np, &pp->cfg);
> @@ -437,8 +435,7 @@ int dw_pcie_host_init(struct pcie_port *pp)
>            pp->cfg1_base = pp->cfg.start + pp->cfg0_size;
> 
>            /* Find the untranslated configuration space address */
> -            pp->cfg0_mod_base = of_read_number(parser.range -
> -                               parser.np + na, ns);
> +            pp->cfg0_mod_base = range.cpu_addr;
>            pp->cfg1_mod_base = pp->cfg0_mod_base +
>                        pp->cfg0_size;
>        }
> -- 
> 1.9.1
> 
> 
>> -----Original Message-----
>> From: linux-pci-owner at vger.kernel.org [mailto:linux-pci-
>> owner at vger.kernel.org] On Behalf Of Rob Herring
>> Sent: Friday, July 31, 2015 5:53 PM
>> To: Kishon Vijay Abraham I
>> Cc: Gabriele Paoloni; Bjorn Helgaas; arnd at arndb.de;
>> lorenzo.pieralisi at arm.com; Wangzhou (B); robh+dt at kernel.org;
>> james.morse at arm.com; Liviu.Dudau at arm.com; linux-pci at vger.kernel.org;
>> linux-arm-kernel at lists.infradead.org; devicetree at vger.kernel.org;
>> Yuanzhichang; Zhudacai; zhangjukuo; qiuzhenfa; Liguozhu (Kenneth);
>> Jingoo Han; Pratyush Anand; Arnd Bergmann
>> Subject: Re: [PATCH v6] PCI: Store PCIe bus address in struct
>> of_pci_range
>> 
>> On Fri, Jul 31, 2015 at 9:57 AM, Kishon Vijay Abraham I <kishon@ti.com>
>> wrote:
>>> +Arnd
>>> 
>>> Hi,
>>> 
>>>> On Friday 31 July 2015 07:55 PM, Gabriele Paoloni wrote:
>>>> [+cc Kishon]
>>>> 
>>>>> -----Original Message-----
>>>>> From: linux-pci-owner at vger.kernel.org [mailto:linux-pci-
>>>>> owner at vger.kernel.org] On Behalf Of Rob Herring
>>>>> Sent: Thursday, July 30, 2015 9:42 PM
>>>>> To: Gabriele Paoloni
>>>>> Cc: Bjorn Helgaas; arnd at arndb.de; lorenzo.pieralisi at arm.com;
>> Wangzhou
>>>>> (B); robh+dt at kernel.org; james.morse at arm.com; Liviu.Dudau at arm.com;
>>>>> linux-pci at vger.kernel.org; linux-arm-kernel at lists.infradead.org;
>>>>> devicetree at vger.kernel.org; Yuanzhichang; Zhudacai; zhangjukuo;
>>>>> qiuzhenfa; Liguozhu (Kenneth); Jingoo Han; Pratyush Anand
>>>>> Subject: Re: [PATCH v6] PCI: Store PCIe bus address in struct
>>>>> of_pci_range
>>>>> 
>>>>> On Thu, Jul 30, 2015 at 12:34 PM, Gabriele Paoloni
>>>>> <gabriele.paoloni@huawei.com> wrote:
>>>>>>> -----Original Message-----
>>>>>>> From: Bjorn Helgaas [mailto:bhelgaas at google.com]
>>>>>>> Sent: 30 July 2015 18:15
>>>>>>> On Thu, Jul 30, 2015 at 04:50:55PM +0000, Gabriele Paoloni wrote:
>>>>>>>>> -----Original Message-----
>>>>>>>>> From: linux-pci-owner at vger.kernel.org [mailto:linux-pci-
>>>>>>>>> owner at vger.kernel.org] On Behalf Of Bjorn Helgaas
>>>>>>>>> Sent: Thursday, July 30, 2015 5:15 PM
>>>>>>>>> On Thu, Jul 30, 2015 at 01:52:13PM +0000, Gabriele Paoloni
>> wrote:
>>>>> 
>>>>> [...]
>>>>> 
>>>>>>>>>> I don?t think we should rely on [CPU] addresses...what if the
>>>>>>>>> intermediate
>>>>>>>>>> translation layer changes the lower significant bits of the
>>>>> "bus
>>>>>>>>> address"
>>>>>>>>>> to translate into a cpu address?
>>>>>>>>> 
>>>>>>>>> Is it really a possiblity that the lower bits could be changed?
>>>>>>>> 
>>>>>>>> I've checked all the current deignware users DTs except "pci-
>>>>>>> layerscape"
>>>>>>>> that I could not find:
>>>>>>>> spear1310.dtsi
>>>>>>>> spear1340.dtsi
>>>>>>>> dra7.dtsi
>>>>>>>> imx6qdl.dtsi
>>>>>>>> imx6sx.dtsi
>>>>>>>> keystone.dtsi
>>>>>>>> exynos5440.dtsi
>>>>>>>> 
>>>>>>>> None of them modifies the lower bits. To be more precise the
>> only
>>>>> guy
>>>>>>>> that provides another translation layer is "dra7.dtsi":
>>>>>>>> axi0
>>>>>>>> http://lxr.free-
>>>>> electrons.com/source/arch/arm/boot/dts/dra7.dtsi#L207
>>>>>>>> 
>>>>>>>> axi1
>>>>>>>> http://lxr.free-
>>>>> electrons.com/source/arch/arm/boot/dts/dra7.dtsi#L241
>>>>>>>> 
>>>>>>>> For this case masking the top 4bits (bits28 to 31) should make
>> the
>>>>> job.
>>>>> 
>>>>> IMO, we should just fix this case. After further study, I don't
>> think
>>>>> this is a DW issue, but rather an SOC integration issue.
>>>>> 
>>>>> I believe you can just fixup the address in the pp->ops->host_init
>> hook.
>>>> 
>>>> Yes I guess that I could just assign pp->(*)_mod_base to the CPU
>> address
>>>> in DW and mask it out in dra7xx_pcie_host_init()...
>>>> 
>>>> Kishon, would you be ok with that?
>>> 
>>> Initially I was using *base-mask* property from dt. Me and Arnd
>> (cc'ed) had
>>> this discussion [1] before we decided the current approach. It'll be
>> good to
>>> check with Arnd too.
>>> 
>>> [1] ->  http://lists.infradead.org/pipermail/linux-arm-kernel/2014-
>> May/253528.html
>> 
>> The problem I have here is the use of ranges does not necessarily mean
>> fewer address bits are available. It can be used just for convenience
>> of not putting the full address into every node's reg property. And
>> vice versa, there are probably plenty of cases where we have the full
>> address in the nodes, but really only some of the address bits are
>> decoded at the IP block. Whether the address bits are present is
>> rarely cared about or known by s/w folks until you hit a problem like
>> this. Given this is an isolated case ATM, I would fix it in an
>> isolated way. It does not affect the binding and could be changed in
>> the kernel later if this becomes a common need.
>> 
>> Rob
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-pci" in
>> the body of a message to majordomo at vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 73+ messages in thread
* RE: [PATCH v6] PCI: Store PCIe bus address in struct of_pci_range
  2015-08-04  4:19                                   ` Jingoo Han
  (?)
@ 2015-08-04 10:12                                     ` Gabriele Paoloni
  -1 siblings, 0 replies; 73+ messages in thread
From: Gabriele Paoloni @ 2015-08-04 10:12 UTC (permalink / raw)
  To: Jingoo Han
  Cc: Rob Herring, Kishon Vijay Abraham I, Bjorn Helgaas, arnd,
	lorenzo.pieralisi, Wangzhou (B),
	robh+dt, james.morse, Liviu.Dudau, linux-pci, linux-arm-kernel,
	devicetree, Yuanzhichang, Zhudacai, zhangjukuo, qiuzhenfa,
	Liguozhu (Kenneth),
	Pratyush Anand, Arnd Bergmann



> -----Original Message-----
> From: Jingoo Han [mailto:jingoohan1@gmail.com]
> Sent: Tuesday, August 04, 2015 5:20 AM
> To: Gabriele Paoloni
> Cc: Rob Herring; Kishon Vijay Abraham I; Bjorn Helgaas; arnd@arndb.de;
> lorenzo.pieralisi@arm.com; Wangzhou (B); robh+dt@kernel.org;
> james.morse@arm.com; Liviu.Dudau@arm.com; linux-pci@vger.kernel.org;
> linux-arm-kernel@lists.infradead.org; devicetree@vger.kernel.org;
> Yuanzhichang; Zhudacai; zhangjukuo; qiuzhenfa; Liguozhu (Kenneth);
> Pratyush Anand; Arnd Bergmann; jingoohan1@gmail.com
> Subject: Re: [PATCH v6] PCI: Store PCIe bus address in struct
> of_pci_range
> 
> On 2015. 8. 3., at PM 8:18, Gabriele Paoloni
> <gabriele.paoloni@huawei.com> wrote:
> >
> > Rob, Kishon what about the following solution?....
> >
> > ---
> > drivers/pci/host/pci-dra7xx.c      | 12 ++++++++++++
> > drivers/pci/host/pcie-designware.c |  9 +++------
> 
> Hi Paoloni,
> 
> OK, it looks good.
> I want you to revert the following patch.
> 
> commit "f4c55c5a3f7f (PCI: designware: Program ATU with untranslated
> address)"
> 
> Would you remove all '*_mode_base's?

Hi Jingoo Han,

If I reverted the commit, in order to get designware working, dra7 
should mask directly the CPU addresses stored in pp->cfg0_base, 
pp->cfg1_base, pp->mem_base and pp->io_base.

The masking would happen at this point: 
http://lxr.free-electrons.com/source/drivers/pci/host/pcie-designware.c#L493

Now:
- I see that pp->cfg<0/1>_base are used in devm_ioremap before that 
  point and nowhere else, so they should be ok.
- pp->mem_base is used in dw_pcie_setup_rc(); dw_pcie_setup_rc() is called
  in dra7xx_pcie_host_init()...so here I should move the masking after
  dw_pcie_setup() retuned, but I think it should be ok
- pp->io_base is used in pci_ioremap_io() in dw_pcie_setup(). Now currently
  in designware this is called by pci_bios_init_hw(); this is after the 
  masking....so here we would have a the wrong value passed

Said that if you look at "[PATCH v5 2/5] PCI: designware: Add ARM64 support",
that is the patchset where this patch is needed, you can see that 
dw_pcie_setup() is removed and pp->io_base is used in pci_remap_iospace() before 
the masking would happen in dra7xx_pcie_host_init()...so for this patchset we 
should be good to revert the commit.

I propose to add a new patch in the patchset "[PATCH v5 0/5] PCI: hisi: Add PCIe 
host support for HiSilicon SoC Hip05" as the one I just posted in this 
thread (this would not revert the commit but just moves the masking to dra7xx)

I would revert the commit in "[PATCH v5 2/5] PCI: designware: Add ARM64 support"
together with the rest of designware rework. 

So we would have always a working version of designware...

Are you ok with this?

> 
> Best regards,
> Jingoo Han
> 
> > 2 files changed, 15 insertions(+), 6 deletions(-)
> >
> > diff --git a/drivers/pci/host/pci-dra7xx.c b/drivers/pci/host/pci-
> dra7xx.c
> > index 80db09e..bb2635f 100644
> > --- a/drivers/pci/host/pci-dra7xx.c
> > +++ b/drivers/pci/host/pci-dra7xx.c
> > @@ -61,6 +61,7 @@
> >
> > #define    PCIECTRL_DRA7XX_CONF_PHY_CS            0x010C
> > #define    LINK_UP                        BIT(16)
> > +#define CPU_TO_BUS_ADDR             0x0FFFFFFF
> >
> > struct dra7xx_pcie {
> >    void __iomem        *base;
> > @@ -138,6 +139,17 @@ static void dra7xx_pcie_enable_interrupts(struct
> pcie_port *pp)
> >
> > static void dra7xx_pcie_host_init(struct pcie_port *pp)
> > {
> > +    if (pp->io_mod_base)
> > +        pp->io_mod_base &= CPU_TO_BUS_ADDR;
> > +
> > +    if (pp->mem_mod_base)
> > +        pp->mem_mod_base &= CPU_TO_BUS_ADDR;
> > +
> > +    if (pp->cfg0_mod_base) {
> > +        pp->cfg0_mod_base &= CPU_TO_BUS_ADDR;
> > +        pp->cfg1_mod_base &= CPU_TO_BUS_ADDR;
> > +    }
> > +
> >    dw_pcie_setup_rc(pp);
> >    dra7xx_pcie_establish_link(pp);
> >    if (IS_ENABLED(CONFIG_PCI_MSI))
> > diff --git a/drivers/pci/host/pcie-designware.c
> b/drivers/pci/host/pcie-designware.c
> > index 69486be..06c682b 100644
> > --- a/drivers/pci/host/pcie-designware.c
> > +++ b/drivers/pci/host/pcie-designware.c
> > @@ -416,8 +416,7 @@ int dw_pcie_host_init(struct pcie_port *pp)
> >            pp->io_base = range.cpu_addr;
> >
> >            /* Find the untranslated IO space address */
> > -            pp->io_mod_base = of_read_number(parser.range -
> > -                             parser.np + na, ns);
> > +            pp->io_mod_base = range.cpu_addr;;
> >        }
> >        if (restype == IORESOURCE_MEM) {
> >            of_pci_range_to_resource(&range, np, &pp->mem);
> > @@ -426,8 +425,7 @@ int dw_pcie_host_init(struct pcie_port *pp)
> >            pp->mem_bus_addr = range.pci_addr;
> >
> >            /* Find the untranslated MEM space address */
> > -            pp->mem_mod_base = of_read_number(parser.range -
> > -                              parser.np + na, ns);
> > +            pp->mem_mod_base = range.cpu_addr;
> >        }
> >        if (restype == 0) {
> >            of_pci_range_to_resource(&range, np, &pp->cfg);
> > @@ -437,8 +435,7 @@ int dw_pcie_host_init(struct pcie_port *pp)
> >            pp->cfg1_base = pp->cfg.start + pp->cfg0_size;
> >
> >            /* Find the untranslated configuration space address */
> > -            pp->cfg0_mod_base = of_read_number(parser.range -
> > -                               parser.np + na, ns);
> > +            pp->cfg0_mod_base = range.cpu_addr;
> >            pp->cfg1_mod_base = pp->cfg0_mod_base +
> >                        pp->cfg0_size;
> >        }
> > --
> > 1.9.1
> >
> >
> >> -----Original Message-----
> >> From: linux-pci-owner@vger.kernel.org [mailto:linux-pci-
> >> owner@vger.kernel.org] On Behalf Of Rob Herring
> >> Sent: Friday, July 31, 2015 5:53 PM
> >> To: Kishon Vijay Abraham I
> >> Cc: Gabriele Paoloni; Bjorn Helgaas; arnd@arndb.de;
> >> lorenzo.pieralisi@arm.com; Wangzhou (B); robh+dt@kernel.org;
> >> james.morse@arm.com; Liviu.Dudau@arm.com; linux-pci@vger.kernel.org;
> >> linux-arm-kernel@lists.infradead.org; devicetree@vger.kernel.org;
> >> Yuanzhichang; Zhudacai; zhangjukuo; qiuzhenfa; Liguozhu (Kenneth);
> >> Jingoo Han; Pratyush Anand; Arnd Bergmann
> >> Subject: Re: [PATCH v6] PCI: Store PCIe bus address in struct
> >> of_pci_range
> >>
> >> On Fri, Jul 31, 2015 at 9:57 AM, Kishon Vijay Abraham I
> <kishon@ti.com>
> >> wrote:
> >>> +Arnd
> >>>
> >>> Hi,
> >>>
> >>>> On Friday 31 July 2015 07:55 PM, Gabriele Paoloni wrote:
> >>>> [+cc Kishon]
> >>>>
> >>>>> -----Original Message-----
> >>>>> From: linux-pci-owner@vger.kernel.org [mailto:linux-pci-
> >>>>> owner@vger.kernel.org] On Behalf Of Rob Herring
> >>>>> Sent: Thursday, July 30, 2015 9:42 PM
> >>>>> To: Gabriele Paoloni
> >>>>> Cc: Bjorn Helgaas; arnd@arndb.de; lorenzo.pieralisi@arm.com;
> >> Wangzhou
> >>>>> (B); robh+dt@kernel.org; james.morse@arm.com; Liviu.Dudau@arm.com;
> >>>>> linux-pci@vger.kernel.org; linux-arm-kernel@lists.infradead.org;
> >>>>> devicetree@vger.kernel.org; Yuanzhichang; Zhudacai; zhangjukuo;
> >>>>> qiuzhenfa; Liguozhu (Kenneth); Jingoo Han; Pratyush Anand
> >>>>> Subject: Re: [PATCH v6] PCI: Store PCIe bus address in struct
> >>>>> of_pci_range
> >>>>>
> >>>>> On Thu, Jul 30, 2015 at 12:34 PM, Gabriele Paoloni
> >>>>> <gabriele.paoloni@huawei.com> wrote:
> >>>>>>> -----Original Message-----
> >>>>>>> From: Bjorn Helgaas [mailto:bhelgaas@google.com]
> >>>>>>> Sent: 30 July 2015 18:15
> >>>>>>> On Thu, Jul 30, 2015 at 04:50:55PM +0000, Gabriele Paoloni
> wrote:
> >>>>>>>>> -----Original Message-----
> >>>>>>>>> From: linux-pci-owner@vger.kernel.org [mailto:linux-pci-
> >>>>>>>>> owner@vger.kernel.org] On Behalf Of Bjorn Helgaas
> >>>>>>>>> Sent: Thursday, July 30, 2015 5:15 PM
> >>>>>>>>> On Thu, Jul 30, 2015 at 01:52:13PM +0000, Gabriele Paoloni
> >> wrote:
> >>>>>
> >>>>> [...]
> >>>>>
> >>>>>>>>>> I don’t think we should rely on [CPU] addresses...what if
> the
> >>>>>>>>> intermediate
> >>>>>>>>>> translation layer changes the lower significant bits of the
> >>>>> "bus
> >>>>>>>>> address"
> >>>>>>>>>> to translate into a cpu address?
> >>>>>>>>>
> >>>>>>>>> Is it really a possiblity that the lower bits could be
> changed?
> >>>>>>>>
> >>>>>>>> I've checked all the current deignware users DTs except "pci-
> >>>>>>> layerscape"
> >>>>>>>> that I could not find:
> >>>>>>>> spear1310.dtsi
> >>>>>>>> spear1340.dtsi
> >>>>>>>> dra7.dtsi
> >>>>>>>> imx6qdl.dtsi
> >>>>>>>> imx6sx.dtsi
> >>>>>>>> keystone.dtsi
> >>>>>>>> exynos5440.dtsi
> >>>>>>>>
> >>>>>>>> None of them modifies the lower bits. To be more precise the
> >> only
> >>>>> guy
> >>>>>>>> that provides another translation layer is "dra7.dtsi":
> >>>>>>>> axi0
> >>>>>>>> http://lxr.free-
> >>>>> electrons.com/source/arch/arm/boot/dts/dra7.dtsi#L207
> >>>>>>>>
> >>>>>>>> axi1
> >>>>>>>> http://lxr.free-
> >>>>> electrons.com/source/arch/arm/boot/dts/dra7.dtsi#L241
> >>>>>>>>
> >>>>>>>> For this case masking the top 4bits (bits28 to 31) should make
> >> the
> >>>>> job.
> >>>>>
> >>>>> IMO, we should just fix this case. After further study, I don't
> >> think
> >>>>> this is a DW issue, but rather an SOC integration issue.
> >>>>>
> >>>>> I believe you can just fixup the address in the pp->ops-
> >host_init
> >> hook.
> >>>>
> >>>> Yes I guess that I could just assign pp->(*)_mod_base to the CPU
> >> address
> >>>> in DW and mask it out in dra7xx_pcie_host_init()...
> >>>>
> >>>> Kishon, would you be ok with that?
> >>>
> >>> Initially I was using *base-mask* property from dt. Me and Arnd
> >> (cc'ed) had
> >>> this discussion [1] before we decided the current approach. It'll
> be
> >> good to
> >>> check with Arnd too.
> >>>
> >>> [1] ->  http://lists.infradead.org/pipermail/linux-arm-kernel/2014-
> >> May/253528.html
> >>
> >> The problem I have here is the use of ranges does not necessarily
> mean
> >> fewer address bits are available. It can be used just for
> convenience
> >> of not putting the full address into every node's reg property. And
> >> vice versa, there are probably plenty of cases where we have the
> full
> >> address in the nodes, but really only some of the address bits are
> >> decoded at the IP block. Whether the address bits are present is
> >> rarely cared about or known by s/w folks until you hit a problem
> like
> >> this. Given this is an isolated case ATM, I would fix it in an
> >> isolated way. It does not affect the binding and could be changed in
> >> the kernel later if this becomes a common need.
> >>
> >> Rob
> >> --
> >> To unsubscribe from this list: send the line "unsubscribe linux-pci"
> in
> >> the body of a message to majordomo@vger.kernel.org
> >> More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 73+ messages in thread
* RE: [PATCH v6] PCI: Store PCIe bus address in struct of_pci_range
@ 2015-08-04 10:12                                     ` Gabriele Paoloni
  0 siblings, 0 replies; 73+ messages in thread
From: Gabriele Paoloni @ 2015-08-04 10:12 UTC (permalink / raw)
  To: Jingoo Han
  Cc: Rob Herring, Kishon Vijay Abraham I, Bjorn Helgaas, arnd,
	lorenzo.pieralisi, Wangzhou (B),
	robh+dt, james.morse, Liviu.Dudau, linux-pci, linux-arm-kernel,
	devicetree, Yuanzhichang, Zhudacai, zhangjukuo, qiuzhenfa,
	Liguozhu (Kenneth),
	Pratyush Anand, Arnd Bergmann

DQoNCj4gLS0tLS1PcmlnaW5hbCBNZXNzYWdlLS0tLS0NCj4gRnJvbTogSmluZ29vIEhhbiBbbWFp
bHRvOmppbmdvb2hhbjFAZ21haWwuY29tXQ0KPiBTZW50OiBUdWVzZGF5LCBBdWd1c3QgMDQsIDIw
MTUgNToyMCBBTQ0KPiBUbzogR2FicmllbGUgUGFvbG9uaQ0KPiBDYzogUm9iIEhlcnJpbmc7IEtp
c2hvbiBWaWpheSBBYnJhaGFtIEk7IEJqb3JuIEhlbGdhYXM7IGFybmRAYXJuZGIuZGU7DQo+IGxv
cmVuem8ucGllcmFsaXNpQGFybS5jb207IFdhbmd6aG91IChCKTsgcm9iaCtkdEBrZXJuZWwub3Jn
Ow0KPiBqYW1lcy5tb3JzZUBhcm0uY29tOyBMaXZpdS5EdWRhdUBhcm0uY29tOyBsaW51eC1wY2lA
dmdlci5rZXJuZWwub3JnOw0KPiBsaW51eC1hcm0ta2VybmVsQGxpc3RzLmluZnJhZGVhZC5vcmc7
IGRldmljZXRyZWVAdmdlci5rZXJuZWwub3JnOw0KPiBZdWFuemhpY2hhbmc7IFpodWRhY2FpOyB6
aGFuZ2p1a3VvOyBxaXV6aGVuZmE7IExpZ3Vvemh1IChLZW5uZXRoKTsNCj4gUHJhdHl1c2ggQW5h
bmQ7IEFybmQgQmVyZ21hbm47IGppbmdvb2hhbjFAZ21haWwuY29tDQo+IFN1YmplY3Q6IFJlOiBb
UEFUQ0ggdjZdIFBDSTogU3RvcmUgUENJZSBidXMgYWRkcmVzcyBpbiBzdHJ1Y3QNCj4gb2ZfcGNp
X3JhbmdlDQo+IA0KPiBPbiAyMDE1LiA4LiAzLiwgYXQgUE0gODoxOCwgR2FicmllbGUgUGFvbG9u
aQ0KPiA8Z2FicmllbGUucGFvbG9uaUBodWF3ZWkuY29tPiB3cm90ZToNCj4gPg0KPiA+IFJvYiwg
S2lzaG9uIHdoYXQgYWJvdXQgdGhlIGZvbGxvd2luZyBzb2x1dGlvbj8uLi4uDQo+ID4NCj4gPiAt
LS0NCj4gPiBkcml2ZXJzL3BjaS9ob3N0L3BjaS1kcmE3eHguYyAgICAgIHwgMTIgKysrKysrKysr
KysrDQo+ID4gZHJpdmVycy9wY2kvaG9zdC9wY2llLWRlc2lnbndhcmUuYyB8ICA5ICsrKy0tLS0t
LQ0KPiANCj4gSGkgUGFvbG9uaSwNCj4gDQo+IE9LLCBpdCBsb29rcyBnb29kLg0KPiBJIHdhbnQg
eW91IHRvIHJldmVydCB0aGUgZm9sbG93aW5nIHBhdGNoLg0KPiANCj4gY29tbWl0ICJmNGM1NWM1
YTNmN2YgKFBDSTogZGVzaWdud2FyZTogUHJvZ3JhbSBBVFUgd2l0aCB1bnRyYW5zbGF0ZWQNCj4g
YWRkcmVzcykiDQo+IA0KPiBXb3VsZCB5b3UgcmVtb3ZlIGFsbCAnKl9tb2RlX2Jhc2Uncz8NCg0K
SGkgSmluZ29vIEhhbiwNCg0KSWYgSSByZXZlcnRlZCB0aGUgY29tbWl0LCBpbiBvcmRlciB0byBn
ZXQgZGVzaWdud2FyZSB3b3JraW5nLCBkcmE3IA0Kc2hvdWxkIG1hc2sgZGlyZWN0bHkgdGhlIENQ
VSBhZGRyZXNzZXMgc3RvcmVkIGluIHBwLT5jZmcwX2Jhc2UsIA0KcHAtPmNmZzFfYmFzZSwgcHAt
Pm1lbV9iYXNlIGFuZCBwcC0+aW9fYmFzZS4NCg0KVGhlIG1hc2tpbmcgd291bGQgaGFwcGVuIGF0
IHRoaXMgcG9pbnQ6IA0KaHR0cDovL2x4ci5mcmVlLWVsZWN0cm9ucy5jb20vc291cmNlL2RyaXZl
cnMvcGNpL2hvc3QvcGNpZS1kZXNpZ253YXJlLmMjTDQ5Mw0KDQpOb3c6DQotIEkgc2VlIHRoYXQg
cHAtPmNmZzwwLzE+X2Jhc2UgYXJlIHVzZWQgaW4gZGV2bV9pb3JlbWFwIGJlZm9yZSB0aGF0IA0K
ICBwb2ludCBhbmQgbm93aGVyZSBlbHNlLCBzbyB0aGV5IHNob3VsZCBiZSBvay4NCi0gcHAtPm1l
bV9iYXNlIGlzIHVzZWQgaW4gZHdfcGNpZV9zZXR1cF9yYygpOyBkd19wY2llX3NldHVwX3JjKCkg
aXMgY2FsbGVkDQogIGluIGRyYTd4eF9wY2llX2hvc3RfaW5pdCgpLi4uc28gaGVyZSBJIHNob3Vs
ZCBtb3ZlIHRoZSBtYXNraW5nIGFmdGVyDQogIGR3X3BjaWVfc2V0dXAoKSByZXR1bmVkLCBidXQg
SSB0aGluayBpdCBzaG91bGQgYmUgb2sNCi0gcHAtPmlvX2Jhc2UgaXMgdXNlZCBpbiBwY2lfaW9y
ZW1hcF9pbygpIGluIGR3X3BjaWVfc2V0dXAoKS4gTm93IGN1cnJlbnRseQ0KICBpbiBkZXNpZ253
YXJlIHRoaXMgaXMgY2FsbGVkIGJ5IHBjaV9iaW9zX2luaXRfaHcoKTsgdGhpcyBpcyBhZnRlciB0
aGUgDQogIG1hc2tpbmcuLi4uc28gaGVyZSB3ZSB3b3VsZCBoYXZlIGEgdGhlIHdyb25nIHZhbHVl
IHBhc3NlZA0KDQpTYWlkIHRoYXQgaWYgeW91IGxvb2sgYXQgIltQQVRDSCB2NSAyLzVdIFBDSTog
ZGVzaWdud2FyZTogQWRkIEFSTTY0IHN1cHBvcnQiLA0KdGhhdCBpcyB0aGUgcGF0Y2hzZXQgd2hl
cmUgdGhpcyBwYXRjaCBpcyBuZWVkZWQsIHlvdSBjYW4gc2VlIHRoYXQgDQpkd19wY2llX3NldHVw
KCkgaXMgcmVtb3ZlZCBhbmQgcHAtPmlvX2Jhc2UgaXMgdXNlZCBpbiBwY2lfcmVtYXBfaW9zcGFj
ZSgpIGJlZm9yZSANCnRoZSBtYXNraW5nIHdvdWxkIGhhcHBlbiBpbiBkcmE3eHhfcGNpZV9ob3N0
X2luaXQoKS4uLnNvIGZvciB0aGlzIHBhdGNoc2V0IHdlIA0Kc2hvdWxkIGJlIGdvb2QgdG8gcmV2
ZXJ0IHRoZSBjb21taXQuDQoNCkkgcHJvcG9zZSB0byBhZGQgYSBuZXcgcGF0Y2ggaW4gdGhlIHBh
dGNoc2V0ICJbUEFUQ0ggdjUgMC81XSBQQ0k6IGhpc2k6IEFkZCBQQ0llIA0KaG9zdCBzdXBwb3J0
IGZvciBIaVNpbGljb24gU29DIEhpcDA1IiBhcyB0aGUgb25lIEkganVzdCBwb3N0ZWQgaW4gdGhp
cyANCnRocmVhZCAodGhpcyB3b3VsZCBub3QgcmV2ZXJ0IHRoZSBjb21taXQgYnV0IGp1c3QgbW92
ZXMgdGhlIG1hc2tpbmcgdG8gZHJhN3h4KQ0KDQpJIHdvdWxkIHJldmVydCB0aGUgY29tbWl0IGlu
ICJbUEFUQ0ggdjUgMi81XSBQQ0k6IGRlc2lnbndhcmU6IEFkZCBBUk02NCBzdXBwb3J0Ig0KdG9n
ZXRoZXIgd2l0aCB0aGUgcmVzdCBvZiBkZXNpZ253YXJlIHJld29yay4gDQoNClNvIHdlIHdvdWxk
IGhhdmUgYWx3YXlzIGEgd29ya2luZyB2ZXJzaW9uIG9mIGRlc2lnbndhcmUuLi4NCg0KQXJlIHlv
dSBvayB3aXRoIHRoaXM/DQoNCj4gDQo+IEJlc3QgcmVnYXJkcywNCj4gSmluZ29vIEhhbg0KPiAN
Cj4gPiAyIGZpbGVzIGNoYW5nZWQsIDE1IGluc2VydGlvbnMoKyksIDYgZGVsZXRpb25zKC0pDQo+
ID4NCj4gPiBkaWZmIC0tZ2l0IGEvZHJpdmVycy9wY2kvaG9zdC9wY2ktZHJhN3h4LmMgYi9kcml2
ZXJzL3BjaS9ob3N0L3BjaS0NCj4gZHJhN3h4LmMNCj4gPiBpbmRleCA4MGRiMDllLi5iYjI2MzVm
IDEwMDY0NA0KPiA+IC0tLSBhL2RyaXZlcnMvcGNpL2hvc3QvcGNpLWRyYTd4eC5jDQo+ID4gKysr
IGIvZHJpdmVycy9wY2kvaG9zdC9wY2ktZHJhN3h4LmMNCj4gPiBAQCAtNjEsNiArNjEsNyBAQA0K
PiA+DQo+ID4gI2RlZmluZSAgICBQQ0lFQ1RSTF9EUkE3WFhfQ09ORl9QSFlfQ1MgICAgICAgICAg
ICAweDAxMEMNCj4gPiAjZGVmaW5lICAgIExJTktfVVAgICAgICAgICAgICAgICAgICAgICAgICBC
SVQoMTYpDQo+ID4gKyNkZWZpbmUgQ1BVX1RPX0JVU19BRERSICAgICAgICAgICAgIDB4MEZGRkZG
RkYNCj4gPg0KPiA+IHN0cnVjdCBkcmE3eHhfcGNpZSB7DQo+ID4gICAgdm9pZCBfX2lvbWVtICAg
ICAgICAqYmFzZTsNCj4gPiBAQCAtMTM4LDYgKzEzOSwxNyBAQCBzdGF0aWMgdm9pZCBkcmE3eHhf
cGNpZV9lbmFibGVfaW50ZXJydXB0cyhzdHJ1Y3QNCj4gcGNpZV9wb3J0ICpwcCkNCj4gPg0KPiA+
IHN0YXRpYyB2b2lkIGRyYTd4eF9wY2llX2hvc3RfaW5pdChzdHJ1Y3QgcGNpZV9wb3J0ICpwcCkN
Cj4gPiB7DQo+ID4gKyAgICBpZiAocHAtPmlvX21vZF9iYXNlKQ0KPiA+ICsgICAgICAgIHBwLT5p
b19tb2RfYmFzZSAmPSBDUFVfVE9fQlVTX0FERFI7DQo+ID4gKw0KPiA+ICsgICAgaWYgKHBwLT5t
ZW1fbW9kX2Jhc2UpDQo+ID4gKyAgICAgICAgcHAtPm1lbV9tb2RfYmFzZSAmPSBDUFVfVE9fQlVT
X0FERFI7DQo+ID4gKw0KPiA+ICsgICAgaWYgKHBwLT5jZmcwX21vZF9iYXNlKSB7DQo+ID4gKyAg
ICAgICAgcHAtPmNmZzBfbW9kX2Jhc2UgJj0gQ1BVX1RPX0JVU19BRERSOw0KPiA+ICsgICAgICAg
IHBwLT5jZmcxX21vZF9iYXNlICY9IENQVV9UT19CVVNfQUREUjsNCj4gPiArICAgIH0NCj4gPiAr
DQo+ID4gICAgZHdfcGNpZV9zZXR1cF9yYyhwcCk7DQo+ID4gICAgZHJhN3h4X3BjaWVfZXN0YWJs
aXNoX2xpbmsocHApOw0KPiA+ICAgIGlmIChJU19FTkFCTEVEKENPTkZJR19QQ0lfTVNJKSkNCj4g
PiBkaWZmIC0tZ2l0IGEvZHJpdmVycy9wY2kvaG9zdC9wY2llLWRlc2lnbndhcmUuYw0KPiBiL2Ry
aXZlcnMvcGNpL2hvc3QvcGNpZS1kZXNpZ253YXJlLmMNCj4gPiBpbmRleCA2OTQ4NmJlLi4wNmM2
ODJiIDEwMDY0NA0KPiA+IC0tLSBhL2RyaXZlcnMvcGNpL2hvc3QvcGNpZS1kZXNpZ253YXJlLmMN
Cj4gPiArKysgYi9kcml2ZXJzL3BjaS9ob3N0L3BjaWUtZGVzaWdud2FyZS5jDQo+ID4gQEAgLTQx
Niw4ICs0MTYsNyBAQCBpbnQgZHdfcGNpZV9ob3N0X2luaXQoc3RydWN0IHBjaWVfcG9ydCAqcHAp
DQo+ID4gICAgICAgICAgICBwcC0+aW9fYmFzZSA9IHJhbmdlLmNwdV9hZGRyOw0KPiA+DQo+ID4g
ICAgICAgICAgICAvKiBGaW5kIHRoZSB1bnRyYW5zbGF0ZWQgSU8gc3BhY2UgYWRkcmVzcyAqLw0K
PiA+IC0gICAgICAgICAgICBwcC0+aW9fbW9kX2Jhc2UgPSBvZl9yZWFkX251bWJlcihwYXJzZXIu
cmFuZ2UgLQ0KPiA+IC0gICAgICAgICAgICAgICAgICAgICAgICAgICAgIHBhcnNlci5ucCArIG5h
LCBucyk7DQo+ID4gKyAgICAgICAgICAgIHBwLT5pb19tb2RfYmFzZSA9IHJhbmdlLmNwdV9hZGRy
OzsNCj4gPiAgICAgICAgfQ0KPiA+ICAgICAgICBpZiAocmVzdHlwZSA9PSBJT1JFU09VUkNFX01F
TSkgew0KPiA+ICAgICAgICAgICAgb2ZfcGNpX3JhbmdlX3RvX3Jlc291cmNlKCZyYW5nZSwgbnAs
ICZwcC0+bWVtKTsNCj4gPiBAQCAtNDI2LDggKzQyNSw3IEBAIGludCBkd19wY2llX2hvc3RfaW5p
dChzdHJ1Y3QgcGNpZV9wb3J0ICpwcCkNCj4gPiAgICAgICAgICAgIHBwLT5tZW1fYnVzX2FkZHIg
PSByYW5nZS5wY2lfYWRkcjsNCj4gPg0KPiA+ICAgICAgICAgICAgLyogRmluZCB0aGUgdW50cmFu
c2xhdGVkIE1FTSBzcGFjZSBhZGRyZXNzICovDQo+ID4gLSAgICAgICAgICAgIHBwLT5tZW1fbW9k
X2Jhc2UgPSBvZl9yZWFkX251bWJlcihwYXJzZXIucmFuZ2UgLQ0KPiA+IC0gICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICBwYXJzZXIubnAgKyBuYSwgbnMpOw0KPiA+ICsgICAgICAgICAgICBw
cC0+bWVtX21vZF9iYXNlID0gcmFuZ2UuY3B1X2FkZHI7DQo+ID4gICAgICAgIH0NCj4gPiAgICAg
ICAgaWYgKHJlc3R5cGUgPT0gMCkgew0KPiA+ICAgICAgICAgICAgb2ZfcGNpX3JhbmdlX3RvX3Jl
c291cmNlKCZyYW5nZSwgbnAsICZwcC0+Y2ZnKTsNCj4gPiBAQCAtNDM3LDggKzQzNSw3IEBAIGlu
dCBkd19wY2llX2hvc3RfaW5pdChzdHJ1Y3QgcGNpZV9wb3J0ICpwcCkNCj4gPiAgICAgICAgICAg
IHBwLT5jZmcxX2Jhc2UgPSBwcC0+Y2ZnLnN0YXJ0ICsgcHAtPmNmZzBfc2l6ZTsNCj4gPg0KPiA+
ICAgICAgICAgICAgLyogRmluZCB0aGUgdW50cmFuc2xhdGVkIGNvbmZpZ3VyYXRpb24gc3BhY2Ug
YWRkcmVzcyAqLw0KPiA+IC0gICAgICAgICAgICBwcC0+Y2ZnMF9tb2RfYmFzZSA9IG9mX3JlYWRf
bnVtYmVyKHBhcnNlci5yYW5nZSAtDQo+ID4gLSAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICBwYXJzZXIubnAgKyBuYSwgbnMpOw0KPiA+ICsgICAgICAgICAgICBwcC0+Y2ZnMF9tb2RfYmFz
ZSA9IHJhbmdlLmNwdV9hZGRyOw0KPiA+ICAgICAgICAgICAgcHAtPmNmZzFfbW9kX2Jhc2UgPSBw
cC0+Y2ZnMF9tb2RfYmFzZSArDQo+ID4gICAgICAgICAgICAgICAgICAgICAgICBwcC0+Y2ZnMF9z
aXplOw0KPiA+ICAgICAgICB9DQo+ID4gLS0NCj4gPiAxLjkuMQ0KPiA+DQo+ID4NCj4gPj4gLS0t
LS1PcmlnaW5hbCBNZXNzYWdlLS0tLS0NCj4gPj4gRnJvbTogbGludXgtcGNpLW93bmVyQHZnZXIu
a2VybmVsLm9yZyBbbWFpbHRvOmxpbnV4LXBjaS0NCj4gPj4gb3duZXJAdmdlci5rZXJuZWwub3Jn
XSBPbiBCZWhhbGYgT2YgUm9iIEhlcnJpbmcNCj4gPj4gU2VudDogRnJpZGF5LCBKdWx5IDMxLCAy
MDE1IDU6NTMgUE0NCj4gPj4gVG86IEtpc2hvbiBWaWpheSBBYnJhaGFtIEkNCj4gPj4gQ2M6IEdh
YnJpZWxlIFBhb2xvbmk7IEJqb3JuIEhlbGdhYXM7IGFybmRAYXJuZGIuZGU7DQo+ID4+IGxvcmVu
em8ucGllcmFsaXNpQGFybS5jb207IFdhbmd6aG91IChCKTsgcm9iaCtkdEBrZXJuZWwub3JnOw0K
PiA+PiBqYW1lcy5tb3JzZUBhcm0uY29tOyBMaXZpdS5EdWRhdUBhcm0uY29tOyBsaW51eC1wY2lA
dmdlci5rZXJuZWwub3JnOw0KPiA+PiBsaW51eC1hcm0ta2VybmVsQGxpc3RzLmluZnJhZGVhZC5v
cmc7IGRldmljZXRyZWVAdmdlci5rZXJuZWwub3JnOw0KPiA+PiBZdWFuemhpY2hhbmc7IFpodWRh
Y2FpOyB6aGFuZ2p1a3VvOyBxaXV6aGVuZmE7IExpZ3Vvemh1IChLZW5uZXRoKTsNCj4gPj4gSmlu
Z29vIEhhbjsgUHJhdHl1c2ggQW5hbmQ7IEFybmQgQmVyZ21hbm4NCj4gPj4gU3ViamVjdDogUmU6
IFtQQVRDSCB2Nl0gUENJOiBTdG9yZSBQQ0llIGJ1cyBhZGRyZXNzIGluIHN0cnVjdA0KPiA+PiBv
Zl9wY2lfcmFuZ2UNCj4gPj4NCj4gPj4gT24gRnJpLCBKdWwgMzEsIDIwMTUgYXQgOTo1NyBBTSwg
S2lzaG9uIFZpamF5IEFicmFoYW0gSQ0KPiA8a2lzaG9uQHRpLmNvbT4NCj4gPj4gd3JvdGU6DQo+
ID4+PiArQXJuZA0KPiA+Pj4NCj4gPj4+IEhpLA0KPiA+Pj4NCj4gPj4+PiBPbiBGcmlkYXkgMzEg
SnVseSAyMDE1IDA3OjU1IFBNLCBHYWJyaWVsZSBQYW9sb25pIHdyb3RlOg0KPiA+Pj4+IFsrY2Mg
S2lzaG9uXQ0KPiA+Pj4+DQo+ID4+Pj4+IC0tLS0tT3JpZ2luYWwgTWVzc2FnZS0tLS0tDQo+ID4+
Pj4+IEZyb206IGxpbnV4LXBjaS1vd25lckB2Z2VyLmtlcm5lbC5vcmcgW21haWx0bzpsaW51eC1w
Y2ktDQo+ID4+Pj4+IG93bmVyQHZnZXIua2VybmVsLm9yZ10gT24gQmVoYWxmIE9mIFJvYiBIZXJy
aW5nDQo+ID4+Pj4+IFNlbnQ6IFRodXJzZGF5LCBKdWx5IDMwLCAyMDE1IDk6NDIgUE0NCj4gPj4+
Pj4gVG86IEdhYnJpZWxlIFBhb2xvbmkNCj4gPj4+Pj4gQ2M6IEJqb3JuIEhlbGdhYXM7IGFybmRA
YXJuZGIuZGU7IGxvcmVuem8ucGllcmFsaXNpQGFybS5jb207DQo+ID4+IFdhbmd6aG91DQo+ID4+
Pj4+IChCKTsgcm9iaCtkdEBrZXJuZWwub3JnOyBqYW1lcy5tb3JzZUBhcm0uY29tOyBMaXZpdS5E
dWRhdUBhcm0uY29tOw0KPiA+Pj4+PiBsaW51eC1wY2lAdmdlci5rZXJuZWwub3JnOyBsaW51eC1h
cm0ta2VybmVsQGxpc3RzLmluZnJhZGVhZC5vcmc7DQo+ID4+Pj4+IGRldmljZXRyZWVAdmdlci5r
ZXJuZWwub3JnOyBZdWFuemhpY2hhbmc7IFpodWRhY2FpOyB6aGFuZ2p1a3VvOw0KPiA+Pj4+PiBx
aXV6aGVuZmE7IExpZ3Vvemh1IChLZW5uZXRoKTsgSmluZ29vIEhhbjsgUHJhdHl1c2ggQW5hbmQN
Cj4gPj4+Pj4gU3ViamVjdDogUmU6IFtQQVRDSCB2Nl0gUENJOiBTdG9yZSBQQ0llIGJ1cyBhZGRy
ZXNzIGluIHN0cnVjdA0KPiA+Pj4+PiBvZl9wY2lfcmFuZ2UNCj4gPj4+Pj4NCj4gPj4+Pj4gT24g
VGh1LCBKdWwgMzAsIDIwMTUgYXQgMTI6MzQgUE0sIEdhYnJpZWxlIFBhb2xvbmkNCj4gPj4+Pj4g
PGdhYnJpZWxlLnBhb2xvbmlAaHVhd2VpLmNvbT4gd3JvdGU6DQo+ID4+Pj4+Pj4gLS0tLS1Pcmln
aW5hbCBNZXNzYWdlLS0tLS0NCj4gPj4+Pj4+PiBGcm9tOiBCam9ybiBIZWxnYWFzIFttYWlsdG86
YmhlbGdhYXNAZ29vZ2xlLmNvbV0NCj4gPj4+Pj4+PiBTZW50OiAzMCBKdWx5IDIwMTUgMTg6MTUN
Cj4gPj4+Pj4+PiBPbiBUaHUsIEp1bCAzMCwgMjAxNSBhdCAwNDo1MDo1NVBNICswMDAwLCBHYWJy
aWVsZSBQYW9sb25pDQo+IHdyb3RlOg0KPiA+Pj4+Pj4+Pj4gLS0tLS1PcmlnaW5hbCBNZXNzYWdl
LS0tLS0NCj4gPj4+Pj4+Pj4+IEZyb206IGxpbnV4LXBjaS1vd25lckB2Z2VyLmtlcm5lbC5vcmcg
W21haWx0bzpsaW51eC1wY2ktDQo+ID4+Pj4+Pj4+PiBvd25lckB2Z2VyLmtlcm5lbC5vcmddIE9u
IEJlaGFsZiBPZiBCam9ybiBIZWxnYWFzDQo+ID4+Pj4+Pj4+PiBTZW50OiBUaHVyc2RheSwgSnVs
eSAzMCwgMjAxNSA1OjE1IFBNDQo+ID4+Pj4+Pj4+PiBPbiBUaHUsIEp1bCAzMCwgMjAxNSBhdCAw
MTo1MjoxM1BNICswMDAwLCBHYWJyaWVsZSBQYW9sb25pDQo+ID4+IHdyb3RlOg0KPiA+Pj4+Pg0K
PiA+Pj4+PiBbLi4uXQ0KPiA+Pj4+Pg0KPiA+Pj4+Pj4+Pj4+IEkgZG9uoa90IHRoaW5rIHdlIHNo
b3VsZCByZWx5IG9uIFtDUFVdIGFkZHJlc3Nlcy4uLndoYXQgaWYNCj4gdGhlDQo+ID4+Pj4+Pj4+
PiBpbnRlcm1lZGlhdGUNCj4gPj4+Pj4+Pj4+PiB0cmFuc2xhdGlvbiBsYXllciBjaGFuZ2VzIHRo
ZSBsb3dlciBzaWduaWZpY2FudCBiaXRzIG9mIHRoZQ0KPiA+Pj4+PiAiYnVzDQo+ID4+Pj4+Pj4+
PiBhZGRyZXNzIg0KPiA+Pj4+Pj4+Pj4+IHRvIHRyYW5zbGF0ZSBpbnRvIGEgY3B1IGFkZHJlc3M/
DQo+ID4+Pj4+Pj4+Pg0KPiA+Pj4+Pj4+Pj4gSXMgaXQgcmVhbGx5IGEgcG9zc2libGl0eSB0aGF0
IHRoZSBsb3dlciBiaXRzIGNvdWxkIGJlDQo+IGNoYW5nZWQ/DQo+ID4+Pj4+Pj4+DQo+ID4+Pj4+
Pj4+IEkndmUgY2hlY2tlZCBhbGwgdGhlIGN1cnJlbnQgZGVpZ253YXJlIHVzZXJzIERUcyBleGNl
cHQgInBjaS0NCj4gPj4+Pj4+PiBsYXllcnNjYXBlIg0KPiA+Pj4+Pj4+PiB0aGF0IEkgY291bGQg
bm90IGZpbmQ6DQo+ID4+Pj4+Pj4+IHNwZWFyMTMxMC5kdHNpDQo+ID4+Pj4+Pj4+IHNwZWFyMTM0
MC5kdHNpDQo+ID4+Pj4+Pj4+IGRyYTcuZHRzaQ0KPiA+Pj4+Pj4+PiBpbXg2cWRsLmR0c2kNCj4g
Pj4+Pj4+Pj4gaW14NnN4LmR0c2kNCj4gPj4+Pj4+Pj4ga2V5c3RvbmUuZHRzaQ0KPiA+Pj4+Pj4+
PiBleHlub3M1NDQwLmR0c2kNCj4gPj4+Pj4+Pj4NCj4gPj4+Pj4+Pj4gTm9uZSBvZiB0aGVtIG1v
ZGlmaWVzIHRoZSBsb3dlciBiaXRzLiBUbyBiZSBtb3JlIHByZWNpc2UgdGhlDQo+ID4+IG9ubHkN
Cj4gPj4+Pj4gZ3V5DQo+ID4+Pj4+Pj4+IHRoYXQgcHJvdmlkZXMgYW5vdGhlciB0cmFuc2xhdGlv
biBsYXllciBpcyAiZHJhNy5kdHNpIjoNCj4gPj4+Pj4+Pj4gYXhpMA0KPiA+Pj4+Pj4+PiBodHRw
Oi8vbHhyLmZyZWUtDQo+ID4+Pj4+IGVsZWN0cm9ucy5jb20vc291cmNlL2FyY2gvYXJtL2Jvb3Qv
ZHRzL2RyYTcuZHRzaSNMMjA3DQo+ID4+Pj4+Pj4+DQo+ID4+Pj4+Pj4+IGF4aTENCj4gPj4+Pj4+
Pj4gaHR0cDovL2x4ci5mcmVlLQ0KPiA+Pj4+PiBlbGVjdHJvbnMuY29tL3NvdXJjZS9hcmNoL2Fy
bS9ib290L2R0cy9kcmE3LmR0c2kjTDI0MQ0KPiA+Pj4+Pj4+Pg0KPiA+Pj4+Pj4+PiBGb3IgdGhp
cyBjYXNlIG1hc2tpbmcgdGhlIHRvcCA0Yml0cyAoYml0czI4IHRvIDMxKSBzaG91bGQgbWFrZQ0K
PiA+PiB0aGUNCj4gPj4+Pj4gam9iLg0KPiA+Pj4+Pg0KPiA+Pj4+PiBJTU8sIHdlIHNob3VsZCBq
dXN0IGZpeCB0aGlzIGNhc2UuIEFmdGVyIGZ1cnRoZXIgc3R1ZHksIEkgZG9uJ3QNCj4gPj4gdGhp
bmsNCj4gPj4+Pj4gdGhpcyBpcyBhIERXIGlzc3VlLCBidXQgcmF0aGVyIGFuIFNPQyBpbnRlZ3Jh
dGlvbiBpc3N1ZS4NCj4gPj4+Pj4NCj4gPj4+Pj4gSSBiZWxpZXZlIHlvdSBjYW4ganVzdCBmaXh1
cCB0aGUgYWRkcmVzcyBpbiB0aGUgcHAtPm9wcy0NCj4gPmhvc3RfaW5pdA0KPiA+PiBob29rLg0K
PiA+Pj4+DQo+ID4+Pj4gWWVzIEkgZ3Vlc3MgdGhhdCBJIGNvdWxkIGp1c3QgYXNzaWduIHBwLT4o
KilfbW9kX2Jhc2UgdG8gdGhlIENQVQ0KPiA+PiBhZGRyZXNzDQo+ID4+Pj4gaW4gRFcgYW5kIG1h
c2sgaXQgb3V0IGluIGRyYTd4eF9wY2llX2hvc3RfaW5pdCgpLi4uDQo+ID4+Pj4NCj4gPj4+PiBL
aXNob24sIHdvdWxkIHlvdSBiZSBvayB3aXRoIHRoYXQ/DQo+ID4+Pg0KPiA+Pj4gSW5pdGlhbGx5
IEkgd2FzIHVzaW5nICpiYXNlLW1hc2sqIHByb3BlcnR5IGZyb20gZHQuIE1lIGFuZCBBcm5kDQo+
ID4+IChjYydlZCkgaGFkDQo+ID4+PiB0aGlzIGRpc2N1c3Npb24gWzFdIGJlZm9yZSB3ZSBkZWNp
ZGVkIHRoZSBjdXJyZW50IGFwcHJvYWNoLiBJdCdsbA0KPiBiZQ0KPiA+PiBnb29kIHRvDQo+ID4+
PiBjaGVjayB3aXRoIEFybmQgdG9vLg0KPiA+Pj4NCj4gPj4+IFsxXSAtPiAgaHR0cDovL2xpc3Rz
LmluZnJhZGVhZC5vcmcvcGlwZXJtYWlsL2xpbnV4LWFybS1rZXJuZWwvMjAxNC0NCj4gPj4gTWF5
LzI1MzUyOC5odG1sDQo+ID4+DQo+ID4+IFRoZSBwcm9ibGVtIEkgaGF2ZSBoZXJlIGlzIHRoZSB1
c2Ugb2YgcmFuZ2VzIGRvZXMgbm90IG5lY2Vzc2FyaWx5DQo+IG1lYW4NCj4gPj4gZmV3ZXIgYWRk
cmVzcyBiaXRzIGFyZSBhdmFpbGFibGUuIEl0IGNhbiBiZSB1c2VkIGp1c3QgZm9yDQo+IGNvbnZl
bmllbmNlDQo+ID4+IG9mIG5vdCBwdXR0aW5nIHRoZSBmdWxsIGFkZHJlc3MgaW50byBldmVyeSBu
b2RlJ3MgcmVnIHByb3BlcnR5LiBBbmQNCj4gPj4gdmljZSB2ZXJzYSwgdGhlcmUgYXJlIHByb2Jh
Ymx5IHBsZW50eSBvZiBjYXNlcyB3aGVyZSB3ZSBoYXZlIHRoZQ0KPiBmdWxsDQo+ID4+IGFkZHJl
c3MgaW4gdGhlIG5vZGVzLCBidXQgcmVhbGx5IG9ubHkgc29tZSBvZiB0aGUgYWRkcmVzcyBiaXRz
IGFyZQ0KPiA+PiBkZWNvZGVkIGF0IHRoZSBJUCBibG9jay4gV2hldGhlciB0aGUgYWRkcmVzcyBi
aXRzIGFyZSBwcmVzZW50IGlzDQo+ID4+IHJhcmVseSBjYXJlZCBhYm91dCBvciBrbm93biBieSBz
L3cgZm9sa3MgdW50aWwgeW91IGhpdCBhIHByb2JsZW0NCj4gbGlrZQ0KPiA+PiB0aGlzLiBHaXZl
biB0aGlzIGlzIGFuIGlzb2xhdGVkIGNhc2UgQVRNLCBJIHdvdWxkIGZpeCBpdCBpbiBhbg0KPiA+
PiBpc29sYXRlZCB3YXkuIEl0IGRvZXMgbm90IGFmZmVjdCB0aGUgYmluZGluZyBhbmQgY291bGQg
YmUgY2hhbmdlZCBpbg0KPiA+PiB0aGUga2VybmVsIGxhdGVyIGlmIHRoaXMgYmVjb21lcyBhIGNv
bW1vbiBuZWVkLg0KPiA+Pg0KPiA+PiBSb2INCj4gPj4gLS0NCj4gPj4gVG8gdW5zdWJzY3JpYmUg
ZnJvbSB0aGlzIGxpc3Q6IHNlbmQgdGhlIGxpbmUgInVuc3Vic2NyaWJlIGxpbnV4LXBjaSINCj4g
aW4NCj4gPj4gdGhlIGJvZHkgb2YgYSBtZXNzYWdlIHRvIG1ham9yZG9tb0B2Z2VyLmtlcm5lbC5v
cmcNCj4gPj4gTW9yZSBtYWpvcmRvbW8gaW5mbyBhdCAgaHR0cDovL3ZnZXIua2VybmVsLm9yZy9t
YWpvcmRvbW8taW5mby5odG1sDQo=

^ permalink raw reply	[flat|nested] 73+ messages in thread
* [PATCH v6] PCI: Store PCIe bus address in struct of_pci_range
@ 2015-08-04 10:12                                     ` Gabriele Paoloni
  0 siblings, 0 replies; 73+ messages in thread
From: Gabriele Paoloni @ 2015-08-04 10:12 UTC (permalink / raw)
  To: linux-arm-kernel



> -----Original Message-----
> From: Jingoo Han [mailto:jingoohan1 at gmail.com]
> Sent: Tuesday, August 04, 2015 5:20 AM
> To: Gabriele Paoloni
> Cc: Rob Herring; Kishon Vijay Abraham I; Bjorn Helgaas; arnd at arndb.de;
> lorenzo.pieralisi at arm.com; Wangzhou (B); robh+dt at kernel.org;
> james.morse at arm.com; Liviu.Dudau at arm.com; linux-pci at vger.kernel.org;
> linux-arm-kernel at lists.infradead.org; devicetree at vger.kernel.org;
> Yuanzhichang; Zhudacai; zhangjukuo; qiuzhenfa; Liguozhu (Kenneth);
> Pratyush Anand; Arnd Bergmann; jingoohan1 at gmail.com
> Subject: Re: [PATCH v6] PCI: Store PCIe bus address in struct
> of_pci_range
> 
> On 2015. 8. 3., at PM 8:18, Gabriele Paoloni
> <gabriele.paoloni@huawei.com> wrote:
> >
> > Rob, Kishon what about the following solution?....
> >
> > ---
> > drivers/pci/host/pci-dra7xx.c      | 12 ++++++++++++
> > drivers/pci/host/pcie-designware.c |  9 +++------
> 
> Hi Paoloni,
> 
> OK, it looks good.
> I want you to revert the following patch.
> 
> commit "f4c55c5a3f7f (PCI: designware: Program ATU with untranslated
> address)"
> 
> Would you remove all '*_mode_base's?

Hi Jingoo Han,

If I reverted the commit, in order to get designware working, dra7 
should mask directly the CPU addresses stored in pp->cfg0_base, 
pp->cfg1_base, pp->mem_base and pp->io_base.

The masking would happen at this point: 
http://lxr.free-electrons.com/source/drivers/pci/host/pcie-designware.c#L493

Now:
- I see that pp->cfg<0/1>_base are used in devm_ioremap before that 
  point and nowhere else, so they should be ok.
- pp->mem_base is used in dw_pcie_setup_rc(); dw_pcie_setup_rc() is called
  in dra7xx_pcie_host_init()...so here I should move the masking after
  dw_pcie_setup() retuned, but I think it should be ok
- pp->io_base is used in pci_ioremap_io() in dw_pcie_setup(). Now currently
  in designware this is called by pci_bios_init_hw(); this is after the 
  masking....so here we would have a the wrong value passed

Said that if you look at "[PATCH v5 2/5] PCI: designware: Add ARM64 support",
that is the patchset where this patch is needed, you can see that 
dw_pcie_setup() is removed and pp->io_base is used in pci_remap_iospace() before 
the masking would happen in dra7xx_pcie_host_init()...so for this patchset we 
should be good to revert the commit.

I propose to add a new patch in the patchset "[PATCH v5 0/5] PCI: hisi: Add PCIe 
host support for HiSilicon SoC Hip05" as the one I just posted in this 
thread (this would not revert the commit but just moves the masking to dra7xx)

I would revert the commit in "[PATCH v5 2/5] PCI: designware: Add ARM64 support"
together with the rest of designware rework. 

So we would have always a working version of designware...

Are you ok with this?

> 
> Best regards,
> Jingoo Han
> 
> > 2 files changed, 15 insertions(+), 6 deletions(-)
> >
> > diff --git a/drivers/pci/host/pci-dra7xx.c b/drivers/pci/host/pci-
> dra7xx.c
> > index 80db09e..bb2635f 100644
> > --- a/drivers/pci/host/pci-dra7xx.c
> > +++ b/drivers/pci/host/pci-dra7xx.c
> > @@ -61,6 +61,7 @@
> >
> > #define    PCIECTRL_DRA7XX_CONF_PHY_CS            0x010C
> > #define    LINK_UP                        BIT(16)
> > +#define CPU_TO_BUS_ADDR             0x0FFFFFFF
> >
> > struct dra7xx_pcie {
> >    void __iomem        *base;
> > @@ -138,6 +139,17 @@ static void dra7xx_pcie_enable_interrupts(struct
> pcie_port *pp)
> >
> > static void dra7xx_pcie_host_init(struct pcie_port *pp)
> > {
> > +    if (pp->io_mod_base)
> > +        pp->io_mod_base &= CPU_TO_BUS_ADDR;
> > +
> > +    if (pp->mem_mod_base)
> > +        pp->mem_mod_base &= CPU_TO_BUS_ADDR;
> > +
> > +    if (pp->cfg0_mod_base) {
> > +        pp->cfg0_mod_base &= CPU_TO_BUS_ADDR;
> > +        pp->cfg1_mod_base &= CPU_TO_BUS_ADDR;
> > +    }
> > +
> >    dw_pcie_setup_rc(pp);
> >    dra7xx_pcie_establish_link(pp);
> >    if (IS_ENABLED(CONFIG_PCI_MSI))
> > diff --git a/drivers/pci/host/pcie-designware.c
> b/drivers/pci/host/pcie-designware.c
> > index 69486be..06c682b 100644
> > --- a/drivers/pci/host/pcie-designware.c
> > +++ b/drivers/pci/host/pcie-designware.c
> > @@ -416,8 +416,7 @@ int dw_pcie_host_init(struct pcie_port *pp)
> >            pp->io_base = range.cpu_addr;
> >
> >            /* Find the untranslated IO space address */
> > -            pp->io_mod_base = of_read_number(parser.range -
> > -                             parser.np + na, ns);
> > +            pp->io_mod_base = range.cpu_addr;;
> >        }
> >        if (restype == IORESOURCE_MEM) {
> >            of_pci_range_to_resource(&range, np, &pp->mem);
> > @@ -426,8 +425,7 @@ int dw_pcie_host_init(struct pcie_port *pp)
> >            pp->mem_bus_addr = range.pci_addr;
> >
> >            /* Find the untranslated MEM space address */
> > -            pp->mem_mod_base = of_read_number(parser.range -
> > -                              parser.np + na, ns);
> > +            pp->mem_mod_base = range.cpu_addr;
> >        }
> >        if (restype == 0) {
> >            of_pci_range_to_resource(&range, np, &pp->cfg);
> > @@ -437,8 +435,7 @@ int dw_pcie_host_init(struct pcie_port *pp)
> >            pp->cfg1_base = pp->cfg.start + pp->cfg0_size;
> >
> >            /* Find the untranslated configuration space address */
> > -            pp->cfg0_mod_base = of_read_number(parser.range -
> > -                               parser.np + na, ns);
> > +            pp->cfg0_mod_base = range.cpu_addr;
> >            pp->cfg1_mod_base = pp->cfg0_mod_base +
> >                        pp->cfg0_size;
> >        }
> > --
> > 1.9.1
> >
> >
> >> -----Original Message-----
> >> From: linux-pci-owner at vger.kernel.org [mailto:linux-pci-
> >> owner at vger.kernel.org] On Behalf Of Rob Herring
> >> Sent: Friday, July 31, 2015 5:53 PM
> >> To: Kishon Vijay Abraham I
> >> Cc: Gabriele Paoloni; Bjorn Helgaas; arnd at arndb.de;
> >> lorenzo.pieralisi at arm.com; Wangzhou (B); robh+dt at kernel.org;
> >> james.morse at arm.com; Liviu.Dudau at arm.com; linux-pci at vger.kernel.org;
> >> linux-arm-kernel at lists.infradead.org; devicetree at vger.kernel.org;
> >> Yuanzhichang; Zhudacai; zhangjukuo; qiuzhenfa; Liguozhu (Kenneth);
> >> Jingoo Han; Pratyush Anand; Arnd Bergmann
> >> Subject: Re: [PATCH v6] PCI: Store PCIe bus address in struct
> >> of_pci_range
> >>
> >> On Fri, Jul 31, 2015 at 9:57 AM, Kishon Vijay Abraham I
> <kishon@ti.com>
> >> wrote:
> >>> +Arnd
> >>>
> >>> Hi,
> >>>
> >>>> On Friday 31 July 2015 07:55 PM, Gabriele Paoloni wrote:
> >>>> [+cc Kishon]
> >>>>
> >>>>> -----Original Message-----
> >>>>> From: linux-pci-owner at vger.kernel.org [mailto:linux-pci-
> >>>>> owner at vger.kernel.org] On Behalf Of Rob Herring
> >>>>> Sent: Thursday, July 30, 2015 9:42 PM
> >>>>> To: Gabriele Paoloni
> >>>>> Cc: Bjorn Helgaas; arnd at arndb.de; lorenzo.pieralisi at arm.com;
> >> Wangzhou
> >>>>> (B); robh+dt at kernel.org; james.morse at arm.com; Liviu.Dudau at arm.com;
> >>>>> linux-pci at vger.kernel.org; linux-arm-kernel at lists.infradead.org;
> >>>>> devicetree at vger.kernel.org; Yuanzhichang; Zhudacai; zhangjukuo;
> >>>>> qiuzhenfa; Liguozhu (Kenneth); Jingoo Han; Pratyush Anand
> >>>>> Subject: Re: [PATCH v6] PCI: Store PCIe bus address in struct
> >>>>> of_pci_range
> >>>>>
> >>>>> On Thu, Jul 30, 2015 at 12:34 PM, Gabriele Paoloni
> >>>>> <gabriele.paoloni@huawei.com> wrote:
> >>>>>>> -----Original Message-----
> >>>>>>> From: Bjorn Helgaas [mailto:bhelgaas at google.com]
> >>>>>>> Sent: 30 July 2015 18:15
> >>>>>>> On Thu, Jul 30, 2015 at 04:50:55PM +0000, Gabriele Paoloni
> wrote:
> >>>>>>>>> -----Original Message-----
> >>>>>>>>> From: linux-pci-owner at vger.kernel.org [mailto:linux-pci-
> >>>>>>>>> owner at vger.kernel.org] On Behalf Of Bjorn Helgaas
> >>>>>>>>> Sent: Thursday, July 30, 2015 5:15 PM
> >>>>>>>>> On Thu, Jul 30, 2015 at 01:52:13PM +0000, Gabriele Paoloni
> >> wrote:
> >>>>>
> >>>>> [...]
> >>>>>
> >>>>>>>>>> I don?t think we should rely on [CPU] addresses...what if
> the
> >>>>>>>>> intermediate
> >>>>>>>>>> translation layer changes the lower significant bits of the
> >>>>> "bus
> >>>>>>>>> address"
> >>>>>>>>>> to translate into a cpu address?
> >>>>>>>>>
> >>>>>>>>> Is it really a possiblity that the lower bits could be
> changed?
> >>>>>>>>
> >>>>>>>> I've checked all the current deignware users DTs except "pci-
> >>>>>>> layerscape"
> >>>>>>>> that I could not find:
> >>>>>>>> spear1310.dtsi
> >>>>>>>> spear1340.dtsi
> >>>>>>>> dra7.dtsi
> >>>>>>>> imx6qdl.dtsi
> >>>>>>>> imx6sx.dtsi
> >>>>>>>> keystone.dtsi
> >>>>>>>> exynos5440.dtsi
> >>>>>>>>
> >>>>>>>> None of them modifies the lower bits. To be more precise the
> >> only
> >>>>> guy
> >>>>>>>> that provides another translation layer is "dra7.dtsi":
> >>>>>>>> axi0
> >>>>>>>> http://lxr.free-
> >>>>> electrons.com/source/arch/arm/boot/dts/dra7.dtsi#L207
> >>>>>>>>
> >>>>>>>> axi1
> >>>>>>>> http://lxr.free-
> >>>>> electrons.com/source/arch/arm/boot/dts/dra7.dtsi#L241
> >>>>>>>>
> >>>>>>>> For this case masking the top 4bits (bits28 to 31) should make
> >> the
> >>>>> job.
> >>>>>
> >>>>> IMO, we should just fix this case. After further study, I don't
> >> think
> >>>>> this is a DW issue, but rather an SOC integration issue.
> >>>>>
> >>>>> I believe you can just fixup the address in the pp->ops-
> >host_init
> >> hook.
> >>>>
> >>>> Yes I guess that I could just assign pp->(*)_mod_base to the CPU
> >> address
> >>>> in DW and mask it out in dra7xx_pcie_host_init()...
> >>>>
> >>>> Kishon, would you be ok with that?
> >>>
> >>> Initially I was using *base-mask* property from dt. Me and Arnd
> >> (cc'ed) had
> >>> this discussion [1] before we decided the current approach. It'll
> be
> >> good to
> >>> check with Arnd too.
> >>>
> >>> [1] ->  http://lists.infradead.org/pipermail/linux-arm-kernel/2014-
> >> May/253528.html
> >>
> >> The problem I have here is the use of ranges does not necessarily
> mean
> >> fewer address bits are available. It can be used just for
> convenience
> >> of not putting the full address into every node's reg property. And
> >> vice versa, there are probably plenty of cases where we have the
> full
> >> address in the nodes, but really only some of the address bits are
> >> decoded at the IP block. Whether the address bits are present is
> >> rarely cared about or known by s/w folks until you hit a problem
> like
> >> this. Given this is an isolated case ATM, I would fix it in an
> >> isolated way. It does not affect the binding and could be changed in
> >> the kernel later if this becomes a common need.
> >>
> >> Rob
> >> --
> >> To unsubscribe from this list: send the line "unsubscribe linux-pci"
> in
> >> the body of a message to majordomo at vger.kernel.org
> >> More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 73+ messages in thread
* RE: [PATCH v6] PCI: Store PCIe bus address in struct of_pci_range
  2015-08-04 10:12                                     ` Gabriele Paoloni
  (?)
@ 2015-08-06 13:52                                       ` Gabriele Paoloni
  -1 siblings, 0 replies; 73+ messages in thread
From: Gabriele Paoloni @ 2015-08-06 13:52 UTC (permalink / raw)
  To: Gabriele Paoloni, Jingoo Han
  Cc: Rob Herring, Kishon Vijay Abraham I, Bjorn Helgaas, arnd,
	lorenzo.pieralisi, Wangzhou (B),
	robh+dt, james.morse, Liviu.Dudau, linux-pci, linux-arm-kernel,
	devicetree, Yuanzhichang, Zhudacai, zhangjukuo, qiuzhenfa,
	Liguozhu (Kenneth),
	Pratyush Anand, Arnd Bergmann

Hi all

This patch has now been moved in "[PATCH v6 1/6] PCI: designware: move calculation of bus addresses to DRA7xx"

commit "f4c55c5a3f7f (PCI: designware: Program ATU with untranslated address)" has been reverted in
"[PATCH v6 3/6] PCI: designware: Add ARM64 support"

Gab

> -----Original Message-----
> From: linux-pci-owner@vger.kernel.org [mailto:linux-pci-
> owner@vger.kernel.org] On Behalf Of Gabriele Paoloni
> Sent: Tuesday, August 04, 2015 11:12 AM
> To: Jingoo Han
> Cc: Rob Herring; Kishon Vijay Abraham I; Bjorn Helgaas; arnd@arndb.de;
> lorenzo.pieralisi@arm.com; Wangzhou (B); robh+dt@kernel.org;
> james.morse@arm.com; Liviu.Dudau@arm.com; linux-pci@vger.kernel.org;
> linux-arm-kernel@lists.infradead.org; devicetree@vger.kernel.org;
> Yuanzhichang; Zhudacai; zhangjukuo; qiuzhenfa; Liguozhu (Kenneth);
> Pratyush Anand; Arnd Bergmann
> Subject: RE: [PATCH v6] PCI: Store PCIe bus address in struct
> of_pci_range
> 
> 
> 
> > -----Original Message-----
> > From: Jingoo Han [mailto:jingoohan1@gmail.com]
> > Sent: Tuesday, August 04, 2015 5:20 AM
> > To: Gabriele Paoloni
> > Cc: Rob Herring; Kishon Vijay Abraham I; Bjorn Helgaas; arnd@arndb.de;
> > lorenzo.pieralisi@arm.com; Wangzhou (B); robh+dt@kernel.org;
> > james.morse@arm.com; Liviu.Dudau@arm.com; linux-pci@vger.kernel.org;
> > linux-arm-kernel@lists.infradead.org; devicetree@vger.kernel.org;
> > Yuanzhichang; Zhudacai; zhangjukuo; qiuzhenfa; Liguozhu (Kenneth);
> > Pratyush Anand; Arnd Bergmann; jingoohan1@gmail.com
> > Subject: Re: [PATCH v6] PCI: Store PCIe bus address in struct
> > of_pci_range
> >
> > On 2015. 8. 3., at PM 8:18, Gabriele Paoloni
> > <gabriele.paoloni@huawei.com> wrote:
> > >
> > > Rob, Kishon what about the following solution?....
> > >
> > > ---
> > > drivers/pci/host/pci-dra7xx.c      | 12 ++++++++++++
> > > drivers/pci/host/pcie-designware.c |  9 +++------
> >
> > Hi Paoloni,
> >
> > OK, it looks good.
> > I want you to revert the following patch.
> >
> > commit "f4c55c5a3f7f (PCI: designware: Program ATU with untranslated
> > address)"
> >
> > Would you remove all '*_mode_base's?
> 
> Hi Jingoo Han,
> 
> If I reverted the commit, in order to get designware working, dra7
> should mask directly the CPU addresses stored in pp->cfg0_base,
> pp->cfg1_base, pp->mem_base and pp->io_base.
> 
> The masking would happen at this point:
> http://lxr.free-electrons.com/source/drivers/pci/host/pcie-
> designware.c#L493
> 
> Now:
> - I see that pp->cfg<0/1>_base are used in devm_ioremap before that
>   point and nowhere else, so they should be ok.
> - pp->mem_base is used in dw_pcie_setup_rc(); dw_pcie_setup_rc() is
> called
>   in dra7xx_pcie_host_init()...so here I should move the masking after
>   dw_pcie_setup() retuned, but I think it should be ok
> - pp->io_base is used in pci_ioremap_io() in dw_pcie_setup(). Now
> currently
>   in designware this is called by pci_bios_init_hw(); this is after the
>   masking....so here we would have a the wrong value passed
> 
> Said that if you look at "[PATCH v5 2/5] PCI: designware: Add ARM64
> support",
> that is the patchset where this patch is needed, you can see that
> dw_pcie_setup() is removed and pp->io_base is used in
> pci_remap_iospace() before
> the masking would happen in dra7xx_pcie_host_init()...so for this
> patchset we
> should be good to revert the commit.
> 
> I propose to add a new patch in the patchset "[PATCH v5 0/5] PCI: hisi:
> Add PCIe
> host support for HiSilicon SoC Hip05" as the one I just posted in this
> thread (this would not revert the commit but just moves the masking to
> dra7xx)
> 
> I would revert the commit in "[PATCH v5 2/5] PCI: designware: Add ARM64
> support"
> together with the rest of designware rework.
> 
> So we would have always a working version of designware...
> 
> Are you ok with this?
> 
> >
> > Best regards,
> > Jingoo Han
> >
> > > 2 files changed, 15 insertions(+), 6 deletions(-)
> > >
> > > diff --git a/drivers/pci/host/pci-dra7xx.c b/drivers/pci/host/pci-
> > dra7xx.c
> > > index 80db09e..bb2635f 100644
> > > --- a/drivers/pci/host/pci-dra7xx.c
> > > +++ b/drivers/pci/host/pci-dra7xx.c
> > > @@ -61,6 +61,7 @@
> > >
> > > #define    PCIECTRL_DRA7XX_CONF_PHY_CS            0x010C
> > > #define    LINK_UP                        BIT(16)
> > > +#define CPU_TO_BUS_ADDR             0x0FFFFFFF
> > >
> > > struct dra7xx_pcie {
> > >    void __iomem        *base;
> > > @@ -138,6 +139,17 @@ static void
> dra7xx_pcie_enable_interrupts(struct
> > pcie_port *pp)
> > >
> > > static void dra7xx_pcie_host_init(struct pcie_port *pp)
> > > {
> > > +    if (pp->io_mod_base)
> > > +        pp->io_mod_base &= CPU_TO_BUS_ADDR;
> > > +
> > > +    if (pp->mem_mod_base)
> > > +        pp->mem_mod_base &= CPU_TO_BUS_ADDR;
> > > +
> > > +    if (pp->cfg0_mod_base) {
> > > +        pp->cfg0_mod_base &= CPU_TO_BUS_ADDR;
> > > +        pp->cfg1_mod_base &= CPU_TO_BUS_ADDR;
> > > +    }
> > > +
> > >    dw_pcie_setup_rc(pp);
> > >    dra7xx_pcie_establish_link(pp);
> > >    if (IS_ENABLED(CONFIG_PCI_MSI))
> > > diff --git a/drivers/pci/host/pcie-designware.c
> > b/drivers/pci/host/pcie-designware.c
> > > index 69486be..06c682b 100644
> > > --- a/drivers/pci/host/pcie-designware.c
> > > +++ b/drivers/pci/host/pcie-designware.c
> > > @@ -416,8 +416,7 @@ int dw_pcie_host_init(struct pcie_port *pp)
> > >            pp->io_base = range.cpu_addr;
> > >
> > >            /* Find the untranslated IO space address */
> > > -            pp->io_mod_base = of_read_number(parser.range -
> > > -                             parser.np + na, ns);
> > > +            pp->io_mod_base = range.cpu_addr;;
> > >        }
> > >        if (restype == IORESOURCE_MEM) {
> > >            of_pci_range_to_resource(&range, np, &pp->mem);
> > > @@ -426,8 +425,7 @@ int dw_pcie_host_init(struct pcie_port *pp)
> > >            pp->mem_bus_addr = range.pci_addr;
> > >
> > >            /* Find the untranslated MEM space address */
> > > -            pp->mem_mod_base = of_read_number(parser.range -
> > > -                              parser.np + na, ns);
> > > +            pp->mem_mod_base = range.cpu_addr;
> > >        }
> > >        if (restype == 0) {
> > >            of_pci_range_to_resource(&range, np, &pp->cfg);
> > > @@ -437,8 +435,7 @@ int dw_pcie_host_init(struct pcie_port *pp)
> > >            pp->cfg1_base = pp->cfg.start + pp->cfg0_size;
> > >
> > >            /* Find the untranslated configuration space address */
> > > -            pp->cfg0_mod_base = of_read_number(parser.range -
> > > -                               parser.np + na, ns);
> > > +            pp->cfg0_mod_base = range.cpu_addr;
> > >            pp->cfg1_mod_base = pp->cfg0_mod_base +
> > >                        pp->cfg0_size;
> > >        }
> > > --
> > > 1.9.1
> > >
> > >
> > >> -----Original Message-----
> > >> From: linux-pci-owner@vger.kernel.org [mailto:linux-pci-
> > >> owner@vger.kernel.org] On Behalf Of Rob Herring
> > >> Sent: Friday, July 31, 2015 5:53 PM
> > >> To: Kishon Vijay Abraham I
> > >> Cc: Gabriele Paoloni; Bjorn Helgaas; arnd@arndb.de;
> > >> lorenzo.pieralisi@arm.com; Wangzhou (B); robh+dt@kernel.org;
> > >> james.morse@arm.com; Liviu.Dudau@arm.com; linux-
> pci@vger.kernel.org;
> > >> linux-arm-kernel@lists.infradead.org; devicetree@vger.kernel.org;
> > >> Yuanzhichang; Zhudacai; zhangjukuo; qiuzhenfa; Liguozhu (Kenneth);
> > >> Jingoo Han; Pratyush Anand; Arnd Bergmann
> > >> Subject: Re: [PATCH v6] PCI: Store PCIe bus address in struct
> > >> of_pci_range
> > >>
> > >> On Fri, Jul 31, 2015 at 9:57 AM, Kishon Vijay Abraham I
> > <kishon@ti.com>
> > >> wrote:
> > >>> +Arnd
> > >>>
> > >>> Hi,
> > >>>
> > >>>> On Friday 31 July 2015 07:55 PM, Gabriele Paoloni wrote:
> > >>>> [+cc Kishon]
> > >>>>
> > >>>>> -----Original Message-----
> > >>>>> From: linux-pci-owner@vger.kernel.org [mailto:linux-pci-
> > >>>>> owner@vger.kernel.org] On Behalf Of Rob Herring
> > >>>>> Sent: Thursday, July 30, 2015 9:42 PM
> > >>>>> To: Gabriele Paoloni
> > >>>>> Cc: Bjorn Helgaas; arnd@arndb.de; lorenzo.pieralisi@arm.com;
> > >> Wangzhou
> > >>>>> (B); robh+dt@kernel.org; james.morse@arm.com;
> Liviu.Dudau@arm.com;
> > >>>>> linux-pci@vger.kernel.org; linux-arm-kernel@lists.infradead.org;
> > >>>>> devicetree@vger.kernel.org; Yuanzhichang; Zhudacai; zhangjukuo;
> > >>>>> qiuzhenfa; Liguozhu (Kenneth); Jingoo Han; Pratyush Anand
> > >>>>> Subject: Re: [PATCH v6] PCI: Store PCIe bus address in struct
> > >>>>> of_pci_range
> > >>>>>
> > >>>>> On Thu, Jul 30, 2015 at 12:34 PM, Gabriele Paoloni
> > >>>>> <gabriele.paoloni@huawei.com> wrote:
> > >>>>>>> -----Original Message-----
> > >>>>>>> From: Bjorn Helgaas [mailto:bhelgaas@google.com]
> > >>>>>>> Sent: 30 July 2015 18:15
> > >>>>>>> On Thu, Jul 30, 2015 at 04:50:55PM +0000, Gabriele Paoloni
> > wrote:
> > >>>>>>>>> -----Original Message-----
> > >>>>>>>>> From: linux-pci-owner@vger.kernel.org [mailto:linux-pci-
> > >>>>>>>>> owner@vger.kernel.org] On Behalf Of Bjorn Helgaas
> > >>>>>>>>> Sent: Thursday, July 30, 2015 5:15 PM
> > >>>>>>>>> On Thu, Jul 30, 2015 at 01:52:13PM +0000, Gabriele Paoloni
> > >> wrote:
> > >>>>>
> > >>>>> [...]
> > >>>>>
> > >>>>>>>>>> I don’t think we should rely on [CPU] addresses...what if
> > the
> > >>>>>>>>> intermediate
> > >>>>>>>>>> translation layer changes the lower significant bits of
> the
> > >>>>> "bus
> > >>>>>>>>> address"
> > >>>>>>>>>> to translate into a cpu address?
> > >>>>>>>>>
> > >>>>>>>>> Is it really a possiblity that the lower bits could be
> > changed?
> > >>>>>>>>
> > >>>>>>>> I've checked all the current deignware users DTs except
> "pci-
> > >>>>>>> layerscape"
> > >>>>>>>> that I could not find:
> > >>>>>>>> spear1310.dtsi
> > >>>>>>>> spear1340.dtsi
> > >>>>>>>> dra7.dtsi
> > >>>>>>>> imx6qdl.dtsi
> > >>>>>>>> imx6sx.dtsi
> > >>>>>>>> keystone.dtsi
> > >>>>>>>> exynos5440.dtsi
> > >>>>>>>>
> > >>>>>>>> None of them modifies the lower bits. To be more precise the
> > >> only
> > >>>>> guy
> > >>>>>>>> that provides another translation layer is "dra7.dtsi":
> > >>>>>>>> axi0
> > >>>>>>>> http://lxr.free-
> > >>>>> electrons.com/source/arch/arm/boot/dts/dra7.dtsi#L207
> > >>>>>>>>
> > >>>>>>>> axi1
> > >>>>>>>> http://lxr.free-
> > >>>>> electrons.com/source/arch/arm/boot/dts/dra7.dtsi#L241
> > >>>>>>>>
> > >>>>>>>> For this case masking the top 4bits (bits28 to 31) should
> make
> > >> the
> > >>>>> job.
> > >>>>>
> > >>>>> IMO, we should just fix this case. After further study, I don't
> > >> think
> > >>>>> this is a DW issue, but rather an SOC integration issue.
> > >>>>>
> > >>>>> I believe you can just fixup the address in the pp->ops-
> > >host_init
> > >> hook.
> > >>>>
> > >>>> Yes I guess that I could just assign pp->(*)_mod_base to the CPU
> > >> address
> > >>>> in DW and mask it out in dra7xx_pcie_host_init()...
> > >>>>
> > >>>> Kishon, would you be ok with that?
> > >>>
> > >>> Initially I was using *base-mask* property from dt. Me and Arnd
> > >> (cc'ed) had
> > >>> this discussion [1] before we decided the current approach. It'll
> > be
> > >> good to
> > >>> check with Arnd too.
> > >>>
> > >>> [1] ->  http://lists.infradead.org/pipermail/linux-arm-
> kernel/2014-
> > >> May/253528.html
> > >>
> > >> The problem I have here is the use of ranges does not necessarily
> > mean
> > >> fewer address bits are available. It can be used just for
> > convenience
> > >> of not putting the full address into every node's reg property.
> And
> > >> vice versa, there are probably plenty of cases where we have the
> > full
> > >> address in the nodes, but really only some of the address bits are
> > >> decoded at the IP block. Whether the address bits are present is
> > >> rarely cared about or known by s/w folks until you hit a problem
> > like
> > >> this. Given this is an isolated case ATM, I would fix it in an
> > >> isolated way. It does not affect the binding and could be changed
> in
> > >> the kernel later if this becomes a common need.
> > >>
> > >> Rob
> > >> --
> > >> To unsubscribe from this list: send the line "unsubscribe linux-
> pci"
> > in
> > >> the body of a message to majordomo@vger.kernel.org
> > >> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> \x13?移?????쵔+-깁負\x17쪐w츩?궝??욎ir(㎍\x17썳變}찠꼿쟺?j:+v돣?쳭喩zZ+€?zf＂
> 톒쉱?넮녬i鎬z?췿ⅱ?솳鈺??刪^[f

^ permalink raw reply	[flat|nested] 73+ messages in thread
* RE: [PATCH v6] PCI: Store PCIe bus address in struct of_pci_range
@ 2015-08-06 13:52                                       ` Gabriele Paoloni
  0 siblings, 0 replies; 73+ messages in thread
From: Gabriele Paoloni @ 2015-08-06 13:52 UTC (permalink / raw)
  To: Gabriele Paoloni, Jingoo Han
  Cc: Rob Herring, Kishon Vijay Abraham I, Bjorn Helgaas, arnd,
	lorenzo.pieralisi, Wangzhou (B),
	robh+dt, james.morse, Liviu.Dudau, linux-pci, linux-arm-kernel,
	devicetree, Yuanzhichang, Zhudacai, zhangjukuo, qiuzhenfa,
	Liguozhu (Kenneth),
	Pratyush Anand, Arnd Bergmann

SGkgYWxsDQoNClRoaXMgcGF0Y2ggaGFzIG5vdyBiZWVuIG1vdmVkIGluICJbUEFUQ0ggdjYgMS82
XSBQQ0k6IGRlc2lnbndhcmU6IG1vdmUgY2FsY3VsYXRpb24gb2YgYnVzIGFkZHJlc3NlcyB0byBE
UkE3eHgiDQoNCmNvbW1pdCAiZjRjNTVjNWEzZjdmIChQQ0k6IGRlc2lnbndhcmU6IFByb2dyYW0g
QVRVIHdpdGggdW50cmFuc2xhdGVkIGFkZHJlc3MpIiBoYXMgYmVlbiByZXZlcnRlZCBpbg0KIltQ
QVRDSCB2NiAzLzZdIFBDSTogZGVzaWdud2FyZTogQWRkIEFSTTY0IHN1cHBvcnQiDQoNCkdhYg0K
DQo+IC0tLS0tT3JpZ2luYWwgTWVzc2FnZS0tLS0tDQo+IEZyb206IGxpbnV4LXBjaS1vd25lckB2
Z2VyLmtlcm5lbC5vcmcgW21haWx0bzpsaW51eC1wY2ktDQo+IG93bmVyQHZnZXIua2VybmVsLm9y
Z10gT24gQmVoYWxmIE9mIEdhYnJpZWxlIFBhb2xvbmkNCj4gU2VudDogVHVlc2RheSwgQXVndXN0
IDA0LCAyMDE1IDExOjEyIEFNDQo+IFRvOiBKaW5nb28gSGFuDQo+IENjOiBSb2IgSGVycmluZzsg
S2lzaG9uIFZpamF5IEFicmFoYW0gSTsgQmpvcm4gSGVsZ2FhczsgYXJuZEBhcm5kYi5kZTsNCj4g
bG9yZW56by5waWVyYWxpc2lAYXJtLmNvbTsgV2FuZ3pob3UgKEIpOyByb2JoK2R0QGtlcm5lbC5v
cmc7DQo+IGphbWVzLm1vcnNlQGFybS5jb207IExpdml1LkR1ZGF1QGFybS5jb207IGxpbnV4LXBj
aUB2Z2VyLmtlcm5lbC5vcmc7DQo+IGxpbnV4LWFybS1rZXJuZWxAbGlzdHMuaW5mcmFkZWFkLm9y
ZzsgZGV2aWNldHJlZUB2Z2VyLmtlcm5lbC5vcmc7DQo+IFl1YW56aGljaGFuZzsgWmh1ZGFjYWk7
IHpoYW5nanVrdW87IHFpdXpoZW5mYTsgTGlndW96aHUgKEtlbm5ldGgpOw0KPiBQcmF0eXVzaCBB
bmFuZDsgQXJuZCBCZXJnbWFubg0KPiBTdWJqZWN0OiBSRTogW1BBVENIIHY2XSBQQ0k6IFN0b3Jl
IFBDSWUgYnVzIGFkZHJlc3MgaW4gc3RydWN0DQo+IG9mX3BjaV9yYW5nZQ0KPiANCj4gDQo+IA0K
PiA+IC0tLS0tT3JpZ2luYWwgTWVzc2FnZS0tLS0tDQo+ID4gRnJvbTogSmluZ29vIEhhbiBbbWFp
bHRvOmppbmdvb2hhbjFAZ21haWwuY29tXQ0KPiA+IFNlbnQ6IFR1ZXNkYXksIEF1Z3VzdCAwNCwg
MjAxNSA1OjIwIEFNDQo+ID4gVG86IEdhYnJpZWxlIFBhb2xvbmkNCj4gPiBDYzogUm9iIEhlcnJp
bmc7IEtpc2hvbiBWaWpheSBBYnJhaGFtIEk7IEJqb3JuIEhlbGdhYXM7IGFybmRAYXJuZGIuZGU7
DQo+ID4gbG9yZW56by5waWVyYWxpc2lAYXJtLmNvbTsgV2FuZ3pob3UgKEIpOyByb2JoK2R0QGtl
cm5lbC5vcmc7DQo+ID4gamFtZXMubW9yc2VAYXJtLmNvbTsgTGl2aXUuRHVkYXVAYXJtLmNvbTsg
bGludXgtcGNpQHZnZXIua2VybmVsLm9yZzsNCj4gPiBsaW51eC1hcm0ta2VybmVsQGxpc3RzLmlu
ZnJhZGVhZC5vcmc7IGRldmljZXRyZWVAdmdlci5rZXJuZWwub3JnOw0KPiA+IFl1YW56aGljaGFu
ZzsgWmh1ZGFjYWk7IHpoYW5nanVrdW87IHFpdXpoZW5mYTsgTGlndW96aHUgKEtlbm5ldGgpOw0K
PiA+IFByYXR5dXNoIEFuYW5kOyBBcm5kIEJlcmdtYW5uOyBqaW5nb29oYW4xQGdtYWlsLmNvbQ0K
PiA+IFN1YmplY3Q6IFJlOiBbUEFUQ0ggdjZdIFBDSTogU3RvcmUgUENJZSBidXMgYWRkcmVzcyBp
biBzdHJ1Y3QNCj4gPiBvZl9wY2lfcmFuZ2UNCj4gPg0KPiA+IE9uIDIwMTUuIDguIDMuLCBhdCBQ
TSA4OjE4LCBHYWJyaWVsZSBQYW9sb25pDQo+ID4gPGdhYnJpZWxlLnBhb2xvbmlAaHVhd2VpLmNv
bT4gd3JvdGU6DQo+ID4gPg0KPiA+ID4gUm9iLCBLaXNob24gd2hhdCBhYm91dCB0aGUgZm9sbG93
aW5nIHNvbHV0aW9uPy4uLi4NCj4gPiA+DQo+ID4gPiAtLS0NCj4gPiA+IGRyaXZlcnMvcGNpL2hv
c3QvcGNpLWRyYTd4eC5jICAgICAgfCAxMiArKysrKysrKysrKysNCj4gPiA+IGRyaXZlcnMvcGNp
L2hvc3QvcGNpZS1kZXNpZ253YXJlLmMgfCAgOSArKystLS0tLS0NCj4gPg0KPiA+IEhpIFBhb2xv
bmksDQo+ID4NCj4gPiBPSywgaXQgbG9va3MgZ29vZC4NCj4gPiBJIHdhbnQgeW91IHRvIHJldmVy
dCB0aGUgZm9sbG93aW5nIHBhdGNoLg0KPiA+DQo+ID4gY29tbWl0ICJmNGM1NWM1YTNmN2YgKFBD
STogZGVzaWdud2FyZTogUHJvZ3JhbSBBVFUgd2l0aCB1bnRyYW5zbGF0ZWQNCj4gPiBhZGRyZXNz
KSINCj4gPg0KPiA+IFdvdWxkIHlvdSByZW1vdmUgYWxsICcqX21vZGVfYmFzZSdzPw0KPiANCj4g
SGkgSmluZ29vIEhhbiwNCj4gDQo+IElmIEkgcmV2ZXJ0ZWQgdGhlIGNvbW1pdCwgaW4gb3JkZXIg
dG8gZ2V0IGRlc2lnbndhcmUgd29ya2luZywgZHJhNw0KPiBzaG91bGQgbWFzayBkaXJlY3RseSB0
aGUgQ1BVIGFkZHJlc3NlcyBzdG9yZWQgaW4gcHAtPmNmZzBfYmFzZSwNCj4gcHAtPmNmZzFfYmFz
ZSwgcHAtPm1lbV9iYXNlIGFuZCBwcC0+aW9fYmFzZS4NCj4gDQo+IFRoZSBtYXNraW5nIHdvdWxk
IGhhcHBlbiBhdCB0aGlzIHBvaW50Og0KPiBodHRwOi8vbHhyLmZyZWUtZWxlY3Ryb25zLmNvbS9z
b3VyY2UvZHJpdmVycy9wY2kvaG9zdC9wY2llLQ0KPiBkZXNpZ253YXJlLmMjTDQ5Mw0KPiANCj4g
Tm93Og0KPiAtIEkgc2VlIHRoYXQgcHAtPmNmZzwwLzE+X2Jhc2UgYXJlIHVzZWQgaW4gZGV2bV9p
b3JlbWFwIGJlZm9yZSB0aGF0DQo+ICAgcG9pbnQgYW5kIG5vd2hlcmUgZWxzZSwgc28gdGhleSBz
aG91bGQgYmUgb2suDQo+IC0gcHAtPm1lbV9iYXNlIGlzIHVzZWQgaW4gZHdfcGNpZV9zZXR1cF9y
YygpOyBkd19wY2llX3NldHVwX3JjKCkgaXMNCj4gY2FsbGVkDQo+ICAgaW4gZHJhN3h4X3BjaWVf
aG9zdF9pbml0KCkuLi5zbyBoZXJlIEkgc2hvdWxkIG1vdmUgdGhlIG1hc2tpbmcgYWZ0ZXINCj4g
ICBkd19wY2llX3NldHVwKCkgcmV0dW5lZCwgYnV0IEkgdGhpbmsgaXQgc2hvdWxkIGJlIG9rDQo+
IC0gcHAtPmlvX2Jhc2UgaXMgdXNlZCBpbiBwY2lfaW9yZW1hcF9pbygpIGluIGR3X3BjaWVfc2V0
dXAoKS4gTm93DQo+IGN1cnJlbnRseQ0KPiAgIGluIGRlc2lnbndhcmUgdGhpcyBpcyBjYWxsZWQg
YnkgcGNpX2Jpb3NfaW5pdF9odygpOyB0aGlzIGlzIGFmdGVyIHRoZQ0KPiAgIG1hc2tpbmcuLi4u
c28gaGVyZSB3ZSB3b3VsZCBoYXZlIGEgdGhlIHdyb25nIHZhbHVlIHBhc3NlZA0KPiANCj4gU2Fp
ZCB0aGF0IGlmIHlvdSBsb29rIGF0ICJbUEFUQ0ggdjUgMi81XSBQQ0k6IGRlc2lnbndhcmU6IEFk
ZCBBUk02NA0KPiBzdXBwb3J0IiwNCj4gdGhhdCBpcyB0aGUgcGF0Y2hzZXQgd2hlcmUgdGhpcyBw
YXRjaCBpcyBuZWVkZWQsIHlvdSBjYW4gc2VlIHRoYXQNCj4gZHdfcGNpZV9zZXR1cCgpIGlzIHJl
bW92ZWQgYW5kIHBwLT5pb19iYXNlIGlzIHVzZWQgaW4NCj4gcGNpX3JlbWFwX2lvc3BhY2UoKSBi
ZWZvcmUNCj4gdGhlIG1hc2tpbmcgd291bGQgaGFwcGVuIGluIGRyYTd4eF9wY2llX2hvc3RfaW5p
dCgpLi4uc28gZm9yIHRoaXMNCj4gcGF0Y2hzZXQgd2UNCj4gc2hvdWxkIGJlIGdvb2QgdG8gcmV2
ZXJ0IHRoZSBjb21taXQuDQo+IA0KPiBJIHByb3Bvc2UgdG8gYWRkIGEgbmV3IHBhdGNoIGluIHRo
ZSBwYXRjaHNldCAiW1BBVENIIHY1IDAvNV0gUENJOiBoaXNpOg0KPiBBZGQgUENJZQ0KPiBob3N0
IHN1cHBvcnQgZm9yIEhpU2lsaWNvbiBTb0MgSGlwMDUiIGFzIHRoZSBvbmUgSSBqdXN0IHBvc3Rl
ZCBpbiB0aGlzDQo+IHRocmVhZCAodGhpcyB3b3VsZCBub3QgcmV2ZXJ0IHRoZSBjb21taXQgYnV0
IGp1c3QgbW92ZXMgdGhlIG1hc2tpbmcgdG8NCj4gZHJhN3h4KQ0KPiANCj4gSSB3b3VsZCByZXZl
cnQgdGhlIGNvbW1pdCBpbiAiW1BBVENIIHY1IDIvNV0gUENJOiBkZXNpZ253YXJlOiBBZGQgQVJN
NjQNCj4gc3VwcG9ydCINCj4gdG9nZXRoZXIgd2l0aCB0aGUgcmVzdCBvZiBkZXNpZ253YXJlIHJl
d29yay4NCj4gDQo+IFNvIHdlIHdvdWxkIGhhdmUgYWx3YXlzIGEgd29ya2luZyB2ZXJzaW9uIG9m
IGRlc2lnbndhcmUuLi4NCj4gDQo+IEFyZSB5b3Ugb2sgd2l0aCB0aGlzPw0KPiANCj4gPg0KPiA+
IEJlc3QgcmVnYXJkcywNCj4gPiBKaW5nb28gSGFuDQo+ID4NCj4gPiA+IDIgZmlsZXMgY2hhbmdl
ZCwgMTUgaW5zZXJ0aW9ucygrKSwgNiBkZWxldGlvbnMoLSkNCj4gPiA+DQo+ID4gPiBkaWZmIC0t
Z2l0IGEvZHJpdmVycy9wY2kvaG9zdC9wY2ktZHJhN3h4LmMgYi9kcml2ZXJzL3BjaS9ob3N0L3Bj
aS0NCj4gPiBkcmE3eHguYw0KPiA+ID4gaW5kZXggODBkYjA5ZS4uYmIyNjM1ZiAxMDA2NDQNCj4g
PiA+IC0tLSBhL2RyaXZlcnMvcGNpL2hvc3QvcGNpLWRyYTd4eC5jDQo+ID4gPiArKysgYi9kcml2
ZXJzL3BjaS9ob3N0L3BjaS1kcmE3eHguYw0KPiA+ID4gQEAgLTYxLDYgKzYxLDcgQEANCj4gPiA+
DQo+ID4gPiAjZGVmaW5lICAgIFBDSUVDVFJMX0RSQTdYWF9DT05GX1BIWV9DUyAgICAgICAgICAg
IDB4MDEwQw0KPiA+ID4gI2RlZmluZSAgICBMSU5LX1VQICAgICAgICAgICAgICAgICAgICAgICAg
QklUKDE2KQ0KPiA+ID4gKyNkZWZpbmUgQ1BVX1RPX0JVU19BRERSICAgICAgICAgICAgIDB4MEZG
RkZGRkYNCj4gPiA+DQo+ID4gPiBzdHJ1Y3QgZHJhN3h4X3BjaWUgew0KPiA+ID4gICAgdm9pZCBf
X2lvbWVtICAgICAgICAqYmFzZTsNCj4gPiA+IEBAIC0xMzgsNiArMTM5LDE3IEBAIHN0YXRpYyB2
b2lkDQo+IGRyYTd4eF9wY2llX2VuYWJsZV9pbnRlcnJ1cHRzKHN0cnVjdA0KPiA+IHBjaWVfcG9y
dCAqcHApDQo+ID4gPg0KPiA+ID4gc3RhdGljIHZvaWQgZHJhN3h4X3BjaWVfaG9zdF9pbml0KHN0
cnVjdCBwY2llX3BvcnQgKnBwKQ0KPiA+ID4gew0KPiA+ID4gKyAgICBpZiAocHAtPmlvX21vZF9i
YXNlKQ0KPiA+ID4gKyAgICAgICAgcHAtPmlvX21vZF9iYXNlICY9IENQVV9UT19CVVNfQUREUjsN
Cj4gPiA+ICsNCj4gPiA+ICsgICAgaWYgKHBwLT5tZW1fbW9kX2Jhc2UpDQo+ID4gPiArICAgICAg
ICBwcC0+bWVtX21vZF9iYXNlICY9IENQVV9UT19CVVNfQUREUjsNCj4gPiA+ICsNCj4gPiA+ICsg
ICAgaWYgKHBwLT5jZmcwX21vZF9iYXNlKSB7DQo+ID4gPiArICAgICAgICBwcC0+Y2ZnMF9tb2Rf
YmFzZSAmPSBDUFVfVE9fQlVTX0FERFI7DQo+ID4gPiArICAgICAgICBwcC0+Y2ZnMV9tb2RfYmFz
ZSAmPSBDUFVfVE9fQlVTX0FERFI7DQo+ID4gPiArICAgIH0NCj4gPiA+ICsNCj4gPiA+ICAgIGR3
X3BjaWVfc2V0dXBfcmMocHApOw0KPiA+ID4gICAgZHJhN3h4X3BjaWVfZXN0YWJsaXNoX2xpbmso
cHApOw0KPiA+ID4gICAgaWYgKElTX0VOQUJMRUQoQ09ORklHX1BDSV9NU0kpKQ0KPiA+ID4gZGlm
ZiAtLWdpdCBhL2RyaXZlcnMvcGNpL2hvc3QvcGNpZS1kZXNpZ253YXJlLmMNCj4gPiBiL2RyaXZl
cnMvcGNpL2hvc3QvcGNpZS1kZXNpZ253YXJlLmMNCj4gPiA+IGluZGV4IDY5NDg2YmUuLjA2YzY4
MmIgMTAwNjQ0DQo+ID4gPiAtLS0gYS9kcml2ZXJzL3BjaS9ob3N0L3BjaWUtZGVzaWdud2FyZS5j
DQo+ID4gPiArKysgYi9kcml2ZXJzL3BjaS9ob3N0L3BjaWUtZGVzaWdud2FyZS5jDQo+ID4gPiBA
QCAtNDE2LDggKzQxNiw3IEBAIGludCBkd19wY2llX2hvc3RfaW5pdChzdHJ1Y3QgcGNpZV9wb3J0
ICpwcCkNCj4gPiA+ICAgICAgICAgICAgcHAtPmlvX2Jhc2UgPSByYW5nZS5jcHVfYWRkcjsNCj4g
PiA+DQo+ID4gPiAgICAgICAgICAgIC8qIEZpbmQgdGhlIHVudHJhbnNsYXRlZCBJTyBzcGFjZSBh
ZGRyZXNzICovDQo+ID4gPiAtICAgICAgICAgICAgcHAtPmlvX21vZF9iYXNlID0gb2ZfcmVhZF9u
dW1iZXIocGFyc2VyLnJhbmdlIC0NCj4gPiA+IC0gICAgICAgICAgICAgICAgICAgICAgICAgICAg
IHBhcnNlci5ucCArIG5hLCBucyk7DQo+ID4gPiArICAgICAgICAgICAgcHAtPmlvX21vZF9iYXNl
ID0gcmFuZ2UuY3B1X2FkZHI7Ow0KPiA+ID4gICAgICAgIH0NCj4gPiA+ICAgICAgICBpZiAocmVz
dHlwZSA9PSBJT1JFU09VUkNFX01FTSkgew0KPiA+ID4gICAgICAgICAgICBvZl9wY2lfcmFuZ2Vf
dG9fcmVzb3VyY2UoJnJhbmdlLCBucCwgJnBwLT5tZW0pOw0KPiA+ID4gQEAgLTQyNiw4ICs0MjUs
NyBAQCBpbnQgZHdfcGNpZV9ob3N0X2luaXQoc3RydWN0IHBjaWVfcG9ydCAqcHApDQo+ID4gPiAg
ICAgICAgICAgIHBwLT5tZW1fYnVzX2FkZHIgPSByYW5nZS5wY2lfYWRkcjsNCj4gPiA+DQo+ID4g
PiAgICAgICAgICAgIC8qIEZpbmQgdGhlIHVudHJhbnNsYXRlZCBNRU0gc3BhY2UgYWRkcmVzcyAq
Lw0KPiA+ID4gLSAgICAgICAgICAgIHBwLT5tZW1fbW9kX2Jhc2UgPSBvZl9yZWFkX251bWJlcihw
YXJzZXIucmFuZ2UgLQ0KPiA+ID4gLSAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHBhcnNl
ci5ucCArIG5hLCBucyk7DQo+ID4gPiArICAgICAgICAgICAgcHAtPm1lbV9tb2RfYmFzZSA9IHJh
bmdlLmNwdV9hZGRyOw0KPiA+ID4gICAgICAgIH0NCj4gPiA+ICAgICAgICBpZiAocmVzdHlwZSA9
PSAwKSB7DQo+ID4gPiAgICAgICAgICAgIG9mX3BjaV9yYW5nZV90b19yZXNvdXJjZSgmcmFuZ2Us
IG5wLCAmcHAtPmNmZyk7DQo+ID4gPiBAQCAtNDM3LDggKzQzNSw3IEBAIGludCBkd19wY2llX2hv
c3RfaW5pdChzdHJ1Y3QgcGNpZV9wb3J0ICpwcCkNCj4gPiA+ICAgICAgICAgICAgcHAtPmNmZzFf
YmFzZSA9IHBwLT5jZmcuc3RhcnQgKyBwcC0+Y2ZnMF9zaXplOw0KPiA+ID4NCj4gPiA+ICAgICAg
ICAgICAgLyogRmluZCB0aGUgdW50cmFuc2xhdGVkIGNvbmZpZ3VyYXRpb24gc3BhY2UgYWRkcmVz
cyAqLw0KPiA+ID4gLSAgICAgICAgICAgIHBwLT5jZmcwX21vZF9iYXNlID0gb2ZfcmVhZF9udW1i
ZXIocGFyc2VyLnJhbmdlIC0NCj4gPiA+IC0gICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
cGFyc2VyLm5wICsgbmEsIG5zKTsNCj4gPiA+ICsgICAgICAgICAgICBwcC0+Y2ZnMF9tb2RfYmFz
ZSA9IHJhbmdlLmNwdV9hZGRyOw0KPiA+ID4gICAgICAgICAgICBwcC0+Y2ZnMV9tb2RfYmFzZSA9
IHBwLT5jZmcwX21vZF9iYXNlICsNCj4gPiA+ICAgICAgICAgICAgICAgICAgICAgICAgcHAtPmNm
ZzBfc2l6ZTsNCj4gPiA+ICAgICAgICB9DQo+ID4gPiAtLQ0KPiA+ID4gMS45LjENCj4gPiA+DQo+
ID4gPg0KPiA+ID4+IC0tLS0tT3JpZ2luYWwgTWVzc2FnZS0tLS0tDQo+ID4gPj4gRnJvbTogbGlu
dXgtcGNpLW93bmVyQHZnZXIua2VybmVsLm9yZyBbbWFpbHRvOmxpbnV4LXBjaS0NCj4gPiA+PiBv
d25lckB2Z2VyLmtlcm5lbC5vcmddIE9uIEJlaGFsZiBPZiBSb2IgSGVycmluZw0KPiA+ID4+IFNl
bnQ6IEZyaWRheSwgSnVseSAzMSwgMjAxNSA1OjUzIFBNDQo+ID4gPj4gVG86IEtpc2hvbiBWaWph
eSBBYnJhaGFtIEkNCj4gPiA+PiBDYzogR2FicmllbGUgUGFvbG9uaTsgQmpvcm4gSGVsZ2Fhczsg
YXJuZEBhcm5kYi5kZTsNCj4gPiA+PiBsb3JlbnpvLnBpZXJhbGlzaUBhcm0uY29tOyBXYW5nemhv
dSAoQik7IHJvYmgrZHRAa2VybmVsLm9yZzsNCj4gPiA+PiBqYW1lcy5tb3JzZUBhcm0uY29tOyBM
aXZpdS5EdWRhdUBhcm0uY29tOyBsaW51eC0NCj4gcGNpQHZnZXIua2VybmVsLm9yZzsNCj4gPiA+
PiBsaW51eC1hcm0ta2VybmVsQGxpc3RzLmluZnJhZGVhZC5vcmc7IGRldmljZXRyZWVAdmdlci5r
ZXJuZWwub3JnOw0KPiA+ID4+IFl1YW56aGljaGFuZzsgWmh1ZGFjYWk7IHpoYW5nanVrdW87IHFp
dXpoZW5mYTsgTGlndW96aHUgKEtlbm5ldGgpOw0KPiA+ID4+IEppbmdvbyBIYW47IFByYXR5dXNo
IEFuYW5kOyBBcm5kIEJlcmdtYW5uDQo+ID4gPj4gU3ViamVjdDogUmU6IFtQQVRDSCB2Nl0gUENJ
OiBTdG9yZSBQQ0llIGJ1cyBhZGRyZXNzIGluIHN0cnVjdA0KPiA+ID4+IG9mX3BjaV9yYW5nZQ0K
PiA+ID4+DQo+ID4gPj4gT24gRnJpLCBKdWwgMzEsIDIwMTUgYXQgOTo1NyBBTSwgS2lzaG9uIFZp
amF5IEFicmFoYW0gSQ0KPiA+IDxraXNob25AdGkuY29tPg0KPiA+ID4+IHdyb3RlOg0KPiA+ID4+
PiArQXJuZA0KPiA+ID4+Pg0KPiA+ID4+PiBIaSwNCj4gPiA+Pj4NCj4gPiA+Pj4+IE9uIEZyaWRh
eSAzMSBKdWx5IDIwMTUgMDc6NTUgUE0sIEdhYnJpZWxlIFBhb2xvbmkgd3JvdGU6DQo+ID4gPj4+
PiBbK2NjIEtpc2hvbl0NCj4gPiA+Pj4+DQo+ID4gPj4+Pj4gLS0tLS1PcmlnaW5hbCBNZXNzYWdl
LS0tLS0NCj4gPiA+Pj4+PiBGcm9tOiBsaW51eC1wY2ktb3duZXJAdmdlci5rZXJuZWwub3JnIFtt
YWlsdG86bGludXgtcGNpLQ0KPiA+ID4+Pj4+IG93bmVyQHZnZXIua2VybmVsLm9yZ10gT24gQmVo
YWxmIE9mIFJvYiBIZXJyaW5nDQo+ID4gPj4+Pj4gU2VudDogVGh1cnNkYXksIEp1bHkgMzAsIDIw
MTUgOTo0MiBQTQ0KPiA+ID4+Pj4+IFRvOiBHYWJyaWVsZSBQYW9sb25pDQo+ID4gPj4+Pj4gQ2M6
IEJqb3JuIEhlbGdhYXM7IGFybmRAYXJuZGIuZGU7IGxvcmVuem8ucGllcmFsaXNpQGFybS5jb207
DQo+ID4gPj4gV2FuZ3pob3UNCj4gPiA+Pj4+PiAoQik7IHJvYmgrZHRAa2VybmVsLm9yZzsgamFt
ZXMubW9yc2VAYXJtLmNvbTsNCj4gTGl2aXUuRHVkYXVAYXJtLmNvbTsNCj4gPiA+Pj4+PiBsaW51
eC1wY2lAdmdlci5rZXJuZWwub3JnOyBsaW51eC1hcm0ta2VybmVsQGxpc3RzLmluZnJhZGVhZC5v
cmc7DQo+ID4gPj4+Pj4gZGV2aWNldHJlZUB2Z2VyLmtlcm5lbC5vcmc7IFl1YW56aGljaGFuZzsg
Wmh1ZGFjYWk7IHpoYW5nanVrdW87DQo+ID4gPj4+Pj4gcWl1emhlbmZhOyBMaWd1b3podSAoS2Vu
bmV0aCk7IEppbmdvbyBIYW47IFByYXR5dXNoIEFuYW5kDQo+ID4gPj4+Pj4gU3ViamVjdDogUmU6
IFtQQVRDSCB2Nl0gUENJOiBTdG9yZSBQQ0llIGJ1cyBhZGRyZXNzIGluIHN0cnVjdA0KPiA+ID4+
Pj4+IG9mX3BjaV9yYW5nZQ0KPiA+ID4+Pj4+DQo+ID4gPj4+Pj4gT24gVGh1LCBKdWwgMzAsIDIw
MTUgYXQgMTI6MzQgUE0sIEdhYnJpZWxlIFBhb2xvbmkNCj4gPiA+Pj4+PiA8Z2FicmllbGUucGFv
bG9uaUBodWF3ZWkuY29tPiB3cm90ZToNCj4gPiA+Pj4+Pj4+IC0tLS0tT3JpZ2luYWwgTWVzc2Fn
ZS0tLS0tDQo+ID4gPj4+Pj4+PiBGcm9tOiBCam9ybiBIZWxnYWFzIFttYWlsdG86YmhlbGdhYXNA
Z29vZ2xlLmNvbV0NCj4gPiA+Pj4+Pj4+IFNlbnQ6IDMwIEp1bHkgMjAxNSAxODoxNQ0KPiA+ID4+
Pj4+Pj4gT24gVGh1LCBKdWwgMzAsIDIwMTUgYXQgMDQ6NTA6NTVQTSArMDAwMCwgR2FicmllbGUg
UGFvbG9uaQ0KPiA+IHdyb3RlOg0KPiA+ID4+Pj4+Pj4+PiAtLS0tLU9yaWdpbmFsIE1lc3NhZ2Ut
LS0tLQ0KPiA+ID4+Pj4+Pj4+PiBGcm9tOiBsaW51eC1wY2ktb3duZXJAdmdlci5rZXJuZWwub3Jn
IFttYWlsdG86bGludXgtcGNpLQ0KPiA+ID4+Pj4+Pj4+PiBvd25lckB2Z2VyLmtlcm5lbC5vcmdd
IE9uIEJlaGFsZiBPZiBCam9ybiBIZWxnYWFzDQo+ID4gPj4+Pj4+Pj4+IFNlbnQ6IFRodXJzZGF5
LCBKdWx5IDMwLCAyMDE1IDU6MTUgUE0NCj4gPiA+Pj4+Pj4+Pj4gT24gVGh1LCBKdWwgMzAsIDIw
MTUgYXQgMDE6NTI6MTNQTSArMDAwMCwgR2FicmllbGUgUGFvbG9uaQ0KPiA+ID4+IHdyb3RlOg0K
PiA+ID4+Pj4+DQo+ID4gPj4+Pj4gWy4uLl0NCj4gPiA+Pj4+Pg0KPiA+ID4+Pj4+Pj4+Pj4gSSBk
b27igJl0IHRoaW5rIHdlIHNob3VsZCByZWx5IG9uIFtDUFVdIGFkZHJlc3Nlcy4uLndoYXQgaWYN
Cj4gPiB0aGUNCj4gPiA+Pj4+Pj4+Pj4gaW50ZXJtZWRpYXRlDQo+ID4gPj4+Pj4+Pj4+PiB0cmFu
c2xhdGlvbiBsYXllciBjaGFuZ2VzIHRoZSBsb3dlciBzaWduaWZpY2FudCBiaXRzIG9mDQo+IHRo
ZQ0KPiA+ID4+Pj4+ICJidXMNCj4gPiA+Pj4+Pj4+Pj4gYWRkcmVzcyINCj4gPiA+Pj4+Pj4+Pj4+
IHRvIHRyYW5zbGF0ZSBpbnRvIGEgY3B1IGFkZHJlc3M/DQo+ID4gPj4+Pj4+Pj4+DQo+ID4gPj4+
Pj4+Pj4+IElzIGl0IHJlYWxseSBhIHBvc3NpYmxpdHkgdGhhdCB0aGUgbG93ZXIgYml0cyBjb3Vs
ZCBiZQ0KPiA+IGNoYW5nZWQ/DQo+ID4gPj4+Pj4+Pj4NCj4gPiA+Pj4+Pj4+PiBJJ3ZlIGNoZWNr
ZWQgYWxsIHRoZSBjdXJyZW50IGRlaWdud2FyZSB1c2VycyBEVHMgZXhjZXB0DQo+ICJwY2ktDQo+
ID4gPj4+Pj4+PiBsYXllcnNjYXBlIg0KPiA+ID4+Pj4+Pj4+IHRoYXQgSSBjb3VsZCBub3QgZmlu
ZDoNCj4gPiA+Pj4+Pj4+PiBzcGVhcjEzMTAuZHRzaQ0KPiA+ID4+Pj4+Pj4+IHNwZWFyMTM0MC5k
dHNpDQo+ID4gPj4+Pj4+Pj4gZHJhNy5kdHNpDQo+ID4gPj4+Pj4+Pj4gaW14NnFkbC5kdHNpDQo+
ID4gPj4+Pj4+Pj4gaW14NnN4LmR0c2kNCj4gPiA+Pj4+Pj4+PiBrZXlzdG9uZS5kdHNpDQo+ID4g
Pj4+Pj4+Pj4gZXh5bm9zNTQ0MC5kdHNpDQo+ID4gPj4+Pj4+Pj4NCj4gPiA+Pj4+Pj4+PiBOb25l
IG9mIHRoZW0gbW9kaWZpZXMgdGhlIGxvd2VyIGJpdHMuIFRvIGJlIG1vcmUgcHJlY2lzZSB0aGUN
Cj4gPiA+PiBvbmx5DQo+ID4gPj4+Pj4gZ3V5DQo+ID4gPj4+Pj4+Pj4gdGhhdCBwcm92aWRlcyBh
bm90aGVyIHRyYW5zbGF0aW9uIGxheWVyIGlzICJkcmE3LmR0c2kiOg0KPiA+ID4+Pj4+Pj4+IGF4
aTANCj4gPiA+Pj4+Pj4+PiBodHRwOi8vbHhyLmZyZWUtDQo+ID4gPj4+Pj4gZWxlY3Ryb25zLmNv
bS9zb3VyY2UvYXJjaC9hcm0vYm9vdC9kdHMvZHJhNy5kdHNpI0wyMDcNCj4gPiA+Pj4+Pj4+Pg0K
PiA+ID4+Pj4+Pj4+IGF4aTENCj4gPiA+Pj4+Pj4+PiBodHRwOi8vbHhyLmZyZWUtDQo+ID4gPj4+
Pj4gZWxlY3Ryb25zLmNvbS9zb3VyY2UvYXJjaC9hcm0vYm9vdC9kdHMvZHJhNy5kdHNpI0wyNDEN
Cj4gPiA+Pj4+Pj4+Pg0KPiA+ID4+Pj4+Pj4+IEZvciB0aGlzIGNhc2UgbWFza2luZyB0aGUgdG9w
IDRiaXRzIChiaXRzMjggdG8gMzEpIHNob3VsZA0KPiBtYWtlDQo+ID4gPj4gdGhlDQo+ID4gPj4+
Pj4gam9iLg0KPiA+ID4+Pj4+DQo+ID4gPj4+Pj4gSU1PLCB3ZSBzaG91bGQganVzdCBmaXggdGhp
cyBjYXNlLiBBZnRlciBmdXJ0aGVyIHN0dWR5LCBJIGRvbid0DQo+ID4gPj4gdGhpbmsNCj4gPiA+
Pj4+PiB0aGlzIGlzIGEgRFcgaXNzdWUsIGJ1dCByYXRoZXIgYW4gU09DIGludGVncmF0aW9uIGlz
c3VlLg0KPiA+ID4+Pj4+DQo+ID4gPj4+Pj4gSSBiZWxpZXZlIHlvdSBjYW4ganVzdCBmaXh1cCB0
aGUgYWRkcmVzcyBpbiB0aGUgcHAtPm9wcy0NCj4gPiA+aG9zdF9pbml0DQo+ID4gPj4gaG9vay4N
Cj4gPiA+Pj4+DQo+ID4gPj4+PiBZZXMgSSBndWVzcyB0aGF0IEkgY291bGQganVzdCBhc3NpZ24g
cHAtPigqKV9tb2RfYmFzZSB0byB0aGUgQ1BVDQo+ID4gPj4gYWRkcmVzcw0KPiA+ID4+Pj4gaW4g
RFcgYW5kIG1hc2sgaXQgb3V0IGluIGRyYTd4eF9wY2llX2hvc3RfaW5pdCgpLi4uDQo+ID4gPj4+
Pg0KPiA+ID4+Pj4gS2lzaG9uLCB3b3VsZCB5b3UgYmUgb2sgd2l0aCB0aGF0Pw0KPiA+ID4+Pg0K
PiA+ID4+PiBJbml0aWFsbHkgSSB3YXMgdXNpbmcgKmJhc2UtbWFzayogcHJvcGVydHkgZnJvbSBk
dC4gTWUgYW5kIEFybmQNCj4gPiA+PiAoY2MnZWQpIGhhZA0KPiA+ID4+PiB0aGlzIGRpc2N1c3Np
b24gWzFdIGJlZm9yZSB3ZSBkZWNpZGVkIHRoZSBjdXJyZW50IGFwcHJvYWNoLiBJdCdsbA0KPiA+
IGJlDQo+ID4gPj4gZ29vZCB0bw0KPiA+ID4+PiBjaGVjayB3aXRoIEFybmQgdG9vLg0KPiA+ID4+
Pg0KPiA+ID4+PiBbMV0gLT4gIGh0dHA6Ly9saXN0cy5pbmZyYWRlYWQub3JnL3BpcGVybWFpbC9s
aW51eC1hcm0tDQo+IGtlcm5lbC8yMDE0LQ0KPiA+ID4+IE1heS8yNTM1MjguaHRtbA0KPiA+ID4+
DQo+ID4gPj4gVGhlIHByb2JsZW0gSSBoYXZlIGhlcmUgaXMgdGhlIHVzZSBvZiByYW5nZXMgZG9l
cyBub3QgbmVjZXNzYXJpbHkNCj4gPiBtZWFuDQo+ID4gPj4gZmV3ZXIgYWRkcmVzcyBiaXRzIGFy
ZSBhdmFpbGFibGUuIEl0IGNhbiBiZSB1c2VkIGp1c3QgZm9yDQo+ID4gY29udmVuaWVuY2UNCj4g
PiA+PiBvZiBub3QgcHV0dGluZyB0aGUgZnVsbCBhZGRyZXNzIGludG8gZXZlcnkgbm9kZSdzIHJl
ZyBwcm9wZXJ0eS4NCj4gQW5kDQo+ID4gPj4gdmljZSB2ZXJzYSwgdGhlcmUgYXJlIHByb2JhYmx5
IHBsZW50eSBvZiBjYXNlcyB3aGVyZSB3ZSBoYXZlIHRoZQ0KPiA+IGZ1bGwNCj4gPiA+PiBhZGRy
ZXNzIGluIHRoZSBub2RlcywgYnV0IHJlYWxseSBvbmx5IHNvbWUgb2YgdGhlIGFkZHJlc3MgYml0
cyBhcmUNCj4gPiA+PiBkZWNvZGVkIGF0IHRoZSBJUCBibG9jay4gV2hldGhlciB0aGUgYWRkcmVz
cyBiaXRzIGFyZSBwcmVzZW50IGlzDQo+ID4gPj4gcmFyZWx5IGNhcmVkIGFib3V0IG9yIGtub3du
IGJ5IHMvdyBmb2xrcyB1bnRpbCB5b3UgaGl0IGEgcHJvYmxlbQ0KPiA+IGxpa2UNCj4gPiA+PiB0
aGlzLiBHaXZlbiB0aGlzIGlzIGFuIGlzb2xhdGVkIGNhc2UgQVRNLCBJIHdvdWxkIGZpeCBpdCBp
biBhbg0KPiA+ID4+IGlzb2xhdGVkIHdheS4gSXQgZG9lcyBub3QgYWZmZWN0IHRoZSBiaW5kaW5n
IGFuZCBjb3VsZCBiZSBjaGFuZ2VkDQo+IGluDQo+ID4gPj4gdGhlIGtlcm5lbCBsYXRlciBpZiB0
aGlzIGJlY29tZXMgYSBjb21tb24gbmVlZC4NCj4gPiA+Pg0KPiA+ID4+IFJvYg0KPiA+ID4+IC0t
DQo+ID4gPj4gVG8gdW5zdWJzY3JpYmUgZnJvbSB0aGlzIGxpc3Q6IHNlbmQgdGhlIGxpbmUgInVu
c3Vic2NyaWJlIGxpbnV4LQ0KPiBwY2kiDQo+ID4gaW4NCj4gPiA+PiB0aGUgYm9keSBvZiBhIG1l
c3NhZ2UgdG8gbWFqb3Jkb21vQHZnZXIua2VybmVsLm9yZw0KPiA+ID4+IE1vcmUgbWFqb3Jkb21v
IGluZm8gYXQgIGh0dHA6Ly92Z2VyLmtlcm5lbC5vcmcvbWFqb3Jkb21vLWluZm8uaHRtbA0KPiAT
P+enuz8/Pz8/7LWUKy3quYHosqAX7KqQd+y4qT/qtp0/P+yajmlyKOOOjRfsjbPorop97LCg6ry/
7J+6P2o6K3brj6M/7LOt5ZapelorwoA/embvvIINCj4g7YaS7ImxP+uEruuFrGnvo7fpjqx6P+y3
v+KFsT/shrPpiLo/P+WIqhtmDQo=

^ permalink raw reply	[flat|nested] 73+ messages in thread
* [PATCH v6] PCI: Store PCIe bus address in struct of_pci_range
@ 2015-08-06 13:52                                       ` Gabriele Paoloni
  0 siblings, 0 replies; 73+ messages in thread
From: Gabriele Paoloni @ 2015-08-06 13:52 UTC (permalink / raw)
  To: linux-arm-kernel

Hi all

This patch has now been moved in "[PATCH v6 1/6] PCI: designware: move calculation of bus addresses to DRA7xx"

commit "f4c55c5a3f7f (PCI: designware: Program ATU with untranslated address)" has been reverted in
"[PATCH v6 3/6] PCI: designware: Add ARM64 support"

Gab

> -----Original Message-----
> From: linux-pci-owner at vger.kernel.org [mailto:linux-pci-
> owner at vger.kernel.org] On Behalf Of Gabriele Paoloni
> Sent: Tuesday, August 04, 2015 11:12 AM
> To: Jingoo Han
> Cc: Rob Herring; Kishon Vijay Abraham I; Bjorn Helgaas; arnd at arndb.de;
> lorenzo.pieralisi at arm.com; Wangzhou (B); robh+dt at kernel.org;
> james.morse at arm.com; Liviu.Dudau at arm.com; linux-pci at vger.kernel.org;
> linux-arm-kernel at lists.infradead.org; devicetree at vger.kernel.org;
> Yuanzhichang; Zhudacai; zhangjukuo; qiuzhenfa; Liguozhu (Kenneth);
> Pratyush Anand; Arnd Bergmann
> Subject: RE: [PATCH v6] PCI: Store PCIe bus address in struct
> of_pci_range
> 
> 
> 
> > -----Original Message-----
> > From: Jingoo Han [mailto:jingoohan1 at gmail.com]
> > Sent: Tuesday, August 04, 2015 5:20 AM
> > To: Gabriele Paoloni
> > Cc: Rob Herring; Kishon Vijay Abraham I; Bjorn Helgaas; arnd at arndb.de;
> > lorenzo.pieralisi at arm.com; Wangzhou (B); robh+dt at kernel.org;
> > james.morse at arm.com; Liviu.Dudau at arm.com; linux-pci at vger.kernel.org;
> > linux-arm-kernel at lists.infradead.org; devicetree at vger.kernel.org;
> > Yuanzhichang; Zhudacai; zhangjukuo; qiuzhenfa; Liguozhu (Kenneth);
> > Pratyush Anand; Arnd Bergmann; jingoohan1 at gmail.com
> > Subject: Re: [PATCH v6] PCI: Store PCIe bus address in struct
> > of_pci_range
> >
> > On 2015. 8. 3., at PM 8:18, Gabriele Paoloni
> > <gabriele.paoloni@huawei.com> wrote:
> > >
> > > Rob, Kishon what about the following solution?....
> > >
> > > ---
> > > drivers/pci/host/pci-dra7xx.c      | 12 ++++++++++++
> > > drivers/pci/host/pcie-designware.c |  9 +++------
> >
> > Hi Paoloni,
> >
> > OK, it looks good.
> > I want you to revert the following patch.
> >
> > commit "f4c55c5a3f7f (PCI: designware: Program ATU with untranslated
> > address)"
> >
> > Would you remove all '*_mode_base's?
> 
> Hi Jingoo Han,
> 
> If I reverted the commit, in order to get designware working, dra7
> should mask directly the CPU addresses stored in pp->cfg0_base,
> pp->cfg1_base, pp->mem_base and pp->io_base.
> 
> The masking would happen at this point:
> http://lxr.free-electrons.com/source/drivers/pci/host/pcie-
> designware.c#L493
> 
> Now:
> - I see that pp->cfg<0/1>_base are used in devm_ioremap before that
>   point and nowhere else, so they should be ok.
> - pp->mem_base is used in dw_pcie_setup_rc(); dw_pcie_setup_rc() is
> called
>   in dra7xx_pcie_host_init()...so here I should move the masking after
>   dw_pcie_setup() retuned, but I think it should be ok
> - pp->io_base is used in pci_ioremap_io() in dw_pcie_setup(). Now
> currently
>   in designware this is called by pci_bios_init_hw(); this is after the
>   masking....so here we would have a the wrong value passed
> 
> Said that if you look at "[PATCH v5 2/5] PCI: designware: Add ARM64
> support",
> that is the patchset where this patch is needed, you can see that
> dw_pcie_setup() is removed and pp->io_base is used in
> pci_remap_iospace() before
> the masking would happen in dra7xx_pcie_host_init()...so for this
> patchset we
> should be good to revert the commit.
> 
> I propose to add a new patch in the patchset "[PATCH v5 0/5] PCI: hisi:
> Add PCIe
> host support for HiSilicon SoC Hip05" as the one I just posted in this
> thread (this would not revert the commit but just moves the masking to
> dra7xx)
> 
> I would revert the commit in "[PATCH v5 2/5] PCI: designware: Add ARM64
> support"
> together with the rest of designware rework.
> 
> So we would have always a working version of designware...
> 
> Are you ok with this?
> 
> >
> > Best regards,
> > Jingoo Han
> >
> > > 2 files changed, 15 insertions(+), 6 deletions(-)
> > >
> > > diff --git a/drivers/pci/host/pci-dra7xx.c b/drivers/pci/host/pci-
> > dra7xx.c
> > > index 80db09e..bb2635f 100644
> > > --- a/drivers/pci/host/pci-dra7xx.c
> > > +++ b/drivers/pci/host/pci-dra7xx.c
> > > @@ -61,6 +61,7 @@
> > >
> > > #define    PCIECTRL_DRA7XX_CONF_PHY_CS            0x010C
> > > #define    LINK_UP                        BIT(16)
> > > +#define CPU_TO_BUS_ADDR             0x0FFFFFFF
> > >
> > > struct dra7xx_pcie {
> > >    void __iomem        *base;
> > > @@ -138,6 +139,17 @@ static void
> dra7xx_pcie_enable_interrupts(struct
> > pcie_port *pp)
> > >
> > > static void dra7xx_pcie_host_init(struct pcie_port *pp)
> > > {
> > > +    if (pp->io_mod_base)
> > > +        pp->io_mod_base &= CPU_TO_BUS_ADDR;
> > > +
> > > +    if (pp->mem_mod_base)
> > > +        pp->mem_mod_base &= CPU_TO_BUS_ADDR;
> > > +
> > > +    if (pp->cfg0_mod_base) {
> > > +        pp->cfg0_mod_base &= CPU_TO_BUS_ADDR;
> > > +        pp->cfg1_mod_base &= CPU_TO_BUS_ADDR;
> > > +    }
> > > +
> > >    dw_pcie_setup_rc(pp);
> > >    dra7xx_pcie_establish_link(pp);
> > >    if (IS_ENABLED(CONFIG_PCI_MSI))
> > > diff --git a/drivers/pci/host/pcie-designware.c
> > b/drivers/pci/host/pcie-designware.c
> > > index 69486be..06c682b 100644
> > > --- a/drivers/pci/host/pcie-designware.c
> > > +++ b/drivers/pci/host/pcie-designware.c
> > > @@ -416,8 +416,7 @@ int dw_pcie_host_init(struct pcie_port *pp)
> > >            pp->io_base = range.cpu_addr;
> > >
> > >            /* Find the untranslated IO space address */
> > > -            pp->io_mod_base = of_read_number(parser.range -
> > > -                             parser.np + na, ns);
> > > +            pp->io_mod_base = range.cpu_addr;;
> > >        }
> > >        if (restype == IORESOURCE_MEM) {
> > >            of_pci_range_to_resource(&range, np, &pp->mem);
> > > @@ -426,8 +425,7 @@ int dw_pcie_host_init(struct pcie_port *pp)
> > >            pp->mem_bus_addr = range.pci_addr;
> > >
> > >            /* Find the untranslated MEM space address */
> > > -            pp->mem_mod_base = of_read_number(parser.range -
> > > -                              parser.np + na, ns);
> > > +            pp->mem_mod_base = range.cpu_addr;
> > >        }
> > >        if (restype == 0) {
> > >            of_pci_range_to_resource(&range, np, &pp->cfg);
> > > @@ -437,8 +435,7 @@ int dw_pcie_host_init(struct pcie_port *pp)
> > >            pp->cfg1_base = pp->cfg.start + pp->cfg0_size;
> > >
> > >            /* Find the untranslated configuration space address */
> > > -            pp->cfg0_mod_base = of_read_number(parser.range -
> > > -                               parser.np + na, ns);
> > > +            pp->cfg0_mod_base = range.cpu_addr;
> > >            pp->cfg1_mod_base = pp->cfg0_mod_base +
> > >                        pp->cfg0_size;
> > >        }
> > > --
> > > 1.9.1
> > >
> > >
> > >> -----Original Message-----
> > >> From: linux-pci-owner at vger.kernel.org [mailto:linux-pci-
> > >> owner at vger.kernel.org] On Behalf Of Rob Herring
> > >> Sent: Friday, July 31, 2015 5:53 PM
> > >> To: Kishon Vijay Abraham I
> > >> Cc: Gabriele Paoloni; Bjorn Helgaas; arnd at arndb.de;
> > >> lorenzo.pieralisi at arm.com; Wangzhou (B); robh+dt at kernel.org;
> > >> james.morse at arm.com; Liviu.Dudau at arm.com; linux-
> pci at vger.kernel.org;
> > >> linux-arm-kernel at lists.infradead.org; devicetree at vger.kernel.org;
> > >> Yuanzhichang; Zhudacai; zhangjukuo; qiuzhenfa; Liguozhu (Kenneth);
> > >> Jingoo Han; Pratyush Anand; Arnd Bergmann
> > >> Subject: Re: [PATCH v6] PCI: Store PCIe bus address in struct
> > >> of_pci_range
> > >>
> > >> On Fri, Jul 31, 2015 at 9:57 AM, Kishon Vijay Abraham I
> > <kishon@ti.com>
> > >> wrote:
> > >>> +Arnd
> > >>>
> > >>> Hi,
> > >>>
> > >>>> On Friday 31 July 2015 07:55 PM, Gabriele Paoloni wrote:
> > >>>> [+cc Kishon]
> > >>>>
> > >>>>> -----Original Message-----
> > >>>>> From: linux-pci-owner at vger.kernel.org [mailto:linux-pci-
> > >>>>> owner at vger.kernel.org] On Behalf Of Rob Herring
> > >>>>> Sent: Thursday, July 30, 2015 9:42 PM
> > >>>>> To: Gabriele Paoloni
> > >>>>> Cc: Bjorn Helgaas; arnd at arndb.de; lorenzo.pieralisi at arm.com;
> > >> Wangzhou
> > >>>>> (B); robh+dt at kernel.org; james.morse at arm.com;
> Liviu.Dudau at arm.com;
> > >>>>> linux-pci at vger.kernel.org; linux-arm-kernel at lists.infradead.org;
> > >>>>> devicetree at vger.kernel.org; Yuanzhichang; Zhudacai; zhangjukuo;
> > >>>>> qiuzhenfa; Liguozhu (Kenneth); Jingoo Han; Pratyush Anand
> > >>>>> Subject: Re: [PATCH v6] PCI: Store PCIe bus address in struct
> > >>>>> of_pci_range
> > >>>>>
> > >>>>> On Thu, Jul 30, 2015 at 12:34 PM, Gabriele Paoloni
> > >>>>> <gabriele.paoloni@huawei.com> wrote:
> > >>>>>>> -----Original Message-----
> > >>>>>>> From: Bjorn Helgaas [mailto:bhelgaas at google.com]
> > >>>>>>> Sent: 30 July 2015 18:15
> > >>>>>>> On Thu, Jul 30, 2015 at 04:50:55PM +0000, Gabriele Paoloni
> > wrote:
> > >>>>>>>>> -----Original Message-----
> > >>>>>>>>> From: linux-pci-owner at vger.kernel.org [mailto:linux-pci-
> > >>>>>>>>> owner at vger.kernel.org] On Behalf Of Bjorn Helgaas
> > >>>>>>>>> Sent: Thursday, July 30, 2015 5:15 PM
> > >>>>>>>>> On Thu, Jul 30, 2015 at 01:52:13PM +0000, Gabriele Paoloni
> > >> wrote:
> > >>>>>
> > >>>>> [...]
> > >>>>>
> > >>>>>>>>>> I don?t think we should rely on [CPU] addresses...what if
> > the
> > >>>>>>>>> intermediate
> > >>>>>>>>>> translation layer changes the lower significant bits of
> the
> > >>>>> "bus
> > >>>>>>>>> address"
> > >>>>>>>>>> to translate into a cpu address?
> > >>>>>>>>>
> > >>>>>>>>> Is it really a possiblity that the lower bits could be
> > changed?
> > >>>>>>>>
> > >>>>>>>> I've checked all the current deignware users DTs except
> "pci-
> > >>>>>>> layerscape"
> > >>>>>>>> that I could not find:
> > >>>>>>>> spear1310.dtsi
> > >>>>>>>> spear1340.dtsi
> > >>>>>>>> dra7.dtsi
> > >>>>>>>> imx6qdl.dtsi
> > >>>>>>>> imx6sx.dtsi
> > >>>>>>>> keystone.dtsi
> > >>>>>>>> exynos5440.dtsi
> > >>>>>>>>
> > >>>>>>>> None of them modifies the lower bits. To be more precise the
> > >> only
> > >>>>> guy
> > >>>>>>>> that provides another translation layer is "dra7.dtsi":
> > >>>>>>>> axi0
> > >>>>>>>> http://lxr.free-
> > >>>>> electrons.com/source/arch/arm/boot/dts/dra7.dtsi#L207
> > >>>>>>>>
> > >>>>>>>> axi1
> > >>>>>>>> http://lxr.free-
> > >>>>> electrons.com/source/arch/arm/boot/dts/dra7.dtsi#L241
> > >>>>>>>>
> > >>>>>>>> For this case masking the top 4bits (bits28 to 31) should
> make
> > >> the
> > >>>>> job.
> > >>>>>
> > >>>>> IMO, we should just fix this case. After further study, I don't
> > >> think
> > >>>>> this is a DW issue, but rather an SOC integration issue.
> > >>>>>
> > >>>>> I believe you can just fixup the address in the pp->ops-
> > >host_init
> > >> hook.
> > >>>>
> > >>>> Yes I guess that I could just assign pp->(*)_mod_base to the CPU
> > >> address
> > >>>> in DW and mask it out in dra7xx_pcie_host_init()...
> > >>>>
> > >>>> Kishon, would you be ok with that?
> > >>>
> > >>> Initially I was using *base-mask* property from dt. Me and Arnd
> > >> (cc'ed) had
> > >>> this discussion [1] before we decided the current approach. It'll
> > be
> > >> good to
> > >>> check with Arnd too.
> > >>>
> > >>> [1] ->  http://lists.infradead.org/pipermail/linux-arm-
> kernel/2014-
> > >> May/253528.html
> > >>
> > >> The problem I have here is the use of ranges does not necessarily
> > mean
> > >> fewer address bits are available. It can be used just for
> > convenience
> > >> of not putting the full address into every node's reg property.
> And
> > >> vice versa, there are probably plenty of cases where we have the
> > full
> > >> address in the nodes, but really only some of the address bits are
> > >> decoded at the IP block. Whether the address bits are present is
> > >> rarely cared about or known by s/w folks until you hit a problem
> > like
> > >> this. Given this is an isolated case ATM, I would fix it in an
> > >> isolated way. It does not affect the binding and could be changed
> in
> > >> the kernel later if this becomes a common need.
> > >>
> > >> Rob
> > >> --
> > >> To unsubscribe from this list: send the line "unsubscribe linux-
> pci"
> > in
> > >> the body of a message to majordomo at vger.kernel.org
> > >> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> \x13????????+-??\x17?w??????ir(?\x17??}????j:+v????zZ+??zf?
> ?????i??z?????????^[f

^ permalink raw reply	[flat|nested] 73+ messages in thread
* Re: [PATCH v6] PCI: Store PCIe bus address in struct of_pci_range
  2015-08-06 13:52                                       ` Gabriele Paoloni
  (?)
@ 2015-08-06 15:06                                         ` Jingoo Han
  -1 siblings, 0 replies; 73+ messages in thread
From: Jingoo Han @ 2015-08-06 15:06 UTC (permalink / raw)
  To: 'Wangzhou (B)'
  Cc: 'Gabriele Paoloni', 'Rob Herring',
	'Kishon Vijay Abraham I', 'Bjorn Helgaas',
	arnd, lorenzo.pieralisi, robh+dt, james.morse, Liviu.Dudau,
	linux-pci, linux-arm-kernel, devicetree, 'Yuanzhichang',
	'Zhudacai', 'zhangjukuo', 'qiuzhenfa',
	'Liguozhu (Kenneth)', 'Pratyush Anand',
	'Arnd Bergmann', 'Jingoo Han'

On Thursday, August 06, 2015 10:53 PM, Gabriele Paoloni wrote:
> 
> Hi all
> 
> This patch has now been moved in "[PATCH v6 1/6] PCI: designware: move calculation of bus addresses to
> DRA7xx"

To Zhou Wang,

Please send your patches to 'jingoohan1@gmail.com', instead of 'jg1.han@samsung.com'.
Even though, I have done mainline works for Samsung SoCs, Samsung does not support me
as a maintainer. So, please don't send your patches to my Samsung e-mail.

Best regards,
Jingoo Han

> 
> commit "f4c55c5a3f7f (PCI: designware: Program ATU with untranslated address)" has been reverted in
> "[PATCH v6 3/6] PCI: designware: Add ARM64 support"
> 
> Gab
> 
> > -----Original Message-----
> > From: linux-pci-owner@vger.kernel.org [mailto:linux-pci-
> > owner@vger.kernel.org] On Behalf Of Gabriele Paoloni
> > Sent: Tuesday, August 04, 2015 11:12 AM
> > To: Jingoo Han
> > Cc: Rob Herring; Kishon Vijay Abraham I; Bjorn Helgaas; arnd@arndb.de;
> > lorenzo.pieralisi@arm.com; Wangzhou (B); robh+dt@kernel.org;
> > james.morse@arm.com; Liviu.Dudau@arm.com; linux-pci@vger.kernel.org;
> > linux-arm-kernel@lists.infradead.org; devicetree@vger.kernel.org;
> > Yuanzhichang; Zhudacai; zhangjukuo; qiuzhenfa; Liguozhu (Kenneth);
> > Pratyush Anand; Arnd Bergmann
> > Subject: RE: [PATCH v6] PCI: Store PCIe bus address in struct
> > of_pci_range
> >
> >
> >
> > > -----Original Message-----
> > > From: Jingoo Han [mailto:jingoohan1@gmail.com]
> > > Sent: Tuesday, August 04, 2015 5:20 AM
> > > To: Gabriele Paoloni
> > > Cc: Rob Herring; Kishon Vijay Abraham I; Bjorn Helgaas; arnd@arndb.de;
> > > lorenzo.pieralisi@arm.com; Wangzhou (B); robh+dt@kernel.org;
> > > james.morse@arm.com; Liviu.Dudau@arm.com; linux-pci@vger.kernel.org;
> > > linux-arm-kernel@lists.infradead.org; devicetree@vger.kernel.org;
> > > Yuanzhichang; Zhudacai; zhangjukuo; qiuzhenfa; Liguozhu (Kenneth);
> > > Pratyush Anand; Arnd Bergmann; jingoohan1@gmail.com
> > > Subject: Re: [PATCH v6] PCI: Store PCIe bus address in struct
> > > of_pci_range
> > >
> > > On 2015. 8. 3., at PM 8:18, Gabriele Paoloni
> > > <gabriele.paoloni@huawei.com> wrote:
> > > >
> > > > Rob, Kishon what about the following solution?....
> > > >
> > > > ---
> > > > drivers/pci/host/pci-dra7xx.c      | 12 ++++++++++++
> > > > drivers/pci/host/pcie-designware.c |  9 +++------
> > >
> > > Hi Paoloni,
> > >
> > > OK, it looks good.
> > > I want you to revert the following patch.
> > >
> > > commit "f4c55c5a3f7f (PCI: designware: Program ATU with untranslated
> > > address)"
> > >
> > > Would you remove all '*_mode_base's?
> >
> > Hi Jingoo Han,
> >
> > If I reverted the commit, in order to get designware working, dra7
> > should mask directly the CPU addresses stored in pp->cfg0_base,
> > pp->cfg1_base, pp->mem_base and pp->io_base.
> >
> > The masking would happen at this point:
> > http://lxr.free-electrons.com/source/drivers/pci/host/pcie-
> > designware.c#L493
> >
> > Now:
> > - I see that pp->cfg<0/1>_base are used in devm_ioremap before that
> >   point and nowhere else, so they should be ok.
> > - pp->mem_base is used in dw_pcie_setup_rc(); dw_pcie_setup_rc() is
> > called
> >   in dra7xx_pcie_host_init()...so here I should move the masking after
> >   dw_pcie_setup() retuned, but I think it should be ok
> > - pp->io_base is used in pci_ioremap_io() in dw_pcie_setup(). Now
> > currently
> >   in designware this is called by pci_bios_init_hw(); this is after the
> >   masking....so here we would have a the wrong value passed
> >
> > Said that if you look at "[PATCH v5 2/5] PCI: designware: Add ARM64
> > support",
> > that is the patchset where this patch is needed, you can see that
> > dw_pcie_setup() is removed and pp->io_base is used in
> > pci_remap_iospace() before
> > the masking would happen in dra7xx_pcie_host_init()...so for this
> > patchset we
> > should be good to revert the commit.
> >
> > I propose to add a new patch in the patchset "[PATCH v5 0/5] PCI: hisi:
> > Add PCIe
> > host support for HiSilicon SoC Hip05" as the one I just posted in this
> > thread (this would not revert the commit but just moves the masking to
> > dra7xx)
> >
> > I would revert the commit in "[PATCH v5 2/5] PCI: designware: Add ARM64
> > support"
> > together with the rest of designware rework.
> >
> > So we would have always a working version of designware...
> >
> > Are you ok with this?
> >
> > >
> > > Best regards,
> > > Jingoo Han
> > >
> > > > 2 files changed, 15 insertions(+), 6 deletions(-)
> > > >
> > > > diff --git a/drivers/pci/host/pci-dra7xx.c b/drivers/pci/host/pci-
> > > dra7xx.c
> > > > index 80db09e..bb2635f 100644
> > > > --- a/drivers/pci/host/pci-dra7xx.c
> > > > +++ b/drivers/pci/host/pci-dra7xx.c
> > > > @@ -61,6 +61,7 @@
> > > >
> > > > #define    PCIECTRL_DRA7XX_CONF_PHY_CS            0x010C
> > > > #define    LINK_UP                        BIT(16)
> > > > +#define CPU_TO_BUS_ADDR             0x0FFFFFFF
> > > >
> > > > struct dra7xx_pcie {
> > > >    void __iomem        *base;
> > > > @@ -138,6 +139,17 @@ static void
> > dra7xx_pcie_enable_interrupts(struct
> > > pcie_port *pp)
> > > >
> > > > static void dra7xx_pcie_host_init(struct pcie_port *pp)
> > > > {
> > > > +    if (pp->io_mod_base)
> > > > +        pp->io_mod_base &= CPU_TO_BUS_ADDR;
> > > > +
> > > > +    if (pp->mem_mod_base)
> > > > +        pp->mem_mod_base &= CPU_TO_BUS_ADDR;
> > > > +
> > > > +    if (pp->cfg0_mod_base) {
> > > > +        pp->cfg0_mod_base &= CPU_TO_BUS_ADDR;
> > > > +        pp->cfg1_mod_base &= CPU_TO_BUS_ADDR;
> > > > +    }
> > > > +
> > > >    dw_pcie_setup_rc(pp);
> > > >    dra7xx_pcie_establish_link(pp);
> > > >    if (IS_ENABLED(CONFIG_PCI_MSI))
> > > > diff --git a/drivers/pci/host/pcie-designware.c
> > > b/drivers/pci/host/pcie-designware.c
> > > > index 69486be..06c682b 100644
> > > > --- a/drivers/pci/host/pcie-designware.c
> > > > +++ b/drivers/pci/host/pcie-designware.c
> > > > @@ -416,8 +416,7 @@ int dw_pcie_host_init(struct pcie_port *pp)
> > > >            pp->io_base = range.cpu_addr;
> > > >
> > > >            /* Find the untranslated IO space address */
> > > > -            pp->io_mod_base = of_read_number(parser.range -
> > > > -                             parser.np + na, ns);
> > > > +            pp->io_mod_base = range.cpu_addr;;
> > > >        }
> > > >        if (restype == IORESOURCE_MEM) {
> > > >            of_pci_range_to_resource(&range, np, &pp->mem);
> > > > @@ -426,8 +425,7 @@ int dw_pcie_host_init(struct pcie_port *pp)
> > > >            pp->mem_bus_addr = range.pci_addr;
> > > >
> > > >            /* Find the untranslated MEM space address */
> > > > -            pp->mem_mod_base = of_read_number(parser.range -
> > > > -                              parser.np + na, ns);
> > > > +            pp->mem_mod_base = range.cpu_addr;
> > > >        }
> > > >        if (restype == 0) {
> > > >            of_pci_range_to_resource(&range, np, &pp->cfg);
> > > > @@ -437,8 +435,7 @@ int dw_pcie_host_init(struct pcie_port *pp)
> > > >            pp->cfg1_base = pp->cfg.start + pp->cfg0_size;
> > > >
> > > >            /* Find the untranslated configuration space address */
> > > > -            pp->cfg0_mod_base = of_read_number(parser.range -
> > > > -                               parser.np + na, ns);
> > > > +            pp->cfg0_mod_base = range.cpu_addr;
> > > >            pp->cfg1_mod_base = pp->cfg0_mod_base +
> > > >                        pp->cfg0_size;
> > > >        }
> > > > --
> > > > 1.9.1
> > > >
> > > >
> > > >> -----Original Message-----
> > > >> From: linux-pci-owner@vger.kernel.org [mailto:linux-pci-
> > > >> owner@vger.kernel.org] On Behalf Of Rob Herring
> > > >> Sent: Friday, July 31, 2015 5:53 PM
> > > >> To: Kishon Vijay Abraham I
> > > >> Cc: Gabriele Paoloni; Bjorn Helgaas; arnd@arndb.de;
> > > >> lorenzo.pieralisi@arm.com; Wangzhou (B); robh+dt@kernel.org;
> > > >> james.morse@arm.com; Liviu.Dudau@arm.com; linux-
> > pci@vger.kernel.org;
> > > >> linux-arm-kernel@lists.infradead.org; devicetree@vger.kernel.org;
> > > >> Yuanzhichang; Zhudacai; zhangjukuo; qiuzhenfa; Liguozhu (Kenneth);
> > > >> Jingoo Han; Pratyush Anand; Arnd Bergmann
> > > >> Subject: Re: [PATCH v6] PCI: Store PCIe bus address in struct
> > > >> of_pci_range
> > > >>
> > > >> On Fri, Jul 31, 2015 at 9:57 AM, Kishon Vijay Abraham I
> > > <kishon@ti.com>
> > > >> wrote:
> > > >>> +Arnd
> > > >>>
> > > >>> Hi,
> > > >>>
> > > >>>> On Friday 31 July 2015 07:55 PM, Gabriele Paoloni wrote:
> > > >>>> [+cc Kishon]
> > > >>>>
> > > >>>>> -----Original Message-----
> > > >>>>> From: linux-pci-owner@vger.kernel.org [mailto:linux-pci-
> > > >>>>> owner@vger.kernel.org] On Behalf Of Rob Herring
> > > >>>>> Sent: Thursday, July 30, 2015 9:42 PM
> > > >>>>> To: Gabriele Paoloni
> > > >>>>> Cc: Bjorn Helgaas; arnd@arndb.de; lorenzo.pieralisi@arm.com;
> > > >> Wangzhou
> > > >>>>> (B); robh+dt@kernel.org; james.morse@arm.com;
> > Liviu.Dudau@arm.com;
> > > >>>>> linux-pci@vger.kernel.org; linux-arm-kernel@lists.infradead.org;
> > > >>>>> devicetree@vger.kernel.org; Yuanzhichang; Zhudacai; zhangjukuo;
> > > >>>>> qiuzhenfa; Liguozhu (Kenneth); Jingoo Han; Pratyush Anand
> > > >>>>> Subject: Re: [PATCH v6] PCI: Store PCIe bus address in struct
> > > >>>>> of_pci_range
> > > >>>>>
> > > >>>>> On Thu, Jul 30, 2015 at 12:34 PM, Gabriele Paoloni
> > > >>>>> <gabriele.paoloni@huawei.com> wrote:
> > > >>>>>>> -----Original Message-----
> > > >>>>>>> From: Bjorn Helgaas [mailto:bhelgaas@google.com]
> > > >>>>>>> Sent: 30 July 2015 18:15
> > > >>>>>>> On Thu, Jul 30, 2015 at 04:50:55PM +0000, Gabriele Paoloni
> > > wrote:
> > > >>>>>>>>> -----Original Message-----
> > > >>>>>>>>> From: linux-pci-owner@vger.kernel.org [mailto:linux-pci-
> > > >>>>>>>>> owner@vger.kernel.org] On Behalf Of Bjorn Helgaas
> > > >>>>>>>>> Sent: Thursday, July 30, 2015 5:15 PM
> > > >>>>>>>>> On Thu, Jul 30, 2015 at 01:52:13PM +0000, Gabriele Paoloni
> > > >> wrote:
> > > >>>>>
> > > >>>>> [...]
> > > >>>>>
> > > >>>>>>>>>> I don’t think we should rely on [CPU] addresses...what if
> > > the
> > > >>>>>>>>> intermediate
> > > >>>>>>>>>> translation layer changes the lower significant bits of
> > the
> > > >>>>> "bus
> > > >>>>>>>>> address"
> > > >>>>>>>>>> to translate into a cpu address?
> > > >>>>>>>>>
> > > >>>>>>>>> Is it really a possiblity that the lower bits could be
> > > changed?
> > > >>>>>>>>
> > > >>>>>>>> I've checked all the current deignware users DTs except
> > "pci-
> > > >>>>>>> layerscape"
> > > >>>>>>>> that I could not find:
> > > >>>>>>>> spear1310.dtsi
> > > >>>>>>>> spear1340.dtsi
> > > >>>>>>>> dra7.dtsi
> > > >>>>>>>> imx6qdl.dtsi
> > > >>>>>>>> imx6sx.dtsi
> > > >>>>>>>> keystone.dtsi
> > > >>>>>>>> exynos5440.dtsi
> > > >>>>>>>>
> > > >>>>>>>> None of them modifies the lower bits. To be more precise the
> > > >> only
> > > >>>>> guy
> > > >>>>>>>> that provides another translation layer is "dra7.dtsi":
> > > >>>>>>>> axi0
> > > >>>>>>>> http://lxr.free-
> > > >>>>> electrons.com/source/arch/arm/boot/dts/dra7.dtsi#L207
> > > >>>>>>>>
> > > >>>>>>>> axi1
> > > >>>>>>>> http://lxr.free-
> > > >>>>> electrons.com/source/arch/arm/boot/dts/dra7.dtsi#L241
> > > >>>>>>>>
> > > >>>>>>>> For this case masking the top 4bits (bits28 to 31) should
> > make
> > > >> the
> > > >>>>> job.
> > > >>>>>
> > > >>>>> IMO, we should just fix this case. After further study, I don't
> > > >> think
> > > >>>>> this is a DW issue, but rather an SOC integration issue.
> > > >>>>>
> > > >>>>> I believe you can just fixup the address in the pp->ops-
> > > >host_init
> > > >> hook.
> > > >>>>
> > > >>>> Yes I guess that I could just assign pp->(*)_mod_base to the CPU
> > > >> address
> > > >>>> in DW and mask it out in dra7xx_pcie_host_init()...
> > > >>>>
> > > >>>> Kishon, would you be ok with that?
> > > >>>
> > > >>> Initially I was using *base-mask* property from dt. Me and Arnd
> > > >> (cc'ed) had
> > > >>> this discussion [1] before we decided the current approach. It'll
> > > be
> > > >> good to
> > > >>> check with Arnd too.
> > > >>>
> > > >>> [1] ->  http://lists.infradead.org/pipermail/linux-arm-
> > kernel/2014-
> > > >> May/253528.html
> > > >>
> > > >> The problem I have here is the use of ranges does not necessarily
> > > mean
> > > >> fewer address bits are available. It can be used just for
> > > convenience
> > > >> of not putting the full address into every node's reg property.
> > And
> > > >> vice versa, there are probably plenty of cases where we have the
> > > full
> > > >> address in the nodes, but really only some of the address bits are
> > > >> decoded at the IP block. Whether the address bits are present is
> > > >> rarely cared about or known by s/w folks until you hit a problem
> > > like
> > > >> this. Given this is an isolated case ATM, I would fix it in an
> > > >> isolated way. It does not affect the binding and could be changed
> > in
> > > >> the kernel later if this becomes a common need.
> > > >>
> > > >> Rob
> > > >> --
> > > >> To unsubscribe from this list: send the line "unsubscribe linux-
> > pci"
> > > in
> > > >> the body of a message to majordomo@vger.kernel.org
> > > >> More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 73+ messages in thread
* Re: [PATCH v6] PCI: Store PCIe bus address in struct of_pci_range
@ 2015-08-06 15:06                                         ` Jingoo Han
  0 siblings, 0 replies; 73+ messages in thread
From: Jingoo Han @ 2015-08-06 15:06 UTC (permalink / raw)
  To: 'Wangzhou (B)'
  Cc: 'Gabriele Paoloni', 'Rob Herring',
	'Kishon Vijay Abraham I', 'Bjorn Helgaas',
	arnd, lorenzo.pieralisi, robh+dt, james.morse, Liviu.Dudau,
	linux-pci, linux-arm-kernel, devicetree, 'Yuanzhichang',
	'Zhudacai', 'zhangjukuo', 'qiuzhenfa',
	'Liguozhu (Kenneth)', 'Pratyush Anand',
	'Arnd Bergmann', 'Jingoo Han'

On Thursday, August 06, 2015 10:53 PM, Gabriele Paoloni wrote:
> 
> Hi all
> 
> This patch has now been moved in "[PATCH v6 1/6] PCI: designware: move calculation of bus addresses to
> DRA7xx"

To Zhou Wang,

Please send your patches to 'jingoohan1@gmail.com', instead of 'jg1.han@samsung.com'.
Even though, I have done mainline works for Samsung SoCs, Samsung does not support me
as a maintainer. So, please don't send your patches to my Samsung e-mail.

Best regards,
Jingoo Han

> 
> commit "f4c55c5a3f7f (PCI: designware: Program ATU with untranslated address)" has been reverted in
> "[PATCH v6 3/6] PCI: designware: Add ARM64 support"
> 
> Gab
> 
> > -----Original Message-----
> > From: linux-pci-owner@vger.kernel.org [mailto:linux-pci-
> > owner@vger.kernel.org] On Behalf Of Gabriele Paoloni
> > Sent: Tuesday, August 04, 2015 11:12 AM
> > To: Jingoo Han
> > Cc: Rob Herring; Kishon Vijay Abraham I; Bjorn Helgaas; arnd@arndb.de;
> > lorenzo.pieralisi@arm.com; Wangzhou (B); robh+dt@kernel.org;
> > james.morse@arm.com; Liviu.Dudau@arm.com; linux-pci@vger.kernel.org;
> > linux-arm-kernel@lists.infradead.org; devicetree@vger.kernel.org;
> > Yuanzhichang; Zhudacai; zhangjukuo; qiuzhenfa; Liguozhu (Kenneth);
> > Pratyush Anand; Arnd Bergmann
> > Subject: RE: [PATCH v6] PCI: Store PCIe bus address in struct
> > of_pci_range
> >
> >
> >
> > > -----Original Message-----
> > > From: Jingoo Han [mailto:jingoohan1@gmail.com]
> > > Sent: Tuesday, August 04, 2015 5:20 AM
> > > To: Gabriele Paoloni
> > > Cc: Rob Herring; Kishon Vijay Abraham I; Bjorn Helgaas; arnd@arndb.de;
> > > lorenzo.pieralisi@arm.com; Wangzhou (B); robh+dt@kernel.org;
> > > james.morse@arm.com; Liviu.Dudau@arm.com; linux-pci@vger.kernel.org;
> > > linux-arm-kernel@lists.infradead.org; devicetree@vger.kernel.org;
> > > Yuanzhichang; Zhudacai; zhangjukuo; qiuzhenfa; Liguozhu (Kenneth);
> > > Pratyush Anand; Arnd Bergmann; jingoohan1@gmail.com
> > > Subject: Re: [PATCH v6] PCI: Store PCIe bus address in struct
> > > of_pci_range
> > >
> > > On 2015. 8. 3., at PM 8:18, Gabriele Paoloni
> > > <gabriele.paoloni@huawei.com> wrote:
> > > >
> > > > Rob, Kishon what about the following solution?....
> > > >
> > > > ---
> > > > drivers/pci/host/pci-dra7xx.c      | 12 ++++++++++++
> > > > drivers/pci/host/pcie-designware.c |  9 +++------
> > >
> > > Hi Paoloni,
> > >
> > > OK, it looks good.
> > > I want you to revert the following patch.
> > >
> > > commit "f4c55c5a3f7f (PCI: designware: Program ATU with untranslated
> > > address)"
> > >
> > > Would you remove all '*_mode_base's?
> >
> > Hi Jingoo Han,
> >
> > If I reverted the commit, in order to get designware working, dra7
> > should mask directly the CPU addresses stored in pp->cfg0_base,
> > pp->cfg1_base, pp->mem_base and pp->io_base.
> >
> > The masking would happen at this point:
> > http://lxr.free-electrons.com/source/drivers/pci/host/pcie-
> > designware.c#L493
> >
> > Now:
> > - I see that pp->cfg<0/1>_base are used in devm_ioremap before that
> >   point and nowhere else, so they should be ok.
> > - pp->mem_base is used in dw_pcie_setup_rc(); dw_pcie_setup_rc() is
> > called
> >   in dra7xx_pcie_host_init()...so here I should move the masking after
> >   dw_pcie_setup() retuned, but I think it should be ok
> > - pp->io_base is used in pci_ioremap_io() in dw_pcie_setup(). Now
> > currently
> >   in designware this is called by pci_bios_init_hw(); this is after the
> >   masking....so here we would have a the wrong value passed
> >
> > Said that if you look at "[PATCH v5 2/5] PCI: designware: Add ARM64
> > support",
> > that is the patchset where this patch is needed, you can see that
> > dw_pcie_setup() is removed and pp->io_base is used in
> > pci_remap_iospace() before
> > the masking would happen in dra7xx_pcie_host_init()...so for this
> > patchset we
> > should be good to revert the commit.
> >
> > I propose to add a new patch in the patchset "[PATCH v5 0/5] PCI: hisi:
> > Add PCIe
> > host support for HiSilicon SoC Hip05" as the one I just posted in this
> > thread (this would not revert the commit but just moves the masking to
> > dra7xx)
> >
> > I would revert the commit in "[PATCH v5 2/5] PCI: designware: Add ARM64
> > support"
> > together with the rest of designware rework.
> >
> > So we would have always a working version of designware...
> >
> > Are you ok with this?
> >
> > >
> > > Best regards,
> > > Jingoo Han
> > >
> > > > 2 files changed, 15 insertions(+), 6 deletions(-)
> > > >
> > > > diff --git a/drivers/pci/host/pci-dra7xx.c b/drivers/pci/host/pci-
> > > dra7xx.c
> > > > index 80db09e..bb2635f 100644
> > > > --- a/drivers/pci/host/pci-dra7xx.c
> > > > +++ b/drivers/pci/host/pci-dra7xx.c
> > > > @@ -61,6 +61,7 @@
> > > >
> > > > #define    PCIECTRL_DRA7XX_CONF_PHY_CS            0x010C
> > > > #define    LINK_UP                        BIT(16)
> > > > +#define CPU_TO_BUS_ADDR             0x0FFFFFFF
> > > >
> > > > struct dra7xx_pcie {
> > > >    void __iomem        *base;
> > > > @@ -138,6 +139,17 @@ static void
> > dra7xx_pcie_enable_interrupts(struct
> > > pcie_port *pp)
> > > >
> > > > static void dra7xx_pcie_host_init(struct pcie_port *pp)
> > > > {
> > > > +    if (pp->io_mod_base)
> > > > +        pp->io_mod_base &= CPU_TO_BUS_ADDR;
> > > > +
> > > > +    if (pp->mem_mod_base)
> > > > +        pp->mem_mod_base &= CPU_TO_BUS_ADDR;
> > > > +
> > > > +    if (pp->cfg0_mod_base) {
> > > > +        pp->cfg0_mod_base &= CPU_TO_BUS_ADDR;
> > > > +        pp->cfg1_mod_base &= CPU_TO_BUS_ADDR;
> > > > +    }
> > > > +
> > > >    dw_pcie_setup_rc(pp);
> > > >    dra7xx_pcie_establish_link(pp);
> > > >    if (IS_ENABLED(CONFIG_PCI_MSI))
> > > > diff --git a/drivers/pci/host/pcie-designware.c
> > > b/drivers/pci/host/pcie-designware.c
> > > > index 69486be..06c682b 100644
> > > > --- a/drivers/pci/host/pcie-designware.c
> > > > +++ b/drivers/pci/host/pcie-designware.c
> > > > @@ -416,8 +416,7 @@ int dw_pcie_host_init(struct pcie_port *pp)
> > > >            pp->io_base = range.cpu_addr;
> > > >
> > > >            /* Find the untranslated IO space address */
> > > > -            pp->io_mod_base = of_read_number(parser.range -
> > > > -                             parser.np + na, ns);
> > > > +            pp->io_mod_base = range.cpu_addr;;
> > > >        }
> > > >        if (restype == IORESOURCE_MEM) {
> > > >            of_pci_range_to_resource(&range, np, &pp->mem);
> > > > @@ -426,8 +425,7 @@ int dw_pcie_host_init(struct pcie_port *pp)
> > > >            pp->mem_bus_addr = range.pci_addr;
> > > >
> > > >            /* Find the untranslated MEM space address */
> > > > -            pp->mem_mod_base = of_read_number(parser.range -
> > > > -                              parser.np + na, ns);
> > > > +            pp->mem_mod_base = range.cpu_addr;
> > > >        }
> > > >        if (restype == 0) {
> > > >            of_pci_range_to_resource(&range, np, &pp->cfg);
> > > > @@ -437,8 +435,7 @@ int dw_pcie_host_init(struct pcie_port *pp)
> > > >            pp->cfg1_base = pp->cfg.start + pp->cfg0_size;
> > > >
> > > >            /* Find the untranslated configuration space address */
> > > > -            pp->cfg0_mod_base = of_read_number(parser.range -
> > > > -                               parser.np + na, ns);
> > > > +            pp->cfg0_mod_base = range.cpu_addr;
> > > >            pp->cfg1_mod_base = pp->cfg0_mod_base +
> > > >                        pp->cfg0_size;
> > > >        }
> > > > --
> > > > 1.9.1
> > > >
> > > >
> > > >> -----Original Message-----
> > > >> From: linux-pci-owner@vger.kernel.org [mailto:linux-pci-
> > > >> owner@vger.kernel.org] On Behalf Of Rob Herring
> > > >> Sent: Friday, July 31, 2015 5:53 PM
> > > >> To: Kishon Vijay Abraham I
> > > >> Cc: Gabriele Paoloni; Bjorn Helgaas; arnd@arndb.de;
> > > >> lorenzo.pieralisi@arm.com; Wangzhou (B); robh+dt@kernel.org;
> > > >> james.morse@arm.com; Liviu.Dudau@arm.com; linux-
> > pci@vger.kernel.org;
> > > >> linux-arm-kernel@lists.infradead.org; devicetree@vger.kernel.org;
> > > >> Yuanzhichang; Zhudacai; zhangjukuo; qiuzhenfa; Liguozhu (Kenneth);
> > > >> Jingoo Han; Pratyush Anand; Arnd Bergmann
> > > >> Subject: Re: [PATCH v6] PCI: Store PCIe bus address in struct
> > > >> of_pci_range
> > > >>
> > > >> On Fri, Jul 31, 2015 at 9:57 AM, Kishon Vijay Abraham I
> > > <kishon@ti.com>
> > > >> wrote:
> > > >>> +Arnd
> > > >>>
> > > >>> Hi,
> > > >>>
> > > >>>> On Friday 31 July 2015 07:55 PM, Gabriele Paoloni wrote:
> > > >>>> [+cc Kishon]
> > > >>>>
> > > >>>>> -----Original Message-----
> > > >>>>> From: linux-pci-owner@vger.kernel.org [mailto:linux-pci-
> > > >>>>> owner@vger.kernel.org] On Behalf Of Rob Herring
> > > >>>>> Sent: Thursday, July 30, 2015 9:42 PM
> > > >>>>> To: Gabriele Paoloni
> > > >>>>> Cc: Bjorn Helgaas; arnd@arndb.de; lorenzo.pieralisi@arm.com;
> > > >> Wangzhou
> > > >>>>> (B); robh+dt@kernel.org; james.morse@arm.com;
> > Liviu.Dudau@arm.com;
> > > >>>>> linux-pci@vger.kernel.org; linux-arm-kernel@lists.infradead.org;
> > > >>>>> devicetree@vger.kernel.org; Yuanzhichang; Zhudacai; zhangjukuo;
> > > >>>>> qiuzhenfa; Liguozhu (Kenneth); Jingoo Han; Pratyush Anand
> > > >>>>> Subject: Re: [PATCH v6] PCI: Store PCIe bus address in struct
> > > >>>>> of_pci_range
> > > >>>>>
> > > >>>>> On Thu, Jul 30, 2015 at 12:34 PM, Gabriele Paoloni
> > > >>>>> <gabriele.paoloni@huawei.com> wrote:
> > > >>>>>>> -----Original Message-----
> > > >>>>>>> From: Bjorn Helgaas [mailto:bhelgaas@google.com]
> > > >>>>>>> Sent: 30 July 2015 18:15
> > > >>>>>>> On Thu, Jul 30, 2015 at 04:50:55PM +0000, Gabriele Paoloni
> > > wrote:
> > > >>>>>>>>> -----Original Message-----
> > > >>>>>>>>> From: linux-pci-owner@vger.kernel.org [mailto:linux-pci-
> > > >>>>>>>>> owner@vger.kernel.org] On Behalf Of Bjorn Helgaas
> > > >>>>>>>>> Sent: Thursday, July 30, 2015 5:15 PM
> > > >>>>>>>>> On Thu, Jul 30, 2015 at 01:52:13PM +0000, Gabriele Paoloni
> > > >> wrote:
> > > >>>>>
> > > >>>>> [...]
> > > >>>>>
> > > >>>>>>>>>> I don’t think we should rely on [CPU] addresses...what if
> > > the
> > > >>>>>>>>> intermediate
> > > >>>>>>>>>> translation layer changes the lower significant bits of
> > the
> > > >>>>> "bus
> > > >>>>>>>>> address"
> > > >>>>>>>>>> to translate into a cpu address?
> > > >>>>>>>>>
> > > >>>>>>>>> Is it really a possiblity that the lower bits could be
> > > changed?
> > > >>>>>>>>
> > > >>>>>>>> I've checked all the current deignware users DTs except
> > "pci-
> > > >>>>>>> layerscape"
> > > >>>>>>>> that I could not find:
> > > >>>>>>>> spear1310.dtsi
> > > >>>>>>>> spear1340.dtsi
> > > >>>>>>>> dra7.dtsi
> > > >>>>>>>> imx6qdl.dtsi
> > > >>>>>>>> imx6sx.dtsi
> > > >>>>>>>> keystone.dtsi
> > > >>>>>>>> exynos5440.dtsi
> > > >>>>>>>>
> > > >>>>>>>> None of them modifies the lower bits. To be more precise the
> > > >> only
> > > >>>>> guy
> > > >>>>>>>> that provides another translation layer is "dra7.dtsi":
> > > >>>>>>>> axi0
> > > >>>>>>>> http://lxr.free-
> > > >>>>> electrons.com/source/arch/arm/boot/dts/dra7.dtsi#L207
> > > >>>>>>>>
> > > >>>>>>>> axi1
> > > >>>>>>>> http://lxr.free-
> > > >>>>> electrons.com/source/arch/arm/boot/dts/dra7.dtsi#L241
> > > >>>>>>>>
> > > >>>>>>>> For this case masking the top 4bits (bits28 to 31) should
> > make
> > > >> the
> > > >>>>> job.
> > > >>>>>
> > > >>>>> IMO, we should just fix this case. After further study, I don't
> > > >> think
> > > >>>>> this is a DW issue, but rather an SOC integration issue.
> > > >>>>>
> > > >>>>> I believe you can just fixup the address in the pp->ops-
> > > >host_init
> > > >> hook.
> > > >>>>
> > > >>>> Yes I guess that I could just assign pp->(*)_mod_base to the CPU
> > > >> address
> > > >>>> in DW and mask it out in dra7xx_pcie_host_init()...
> > > >>>>
> > > >>>> Kishon, would you be ok with that?
> > > >>>
> > > >>> Initially I was using *base-mask* property from dt. Me and Arnd
> > > >> (cc'ed) had
> > > >>> this discussion [1] before we decided the current approach. It'll
> > > be
> > > >> good to
> > > >>> check with Arnd too.
> > > >>>
> > > >>> [1] ->  http://lists.infradead.org/pipermail/linux-arm-
> > kernel/2014-
> > > >> May/253528.html
> > > >>
> > > >> The problem I have here is the use of ranges does not necessarily
> > > mean
> > > >> fewer address bits are available. It can be used just for
> > > convenience
> > > >> of not putting the full address into every node's reg property.
> > And
> > > >> vice versa, there are probably plenty of cases where we have the
> > > full
> > > >> address in the nodes, but really only some of the address bits are
> > > >> decoded at the IP block. Whether the address bits are present is
> > > >> rarely cared about or known by s/w folks until you hit a problem
> > > like
> > > >> this. Given this is an isolated case ATM, I would fix it in an
> > > >> isolated way. It does not affect the binding and could be changed
> > in
> > > >> the kernel later if this becomes a common need.
> > > >>
> > > >> Rob
> > > >> --
> > > >> To unsubscribe from this list: send the line "unsubscribe linux-
> > pci"
> > > in
> > > >> the body of a message to majordomo@vger.kernel.org
> > > >> More majordomo info at  http://vger.kernel.org/majordomo-info.html


^ permalink raw reply	[flat|nested] 73+ messages in thread
* [PATCH v6] PCI: Store PCIe bus address in struct of_pci_range
@ 2015-08-06 15:06                                         ` Jingoo Han
  0 siblings, 0 replies; 73+ messages in thread
From: Jingoo Han @ 2015-08-06 15:06 UTC (permalink / raw)
  To: linux-arm-kernel

On Thursday, August 06, 2015 10:53 PM, Gabriele Paoloni wrote:
> 
> Hi all
> 
> This patch has now been moved in "[PATCH v6 1/6] PCI: designware: move calculation of bus addresses to
> DRA7xx"

To Zhou Wang,

Please send your patches to 'jingoohan1 at gmail.com', instead of 'jg1.han at samsung.com'.
Even though, I have done mainline works for Samsung SoCs, Samsung does not support me
as a maintainer. So, please don't send your patches to my Samsung e-mail.

Best regards,
Jingoo Han

> 
> commit "f4c55c5a3f7f (PCI: designware: Program ATU with untranslated address)" has been reverted in
> "[PATCH v6 3/6] PCI: designware: Add ARM64 support"
> 
> Gab
> 
> > -----Original Message-----
> > From: linux-pci-owner at vger.kernel.org [mailto:linux-pci-
> > owner at vger.kernel.org] On Behalf Of Gabriele Paoloni
> > Sent: Tuesday, August 04, 2015 11:12 AM
> > To: Jingoo Han
> > Cc: Rob Herring; Kishon Vijay Abraham I; Bjorn Helgaas; arnd at arndb.de;
> > lorenzo.pieralisi at arm.com; Wangzhou (B); robh+dt at kernel.org;
> > james.morse at arm.com; Liviu.Dudau at arm.com; linux-pci at vger.kernel.org;
> > linux-arm-kernel at lists.infradead.org; devicetree at vger.kernel.org;
> > Yuanzhichang; Zhudacai; zhangjukuo; qiuzhenfa; Liguozhu (Kenneth);
> > Pratyush Anand; Arnd Bergmann
> > Subject: RE: [PATCH v6] PCI: Store PCIe bus address in struct
> > of_pci_range
> >
> >
> >
> > > -----Original Message-----
> > > From: Jingoo Han [mailto:jingoohan1 at gmail.com]
> > > Sent: Tuesday, August 04, 2015 5:20 AM
> > > To: Gabriele Paoloni
> > > Cc: Rob Herring; Kishon Vijay Abraham I; Bjorn Helgaas; arnd at arndb.de;
> > > lorenzo.pieralisi at arm.com; Wangzhou (B); robh+dt at kernel.org;
> > > james.morse at arm.com; Liviu.Dudau at arm.com; linux-pci at vger.kernel.org;
> > > linux-arm-kernel at lists.infradead.org; devicetree at vger.kernel.org;
> > > Yuanzhichang; Zhudacai; zhangjukuo; qiuzhenfa; Liguozhu (Kenneth);
> > > Pratyush Anand; Arnd Bergmann; jingoohan1 at gmail.com
> > > Subject: Re: [PATCH v6] PCI: Store PCIe bus address in struct
> > > of_pci_range
> > >
> > > On 2015. 8. 3., at PM 8:18, Gabriele Paoloni
> > > <gabriele.paoloni@huawei.com> wrote:
> > > >
> > > > Rob, Kishon what about the following solution?....
> > > >
> > > > ---
> > > > drivers/pci/host/pci-dra7xx.c      | 12 ++++++++++++
> > > > drivers/pci/host/pcie-designware.c |  9 +++------
> > >
> > > Hi Paoloni,
> > >
> > > OK, it looks good.
> > > I want you to revert the following patch.
> > >
> > > commit "f4c55c5a3f7f (PCI: designware: Program ATU with untranslated
> > > address)"
> > >
> > > Would you remove all '*_mode_base's?
> >
> > Hi Jingoo Han,
> >
> > If I reverted the commit, in order to get designware working, dra7
> > should mask directly the CPU addresses stored in pp->cfg0_base,
> > pp->cfg1_base, pp->mem_base and pp->io_base.
> >
> > The masking would happen at this point:
> > http://lxr.free-electrons.com/source/drivers/pci/host/pcie-
> > designware.c#L493
> >
> > Now:
> > - I see that pp->cfg<0/1>_base are used in devm_ioremap before that
> >   point and nowhere else, so they should be ok.
> > - pp->mem_base is used in dw_pcie_setup_rc(); dw_pcie_setup_rc() is
> > called
> >   in dra7xx_pcie_host_init()...so here I should move the masking after
> >   dw_pcie_setup() retuned, but I think it should be ok
> > - pp->io_base is used in pci_ioremap_io() in dw_pcie_setup(). Now
> > currently
> >   in designware this is called by pci_bios_init_hw(); this is after the
> >   masking....so here we would have a the wrong value passed
> >
> > Said that if you look at "[PATCH v5 2/5] PCI: designware: Add ARM64
> > support",
> > that is the patchset where this patch is needed, you can see that
> > dw_pcie_setup() is removed and pp->io_base is used in
> > pci_remap_iospace() before
> > the masking would happen in dra7xx_pcie_host_init()...so for this
> > patchset we
> > should be good to revert the commit.
> >
> > I propose to add a new patch in the patchset "[PATCH v5 0/5] PCI: hisi:
> > Add PCIe
> > host support for HiSilicon SoC Hip05" as the one I just posted in this
> > thread (this would not revert the commit but just moves the masking to
> > dra7xx)
> >
> > I would revert the commit in "[PATCH v5 2/5] PCI: designware: Add ARM64
> > support"
> > together with the rest of designware rework.
> >
> > So we would have always a working version of designware...
> >
> > Are you ok with this?
> >
> > >
> > > Best regards,
> > > Jingoo Han
> > >
> > > > 2 files changed, 15 insertions(+), 6 deletions(-)
> > > >
> > > > diff --git a/drivers/pci/host/pci-dra7xx.c b/drivers/pci/host/pci-
> > > dra7xx.c
> > > > index 80db09e..bb2635f 100644
> > > > --- a/drivers/pci/host/pci-dra7xx.c
> > > > +++ b/drivers/pci/host/pci-dra7xx.c
> > > > @@ -61,6 +61,7 @@
> > > >
> > > > #define    PCIECTRL_DRA7XX_CONF_PHY_CS            0x010C
> > > > #define    LINK_UP                        BIT(16)
> > > > +#define CPU_TO_BUS_ADDR             0x0FFFFFFF
> > > >
> > > > struct dra7xx_pcie {
> > > >    void __iomem        *base;
> > > > @@ -138,6 +139,17 @@ static void
> > dra7xx_pcie_enable_interrupts(struct
> > > pcie_port *pp)
> > > >
> > > > static void dra7xx_pcie_host_init(struct pcie_port *pp)
> > > > {
> > > > +    if (pp->io_mod_base)
> > > > +        pp->io_mod_base &= CPU_TO_BUS_ADDR;
> > > > +
> > > > +    if (pp->mem_mod_base)
> > > > +        pp->mem_mod_base &= CPU_TO_BUS_ADDR;
> > > > +
> > > > +    if (pp->cfg0_mod_base) {
> > > > +        pp->cfg0_mod_base &= CPU_TO_BUS_ADDR;
> > > > +        pp->cfg1_mod_base &= CPU_TO_BUS_ADDR;
> > > > +    }
> > > > +
> > > >    dw_pcie_setup_rc(pp);
> > > >    dra7xx_pcie_establish_link(pp);
> > > >    if (IS_ENABLED(CONFIG_PCI_MSI))
> > > > diff --git a/drivers/pci/host/pcie-designware.c
> > > b/drivers/pci/host/pcie-designware.c
> > > > index 69486be..06c682b 100644
> > > > --- a/drivers/pci/host/pcie-designware.c
> > > > +++ b/drivers/pci/host/pcie-designware.c
> > > > @@ -416,8 +416,7 @@ int dw_pcie_host_init(struct pcie_port *pp)
> > > >            pp->io_base = range.cpu_addr;
> > > >
> > > >            /* Find the untranslated IO space address */
> > > > -            pp->io_mod_base = of_read_number(parser.range -
> > > > -                             parser.np + na, ns);
> > > > +            pp->io_mod_base = range.cpu_addr;;
> > > >        }
> > > >        if (restype == IORESOURCE_MEM) {
> > > >            of_pci_range_to_resource(&range, np, &pp->mem);
> > > > @@ -426,8 +425,7 @@ int dw_pcie_host_init(struct pcie_port *pp)
> > > >            pp->mem_bus_addr = range.pci_addr;
> > > >
> > > >            /* Find the untranslated MEM space address */
> > > > -            pp->mem_mod_base = of_read_number(parser.range -
> > > > -                              parser.np + na, ns);
> > > > +            pp->mem_mod_base = range.cpu_addr;
> > > >        }
> > > >        if (restype == 0) {
> > > >            of_pci_range_to_resource(&range, np, &pp->cfg);
> > > > @@ -437,8 +435,7 @@ int dw_pcie_host_init(struct pcie_port *pp)
> > > >            pp->cfg1_base = pp->cfg.start + pp->cfg0_size;
> > > >
> > > >            /* Find the untranslated configuration space address */
> > > > -            pp->cfg0_mod_base = of_read_number(parser.range -
> > > > -                               parser.np + na, ns);
> > > > +            pp->cfg0_mod_base = range.cpu_addr;
> > > >            pp->cfg1_mod_base = pp->cfg0_mod_base +
> > > >                        pp->cfg0_size;
> > > >        }
> > > > --
> > > > 1.9.1
> > > >
> > > >
> > > >> -----Original Message-----
> > > >> From: linux-pci-owner at vger.kernel.org [mailto:linux-pci-
> > > >> owner at vger.kernel.org] On Behalf Of Rob Herring
> > > >> Sent: Friday, July 31, 2015 5:53 PM
> > > >> To: Kishon Vijay Abraham I
> > > >> Cc: Gabriele Paoloni; Bjorn Helgaas; arnd at arndb.de;
> > > >> lorenzo.pieralisi at arm.com; Wangzhou (B); robh+dt at kernel.org;
> > > >> james.morse at arm.com; Liviu.Dudau at arm.com; linux-
> > pci at vger.kernel.org;
> > > >> linux-arm-kernel at lists.infradead.org; devicetree at vger.kernel.org;
> > > >> Yuanzhichang; Zhudacai; zhangjukuo; qiuzhenfa; Liguozhu (Kenneth);
> > > >> Jingoo Han; Pratyush Anand; Arnd Bergmann
> > > >> Subject: Re: [PATCH v6] PCI: Store PCIe bus address in struct
> > > >> of_pci_range
> > > >>
> > > >> On Fri, Jul 31, 2015 at 9:57 AM, Kishon Vijay Abraham I
> > > <kishon@ti.com>
> > > >> wrote:
> > > >>> +Arnd
> > > >>>
> > > >>> Hi,
> > > >>>
> > > >>>> On Friday 31 July 2015 07:55 PM, Gabriele Paoloni wrote:
> > > >>>> [+cc Kishon]
> > > >>>>
> > > >>>>> -----Original Message-----
> > > >>>>> From: linux-pci-owner at vger.kernel.org [mailto:linux-pci-
> > > >>>>> owner at vger.kernel.org] On Behalf Of Rob Herring
> > > >>>>> Sent: Thursday, July 30, 2015 9:42 PM
> > > >>>>> To: Gabriele Paoloni
> > > >>>>> Cc: Bjorn Helgaas; arnd at arndb.de; lorenzo.pieralisi at arm.com;
> > > >> Wangzhou
> > > >>>>> (B); robh+dt at kernel.org; james.morse at arm.com;
> > Liviu.Dudau at arm.com;
> > > >>>>> linux-pci at vger.kernel.org; linux-arm-kernel at lists.infradead.org;
> > > >>>>> devicetree at vger.kernel.org; Yuanzhichang; Zhudacai; zhangjukuo;
> > > >>>>> qiuzhenfa; Liguozhu (Kenneth); Jingoo Han; Pratyush Anand
> > > >>>>> Subject: Re: [PATCH v6] PCI: Store PCIe bus address in struct
> > > >>>>> of_pci_range
> > > >>>>>
> > > >>>>> On Thu, Jul 30, 2015 at 12:34 PM, Gabriele Paoloni
> > > >>>>> <gabriele.paoloni@huawei.com> wrote:
> > > >>>>>>> -----Original Message-----
> > > >>>>>>> From: Bjorn Helgaas [mailto:bhelgaas at google.com]
> > > >>>>>>> Sent: 30 July 2015 18:15
> > > >>>>>>> On Thu, Jul 30, 2015 at 04:50:55PM +0000, Gabriele Paoloni
> > > wrote:
> > > >>>>>>>>> -----Original Message-----
> > > >>>>>>>>> From: linux-pci-owner at vger.kernel.org [mailto:linux-pci-
> > > >>>>>>>>> owner at vger.kernel.org] On Behalf Of Bjorn Helgaas
> > > >>>>>>>>> Sent: Thursday, July 30, 2015 5:15 PM
> > > >>>>>>>>> On Thu, Jul 30, 2015 at 01:52:13PM +0000, Gabriele Paoloni
> > > >> wrote:
> > > >>>>>
> > > >>>>> [...]
> > > >>>>>
> > > >>>>>>>>>> I don?t think we should rely on [CPU] addresses...what if
> > > the
> > > >>>>>>>>> intermediate
> > > >>>>>>>>>> translation layer changes the lower significant bits of
> > the
> > > >>>>> "bus
> > > >>>>>>>>> address"
> > > >>>>>>>>>> to translate into a cpu address?
> > > >>>>>>>>>
> > > >>>>>>>>> Is it really a possiblity that the lower bits could be
> > > changed?
> > > >>>>>>>>
> > > >>>>>>>> I've checked all the current deignware users DTs except
> > "pci-
> > > >>>>>>> layerscape"
> > > >>>>>>>> that I could not find:
> > > >>>>>>>> spear1310.dtsi
> > > >>>>>>>> spear1340.dtsi
> > > >>>>>>>> dra7.dtsi
> > > >>>>>>>> imx6qdl.dtsi
> > > >>>>>>>> imx6sx.dtsi
> > > >>>>>>>> keystone.dtsi
> > > >>>>>>>> exynos5440.dtsi
> > > >>>>>>>>
> > > >>>>>>>> None of them modifies the lower bits. To be more precise the
> > > >> only
> > > >>>>> guy
> > > >>>>>>>> that provides another translation layer is "dra7.dtsi":
> > > >>>>>>>> axi0
> > > >>>>>>>> http://lxr.free-
> > > >>>>> electrons.com/source/arch/arm/boot/dts/dra7.dtsi#L207
> > > >>>>>>>>
> > > >>>>>>>> axi1
> > > >>>>>>>> http://lxr.free-
> > > >>>>> electrons.com/source/arch/arm/boot/dts/dra7.dtsi#L241
> > > >>>>>>>>
> > > >>>>>>>> For this case masking the top 4bits (bits28 to 31) should
> > make
> > > >> the
> > > >>>>> job.
> > > >>>>>
> > > >>>>> IMO, we should just fix this case. After further study, I don't
> > > >> think
> > > >>>>> this is a DW issue, but rather an SOC integration issue.
> > > >>>>>
> > > >>>>> I believe you can just fixup the address in the pp->ops-
> > > >host_init
> > > >> hook.
> > > >>>>
> > > >>>> Yes I guess that I could just assign pp->(*)_mod_base to the CPU
> > > >> address
> > > >>>> in DW and mask it out in dra7xx_pcie_host_init()...
> > > >>>>
> > > >>>> Kishon, would you be ok with that?
> > > >>>
> > > >>> Initially I was using *base-mask* property from dt. Me and Arnd
> > > >> (cc'ed) had
> > > >>> this discussion [1] before we decided the current approach. It'll
> > > be
> > > >> good to
> > > >>> check with Arnd too.
> > > >>>
> > > >>> [1] ->  http://lists.infradead.org/pipermail/linux-arm-
> > kernel/2014-
> > > >> May/253528.html
> > > >>
> > > >> The problem I have here is the use of ranges does not necessarily
> > > mean
> > > >> fewer address bits are available. It can be used just for
> > > convenience
> > > >> of not putting the full address into every node's reg property.
> > And
> > > >> vice versa, there are probably plenty of cases where we have the
> > > full
> > > >> address in the nodes, but really only some of the address bits are
> > > >> decoded at the IP block. Whether the address bits are present is
> > > >> rarely cared about or known by s/w folks until you hit a problem
> > > like
> > > >> this. Given this is an isolated case ATM, I would fix it in an
> > > >> isolated way. It does not affect the binding and could be changed
> > in
> > > >> the kernel later if this becomes a common need.
> > > >>
> > > >> Rob
> > > >> --
> > > >> To unsubscribe from this list: send the line "unsubscribe linux-
> > pci"
> > > in
> > > >> the body of a message to majordomo at vger.kernel.org
> > > >> More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 73+ messages in thread
* Re: [PATCH v6] PCI: Store PCIe bus address in struct of_pci_range
  2015-08-06 15:06                                         ` Jingoo Han
  (?)
@ 2015-08-07  5:46                                           ` Zhou Wang
  -1 siblings, 0 replies; 73+ messages in thread
From: Zhou Wang @ 2015-08-07  5:46 UTC (permalink / raw)
  To: Jingoo Han
  Cc: 'Gabriele Paoloni', 'Rob Herring',
	'Kishon Vijay Abraham I', 'Bjorn Helgaas',
	arnd, lorenzo.pieralisi, robh+dt, james.morse, Liviu.Dudau,
	linux-pci, linux-arm-kernel, devicetree, 'Yuanzhichang',
	'Zhudacai', 'zhangjukuo', 'qiuzhenfa',
	'Liguozhu (Kenneth)', 'Pratyush Anand',
	'Arnd Bergmann'

On 2015/8/6 23:06, Jingoo Han wrote:
> On Thursday, August 06, 2015 10:53 PM, Gabriele Paoloni wrote:
>>
>> Hi all
>>
>> This patch has now been moved in "[PATCH v6 1/6] PCI: designware: move calculation of bus addresses to
>> DRA7xx"
> 
> To Zhou Wang,
> 
> Please send your patches to 'jingoohan1@gmail.com', instead of 'jg1.han@samsung.com'.
> Even though, I have done mainline works for Samsung SoCs, Samsung does not support me
> as a maintainer. So, please don't send your patches to my Samsung e-mail.
> 
> Best regards,
> Jingoo Han
>

Hi Jingoo,

Sorry for that, I will add jingoohan1@gmail.com to v6 thread.

Thanks for reminding,
Zhou

>>
>> commit "f4c55c5a3f7f (PCI: designware: Program ATU with untranslated address)" has been reverted in
>> "[PATCH v6 3/6] PCI: designware: Add ARM64 support"
>>
>> Gab
>>
>>> -----Original Message-----
>>> From: linux-pci-owner@vger.kernel.org [mailto:linux-pci-
>>> owner@vger.kernel.org] On Behalf Of Gabriele Paoloni
>>> Sent: Tuesday, August 04, 2015 11:12 AM
>>> To: Jingoo Han
>>> Cc: Rob Herring; Kishon Vijay Abraham I; Bjorn Helgaas; arnd@arndb.de;
>>> lorenzo.pieralisi@arm.com; Wangzhou (B); robh+dt@kernel.org;
>>> james.morse@arm.com; Liviu.Dudau@arm.com; linux-pci@vger.kernel.org;
>>> linux-arm-kernel@lists.infradead.org; devicetree@vger.kernel.org;
>>> Yuanzhichang; Zhudacai; zhangjukuo; qiuzhenfa; Liguozhu (Kenneth);
>>> Pratyush Anand; Arnd Bergmann
>>> Subject: RE: [PATCH v6] PCI: Store PCIe bus address in struct
>>> of_pci_range
>>>
>>>
>>>
>>>> -----Original Message-----
>>>> From: Jingoo Han [mailto:jingoohan1@gmail.com]
>>>> Sent: Tuesday, August 04, 2015 5:20 AM
>>>> To: Gabriele Paoloni
>>>> Cc: Rob Herring; Kishon Vijay Abraham I; Bjorn Helgaas; arnd@arndb.de;
>>>> lorenzo.pieralisi@arm.com; Wangzhou (B); robh+dt@kernel.org;
>>>> james.morse@arm.com; Liviu.Dudau@arm.com; linux-pci@vger.kernel.org;
>>>> linux-arm-kernel@lists.infradead.org; devicetree@vger.kernel.org;
>>>> Yuanzhichang; Zhudacai; zhangjukuo; qiuzhenfa; Liguozhu (Kenneth);
>>>> Pratyush Anand; Arnd Bergmann; jingoohan1@gmail.com
>>>> Subject: Re: [PATCH v6] PCI: Store PCIe bus address in struct
>>>> of_pci_range
>>>>
>>>> On 2015. 8. 3., at PM 8:18, Gabriele Paoloni
>>>> <gabriele.paoloni@huawei.com> wrote:
>>>>>
>>>>> Rob, Kishon what about the following solution?....
>>>>>
>>>>> ---
>>>>> drivers/pci/host/pci-dra7xx.c      | 12 ++++++++++++
>>>>> drivers/pci/host/pcie-designware.c |  9 +++------
>>>>
>>>> Hi Paoloni,
>>>>
>>>> OK, it looks good.
>>>> I want you to revert the following patch.
>>>>
>>>> commit "f4c55c5a3f7f (PCI: designware: Program ATU with untranslated
>>>> address)"
>>>>
>>>> Would you remove all '*_mode_base's?
>>>
>>> Hi Jingoo Han,
>>>
>>> If I reverted the commit, in order to get designware working, dra7
>>> should mask directly the CPU addresses stored in pp->cfg0_base,
>>> pp->cfg1_base, pp->mem_base and pp->io_base.
>>>
>>> The masking would happen at this point:
>>> http://lxr.free-electrons.com/source/drivers/pci/host/pcie-
>>> designware.c#L493
>>>
>>> Now:
>>> - I see that pp->cfg<0/1>_base are used in devm_ioremap before that
>>>   point and nowhere else, so they should be ok.
>>> - pp->mem_base is used in dw_pcie_setup_rc(); dw_pcie_setup_rc() is
>>> called
>>>   in dra7xx_pcie_host_init()...so here I should move the masking after
>>>   dw_pcie_setup() retuned, but I think it should be ok
>>> - pp->io_base is used in pci_ioremap_io() in dw_pcie_setup(). Now
>>> currently
>>>   in designware this is called by pci_bios_init_hw(); this is after the
>>>   masking....so here we would have a the wrong value passed
>>>
>>> Said that if you look at "[PATCH v5 2/5] PCI: designware: Add ARM64
>>> support",
>>> that is the patchset where this patch is needed, you can see that
>>> dw_pcie_setup() is removed and pp->io_base is used in
>>> pci_remap_iospace() before
>>> the masking would happen in dra7xx_pcie_host_init()...so for this
>>> patchset we
>>> should be good to revert the commit.
>>>
>>> I propose to add a new patch in the patchset "[PATCH v5 0/5] PCI: hisi:
>>> Add PCIe
>>> host support for HiSilicon SoC Hip05" as the one I just posted in this
>>> thread (this would not revert the commit but just moves the masking to
>>> dra7xx)
>>>
>>> I would revert the commit in "[PATCH v5 2/5] PCI: designware: Add ARM64
>>> support"
>>> together with the rest of designware rework.
>>>
>>> So we would have always a working version of designware...
>>>
>>> Are you ok with this?
>>>
>>>>
>>>> Best regards,
>>>> Jingoo Han
>>>>
>>>>> 2 files changed, 15 insertions(+), 6 deletions(-)
>>>>>
>>>>> diff --git a/drivers/pci/host/pci-dra7xx.c b/drivers/pci/host/pci-
>>>> dra7xx.c
>>>>> index 80db09e..bb2635f 100644
>>>>> --- a/drivers/pci/host/pci-dra7xx.c
>>>>> +++ b/drivers/pci/host/pci-dra7xx.c
>>>>> @@ -61,6 +61,7 @@
>>>>>
>>>>> #define    PCIECTRL_DRA7XX_CONF_PHY_CS            0x010C
>>>>> #define    LINK_UP                        BIT(16)
>>>>> +#define CPU_TO_BUS_ADDR             0x0FFFFFFF
>>>>>
>>>>> struct dra7xx_pcie {
>>>>>    void __iomem        *base;
>>>>> @@ -138,6 +139,17 @@ static void
>>> dra7xx_pcie_enable_interrupts(struct
>>>> pcie_port *pp)
>>>>>
>>>>> static void dra7xx_pcie_host_init(struct pcie_port *pp)
>>>>> {
>>>>> +    if (pp->io_mod_base)
>>>>> +        pp->io_mod_base &= CPU_TO_BUS_ADDR;
>>>>> +
>>>>> +    if (pp->mem_mod_base)
>>>>> +        pp->mem_mod_base &= CPU_TO_BUS_ADDR;
>>>>> +
>>>>> +    if (pp->cfg0_mod_base) {
>>>>> +        pp->cfg0_mod_base &= CPU_TO_BUS_ADDR;
>>>>> +        pp->cfg1_mod_base &= CPU_TO_BUS_ADDR;
>>>>> +    }
>>>>> +
>>>>>    dw_pcie_setup_rc(pp);
>>>>>    dra7xx_pcie_establish_link(pp);
>>>>>    if (IS_ENABLED(CONFIG_PCI_MSI))
>>>>> diff --git a/drivers/pci/host/pcie-designware.c
>>>> b/drivers/pci/host/pcie-designware.c
>>>>> index 69486be..06c682b 100644
>>>>> --- a/drivers/pci/host/pcie-designware.c
>>>>> +++ b/drivers/pci/host/pcie-designware.c
>>>>> @@ -416,8 +416,7 @@ int dw_pcie_host_init(struct pcie_port *pp)
>>>>>            pp->io_base = range.cpu_addr;
>>>>>
>>>>>            /* Find the untranslated IO space address */
>>>>> -            pp->io_mod_base = of_read_number(parser.range -
>>>>> -                             parser.np + na, ns);
>>>>> +            pp->io_mod_base = range.cpu_addr;;
>>>>>        }
>>>>>        if (restype == IORESOURCE_MEM) {
>>>>>            of_pci_range_to_resource(&range, np, &pp->mem);
>>>>> @@ -426,8 +425,7 @@ int dw_pcie_host_init(struct pcie_port *pp)
>>>>>            pp->mem_bus_addr = range.pci_addr;
>>>>>
>>>>>            /* Find the untranslated MEM space address */
>>>>> -            pp->mem_mod_base = of_read_number(parser.range -
>>>>> -                              parser.np + na, ns);
>>>>> +            pp->mem_mod_base = range.cpu_addr;
>>>>>        }
>>>>>        if (restype == 0) {
>>>>>            of_pci_range_to_resource(&range, np, &pp->cfg);
>>>>> @@ -437,8 +435,7 @@ int dw_pcie_host_init(struct pcie_port *pp)
>>>>>            pp->cfg1_base = pp->cfg.start + pp->cfg0_size;
>>>>>
>>>>>            /* Find the untranslated configuration space address */
>>>>> -            pp->cfg0_mod_base = of_read_number(parser.range -
>>>>> -                               parser.np + na, ns);
>>>>> +            pp->cfg0_mod_base = range.cpu_addr;
>>>>>            pp->cfg1_mod_base = pp->cfg0_mod_base +
>>>>>                        pp->cfg0_size;
>>>>>        }
>>>>> --
>>>>> 1.9.1
>>>>>
>>>>>
>>>>>> -----Original Message-----
>>>>>> From: linux-pci-owner@vger.kernel.org [mailto:linux-pci-
>>>>>> owner@vger.kernel.org] On Behalf Of Rob Herring
>>>>>> Sent: Friday, July 31, 2015 5:53 PM
>>>>>> To: Kishon Vijay Abraham I
>>>>>> Cc: Gabriele Paoloni; Bjorn Helgaas; arnd@arndb.de;
>>>>>> lorenzo.pieralisi@arm.com; Wangzhou (B); robh+dt@kernel.org;
>>>>>> james.morse@arm.com; Liviu.Dudau@arm.com; linux-
>>> pci@vger.kernel.org;
>>>>>> linux-arm-kernel@lists.infradead.org; devicetree@vger.kernel.org;
>>>>>> Yuanzhichang; Zhudacai; zhangjukuo; qiuzhenfa; Liguozhu (Kenneth);
>>>>>> Jingoo Han; Pratyush Anand; Arnd Bergmann
>>>>>> Subject: Re: [PATCH v6] PCI: Store PCIe bus address in struct
>>>>>> of_pci_range
>>>>>>
>>>>>> On Fri, Jul 31, 2015 at 9:57 AM, Kishon Vijay Abraham I
>>>> <kishon@ti.com>
>>>>>> wrote:
>>>>>>> +Arnd
>>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>>> On Friday 31 July 2015 07:55 PM, Gabriele Paoloni wrote:
>>>>>>>> [+cc Kishon]
>>>>>>>>
>>>>>>>>> -----Original Message-----
>>>>>>>>> From: linux-pci-owner@vger.kernel.org [mailto:linux-pci-
>>>>>>>>> owner@vger.kernel.org] On Behalf Of Rob Herring
>>>>>>>>> Sent: Thursday, July 30, 2015 9:42 PM
>>>>>>>>> To: Gabriele Paoloni
>>>>>>>>> Cc: Bjorn Helgaas; arnd@arndb.de; lorenzo.pieralisi@arm.com;
>>>>>> Wangzhou
>>>>>>>>> (B); robh+dt@kernel.org; james.morse@arm.com;
>>> Liviu.Dudau@arm.com;
>>>>>>>>> linux-pci@vger.kernel.org; linux-arm-kernel@lists.infradead.org;
>>>>>>>>> devicetree@vger.kernel.org; Yuanzhichang; Zhudacai; zhangjukuo;
>>>>>>>>> qiuzhenfa; Liguozhu (Kenneth); Jingoo Han; Pratyush Anand
>>>>>>>>> Subject: Re: [PATCH v6] PCI: Store PCIe bus address in struct
>>>>>>>>> of_pci_range
>>>>>>>>>
>>>>>>>>> On Thu, Jul 30, 2015 at 12:34 PM, Gabriele Paoloni
>>>>>>>>> <gabriele.paoloni@huawei.com> wrote:
>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>> From: Bjorn Helgaas [mailto:bhelgaas@google.com]
>>>>>>>>>>> Sent: 30 July 2015 18:15
>>>>>>>>>>> On Thu, Jul 30, 2015 at 04:50:55PM +0000, Gabriele Paoloni
>>>> wrote:
>>>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>>>> From: linux-pci-owner@vger.kernel.org [mailto:linux-pci-
>>>>>>>>>>>>> owner@vger.kernel.org] On Behalf Of Bjorn Helgaas
>>>>>>>>>>>>> Sent: Thursday, July 30, 2015 5:15 PM
>>>>>>>>>>>>> On Thu, Jul 30, 2015 at 01:52:13PM +0000, Gabriele Paoloni
>>>>>> wrote:
>>>>>>>>>
>>>>>>>>> [...]
>>>>>>>>>
>>>>>>>>>>>>>> I don’t think we should rely on [CPU] addresses...what if
>>>> the
>>>>>>>>>>>>> intermediate
>>>>>>>>>>>>>> translation layer changes the lower significant bits of
>>> the
>>>>>>>>> "bus
>>>>>>>>>>>>> address"
>>>>>>>>>>>>>> to translate into a cpu address?
>>>>>>>>>>>>>
>>>>>>>>>>>>> Is it really a possiblity that the lower bits could be
>>>> changed?
>>>>>>>>>>>>
>>>>>>>>>>>> I've checked all the current deignware users DTs except
>>> "pci-
>>>>>>>>>>> layerscape"
>>>>>>>>>>>> that I could not find:
>>>>>>>>>>>> spear1310.dtsi
>>>>>>>>>>>> spear1340.dtsi
>>>>>>>>>>>> dra7.dtsi
>>>>>>>>>>>> imx6qdl.dtsi
>>>>>>>>>>>> imx6sx.dtsi
>>>>>>>>>>>> keystone.dtsi
>>>>>>>>>>>> exynos5440.dtsi
>>>>>>>>>>>>
>>>>>>>>>>>> None of them modifies the lower bits. To be more precise the
>>>>>> only
>>>>>>>>> guy
>>>>>>>>>>>> that provides another translation layer is "dra7.dtsi":
>>>>>>>>>>>> axi0
>>>>>>>>>>>> http://lxr.free-
>>>>>>>>> electrons.com/source/arch/arm/boot/dts/dra7.dtsi#L207
>>>>>>>>>>>>
>>>>>>>>>>>> axi1
>>>>>>>>>>>> http://lxr.free-
>>>>>>>>> electrons.com/source/arch/arm/boot/dts/dra7.dtsi#L241
>>>>>>>>>>>>
>>>>>>>>>>>> For this case masking the top 4bits (bits28 to 31) should
>>> make
>>>>>> the
>>>>>>>>> job.
>>>>>>>>>
>>>>>>>>> IMO, we should just fix this case. After further study, I don't
>>>>>> think
>>>>>>>>> this is a DW issue, but rather an SOC integration issue.
>>>>>>>>>
>>>>>>>>> I believe you can just fixup the address in the pp->ops-
>>>>> host_init
>>>>>> hook.
>>>>>>>>
>>>>>>>> Yes I guess that I could just assign pp->(*)_mod_base to the CPU
>>>>>> address
>>>>>>>> in DW and mask it out in dra7xx_pcie_host_init()...
>>>>>>>>
>>>>>>>> Kishon, would you be ok with that?
>>>>>>>
>>>>>>> Initially I was using *base-mask* property from dt. Me and Arnd
>>>>>> (cc'ed) had
>>>>>>> this discussion [1] before we decided the current approach. It'll
>>>> be
>>>>>> good to
>>>>>>> check with Arnd too.
>>>>>>>
>>>>>>> [1] ->  http://lists.infradead.org/pipermail/linux-arm-
>>> kernel/2014-
>>>>>> May/253528.html
>>>>>>
>>>>>> The problem I have here is the use of ranges does not necessarily
>>>> mean
>>>>>> fewer address bits are available. It can be used just for
>>>> convenience
>>>>>> of not putting the full address into every node's reg property.
>>> And
>>>>>> vice versa, there are probably plenty of cases where we have the
>>>> full
>>>>>> address in the nodes, but really only some of the address bits are
>>>>>> decoded at the IP block. Whether the address bits are present is
>>>>>> rarely cared about or known by s/w folks until you hit a problem
>>>> like
>>>>>> this. Given this is an isolated case ATM, I would fix it in an
>>>>>> isolated way. It does not affect the binding and could be changed
>>> in
>>>>>> the kernel later if this becomes a common need.
>>>>>>
>>>>>> Rob
>>>>>> --
>>>>>> To unsubscribe from this list: send the line "unsubscribe linux-
>>> pci"
>>>> in
>>>>>> the body of a message to majordomo@vger.kernel.org
>>>>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 
> 
> .
> 

^ permalink raw reply	[flat|nested] 73+ messages in thread
* Re: [PATCH v6] PCI: Store PCIe bus address in struct of_pci_range
@ 2015-08-07  5:46                                           ` Zhou Wang
  0 siblings, 0 replies; 73+ messages in thread
From: Zhou Wang @ 2015-08-07  5:46 UTC (permalink / raw)
  To: Jingoo Han
  Cc: 'Gabriele Paoloni', 'Rob Herring',
	'Kishon Vijay Abraham I', 'Bjorn Helgaas',
	arnd, lorenzo.pieralisi, robh+dt, james.morse, Liviu.Dudau,
	linux-pci, linux-arm-kernel, devicetree, 'Yuanzhichang',
	'Zhudacai', 'zhangjukuo', 'qiuzhenfa',
	'Liguozhu (Kenneth)', 'Pratyush Anand',
	'Arnd Bergmann'

On 2015/8/6 23:06, Jingoo Han wrote:
> On Thursday, August 06, 2015 10:53 PM, Gabriele Paoloni wrote:
>>
>> Hi all
>>
>> This patch has now been moved in "[PATCH v6 1/6] PCI: designware: move calculation of bus addresses to
>> DRA7xx"
> 
> To Zhou Wang,
> 
> Please send your patches to 'jingoohan1@gmail.com', instead of 'jg1.han@samsung.com'.
> Even though, I have done mainline works for Samsung SoCs, Samsung does not support me
> as a maintainer. So, please don't send your patches to my Samsung e-mail.
> 
> Best regards,
> Jingoo Han
>

Hi Jingoo,

Sorry for that, I will add jingoohan1@gmail.com to v6 thread.

Thanks for reminding,
Zhou

>>
>> commit "f4c55c5a3f7f (PCI: designware: Program ATU with untranslated address)" has been reverted in
>> "[PATCH v6 3/6] PCI: designware: Add ARM64 support"
>>
>> Gab
>>
>>> -----Original Message-----
>>> From: linux-pci-owner@vger.kernel.org [mailto:linux-pci-
>>> owner@vger.kernel.org] On Behalf Of Gabriele Paoloni
>>> Sent: Tuesday, August 04, 2015 11:12 AM
>>> To: Jingoo Han
>>> Cc: Rob Herring; Kishon Vijay Abraham I; Bjorn Helgaas; arnd@arndb.de;
>>> lorenzo.pieralisi@arm.com; Wangzhou (B); robh+dt@kernel.org;
>>> james.morse@arm.com; Liviu.Dudau@arm.com; linux-pci@vger.kernel.org;
>>> linux-arm-kernel@lists.infradead.org; devicetree@vger.kernel.org;
>>> Yuanzhichang; Zhudacai; zhangjukuo; qiuzhenfa; Liguozhu (Kenneth);
>>> Pratyush Anand; Arnd Bergmann
>>> Subject: RE: [PATCH v6] PCI: Store PCIe bus address in struct
>>> of_pci_range
>>>
>>>
>>>
>>>> -----Original Message-----
>>>> From: Jingoo Han [mailto:jingoohan1@gmail.com]
>>>> Sent: Tuesday, August 04, 2015 5:20 AM
>>>> To: Gabriele Paoloni
>>>> Cc: Rob Herring; Kishon Vijay Abraham I; Bjorn Helgaas; arnd@arndb.de;
>>>> lorenzo.pieralisi@arm.com; Wangzhou (B); robh+dt@kernel.org;
>>>> james.morse@arm.com; Liviu.Dudau@arm.com; linux-pci@vger.kernel.org;
>>>> linux-arm-kernel@lists.infradead.org; devicetree@vger.kernel.org;
>>>> Yuanzhichang; Zhudacai; zhangjukuo; qiuzhenfa; Liguozhu (Kenneth);
>>>> Pratyush Anand; Arnd Bergmann; jingoohan1@gmail.com
>>>> Subject: Re: [PATCH v6] PCI: Store PCIe bus address in struct
>>>> of_pci_range
>>>>
>>>> On 2015. 8. 3., at PM 8:18, Gabriele Paoloni
>>>> <gabriele.paoloni@huawei.com> wrote:
>>>>>
>>>>> Rob, Kishon what about the following solution?....
>>>>>
>>>>> ---
>>>>> drivers/pci/host/pci-dra7xx.c      | 12 ++++++++++++
>>>>> drivers/pci/host/pcie-designware.c |  9 +++------
>>>>
>>>> Hi Paoloni,
>>>>
>>>> OK, it looks good.
>>>> I want you to revert the following patch.
>>>>
>>>> commit "f4c55c5a3f7f (PCI: designware: Program ATU with untranslated
>>>> address)"
>>>>
>>>> Would you remove all '*_mode_base's?
>>>
>>> Hi Jingoo Han,
>>>
>>> If I reverted the commit, in order to get designware working, dra7
>>> should mask directly the CPU addresses stored in pp->cfg0_base,
>>> pp->cfg1_base, pp->mem_base and pp->io_base.
>>>
>>> The masking would happen at this point:
>>> http://lxr.free-electrons.com/source/drivers/pci/host/pcie-
>>> designware.c#L493
>>>
>>> Now:
>>> - I see that pp->cfg<0/1>_base are used in devm_ioremap before that
>>>   point and nowhere else, so they should be ok.
>>> - pp->mem_base is used in dw_pcie_setup_rc(); dw_pcie_setup_rc() is
>>> called
>>>   in dra7xx_pcie_host_init()...so here I should move the masking after
>>>   dw_pcie_setup() retuned, but I think it should be ok
>>> - pp->io_base is used in pci_ioremap_io() in dw_pcie_setup(). Now
>>> currently
>>>   in designware this is called by pci_bios_init_hw(); this is after the
>>>   masking....so here we would have a the wrong value passed
>>>
>>> Said that if you look at "[PATCH v5 2/5] PCI: designware: Add ARM64
>>> support",
>>> that is the patchset where this patch is needed, you can see that
>>> dw_pcie_setup() is removed and pp->io_base is used in
>>> pci_remap_iospace() before
>>> the masking would happen in dra7xx_pcie_host_init()...so for this
>>> patchset we
>>> should be good to revert the commit.
>>>
>>> I propose to add a new patch in the patchset "[PATCH v5 0/5] PCI: hisi:
>>> Add PCIe
>>> host support for HiSilicon SoC Hip05" as the one I just posted in this
>>> thread (this would not revert the commit but just moves the masking to
>>> dra7xx)
>>>
>>> I would revert the commit in "[PATCH v5 2/5] PCI: designware: Add ARM64
>>> support"
>>> together with the rest of designware rework.
>>>
>>> So we would have always a working version of designware...
>>>
>>> Are you ok with this?
>>>
>>>>
>>>> Best regards,
>>>> Jingoo Han
>>>>
>>>>> 2 files changed, 15 insertions(+), 6 deletions(-)
>>>>>
>>>>> diff --git a/drivers/pci/host/pci-dra7xx.c b/drivers/pci/host/pci-
>>>> dra7xx.c
>>>>> index 80db09e..bb2635f 100644
>>>>> --- a/drivers/pci/host/pci-dra7xx.c
>>>>> +++ b/drivers/pci/host/pci-dra7xx.c
>>>>> @@ -61,6 +61,7 @@
>>>>>
>>>>> #define    PCIECTRL_DRA7XX_CONF_PHY_CS            0x010C
>>>>> #define    LINK_UP                        BIT(16)
>>>>> +#define CPU_TO_BUS_ADDR             0x0FFFFFFF
>>>>>
>>>>> struct dra7xx_pcie {
>>>>>    void __iomem        *base;
>>>>> @@ -138,6 +139,17 @@ static void
>>> dra7xx_pcie_enable_interrupts(struct
>>>> pcie_port *pp)
>>>>>
>>>>> static void dra7xx_pcie_host_init(struct pcie_port *pp)
>>>>> {
>>>>> +    if (pp->io_mod_base)
>>>>> +        pp->io_mod_base &= CPU_TO_BUS_ADDR;
>>>>> +
>>>>> +    if (pp->mem_mod_base)
>>>>> +        pp->mem_mod_base &= CPU_TO_BUS_ADDR;
>>>>> +
>>>>> +    if (pp->cfg0_mod_base) {
>>>>> +        pp->cfg0_mod_base &= CPU_TO_BUS_ADDR;
>>>>> +        pp->cfg1_mod_base &= CPU_TO_BUS_ADDR;
>>>>> +    }
>>>>> +
>>>>>    dw_pcie_setup_rc(pp);
>>>>>    dra7xx_pcie_establish_link(pp);
>>>>>    if (IS_ENABLED(CONFIG_PCI_MSI))
>>>>> diff --git a/drivers/pci/host/pcie-designware.c
>>>> b/drivers/pci/host/pcie-designware.c
>>>>> index 69486be..06c682b 100644
>>>>> --- a/drivers/pci/host/pcie-designware.c
>>>>> +++ b/drivers/pci/host/pcie-designware.c
>>>>> @@ -416,8 +416,7 @@ int dw_pcie_host_init(struct pcie_port *pp)
>>>>>            pp->io_base = range.cpu_addr;
>>>>>
>>>>>            /* Find the untranslated IO space address */
>>>>> -            pp->io_mod_base = of_read_number(parser.range -
>>>>> -                             parser.np + na, ns);
>>>>> +            pp->io_mod_base = range.cpu_addr;;
>>>>>        }
>>>>>        if (restype == IORESOURCE_MEM) {
>>>>>            of_pci_range_to_resource(&range, np, &pp->mem);
>>>>> @@ -426,8 +425,7 @@ int dw_pcie_host_init(struct pcie_port *pp)
>>>>>            pp->mem_bus_addr = range.pci_addr;
>>>>>
>>>>>            /* Find the untranslated MEM space address */
>>>>> -            pp->mem_mod_base = of_read_number(parser.range -
>>>>> -                              parser.np + na, ns);
>>>>> +            pp->mem_mod_base = range.cpu_addr;
>>>>>        }
>>>>>        if (restype == 0) {
>>>>>            of_pci_range_to_resource(&range, np, &pp->cfg);
>>>>> @@ -437,8 +435,7 @@ int dw_pcie_host_init(struct pcie_port *pp)
>>>>>            pp->cfg1_base = pp->cfg.start + pp->cfg0_size;
>>>>>
>>>>>            /* Find the untranslated configuration space address */
>>>>> -            pp->cfg0_mod_base = of_read_number(parser.range -
>>>>> -                               parser.np + na, ns);
>>>>> +            pp->cfg0_mod_base = range.cpu_addr;
>>>>>            pp->cfg1_mod_base = pp->cfg0_mod_base +
>>>>>                        pp->cfg0_size;
>>>>>        }
>>>>> --
>>>>> 1.9.1
>>>>>
>>>>>
>>>>>> -----Original Message-----
>>>>>> From: linux-pci-owner@vger.kernel.org [mailto:linux-pci-
>>>>>> owner@vger.kernel.org] On Behalf Of Rob Herring
>>>>>> Sent: Friday, July 31, 2015 5:53 PM
>>>>>> To: Kishon Vijay Abraham I
>>>>>> Cc: Gabriele Paoloni; Bjorn Helgaas; arnd@arndb.de;
>>>>>> lorenzo.pieralisi@arm.com; Wangzhou (B); robh+dt@kernel.org;
>>>>>> james.morse@arm.com; Liviu.Dudau@arm.com; linux-
>>> pci@vger.kernel.org;
>>>>>> linux-arm-kernel@lists.infradead.org; devicetree@vger.kernel.org;
>>>>>> Yuanzhichang; Zhudacai; zhangjukuo; qiuzhenfa; Liguozhu (Kenneth);
>>>>>> Jingoo Han; Pratyush Anand; Arnd Bergmann
>>>>>> Subject: Re: [PATCH v6] PCI: Store PCIe bus address in struct
>>>>>> of_pci_range
>>>>>>
>>>>>> On Fri, Jul 31, 2015 at 9:57 AM, Kishon Vijay Abraham I
>>>> <kishon@ti.com>
>>>>>> wrote:
>>>>>>> +Arnd
>>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>>> On Friday 31 July 2015 07:55 PM, Gabriele Paoloni wrote:
>>>>>>>> [+cc Kishon]
>>>>>>>>
>>>>>>>>> -----Original Message-----
>>>>>>>>> From: linux-pci-owner@vger.kernel.org [mailto:linux-pci-
>>>>>>>>> owner@vger.kernel.org] On Behalf Of Rob Herring
>>>>>>>>> Sent: Thursday, July 30, 2015 9:42 PM
>>>>>>>>> To: Gabriele Paoloni
>>>>>>>>> Cc: Bjorn Helgaas; arnd@arndb.de; lorenzo.pieralisi@arm.com;
>>>>>> Wangzhou
>>>>>>>>> (B); robh+dt@kernel.org; james.morse@arm.com;
>>> Liviu.Dudau@arm.com;
>>>>>>>>> linux-pci@vger.kernel.org; linux-arm-kernel@lists.infradead.org;
>>>>>>>>> devicetree@vger.kernel.org; Yuanzhichang; Zhudacai; zhangjukuo;
>>>>>>>>> qiuzhenfa; Liguozhu (Kenneth); Jingoo Han; Pratyush Anand
>>>>>>>>> Subject: Re: [PATCH v6] PCI: Store PCIe bus address in struct
>>>>>>>>> of_pci_range
>>>>>>>>>
>>>>>>>>> On Thu, Jul 30, 2015 at 12:34 PM, Gabriele Paoloni
>>>>>>>>> <gabriele.paoloni@huawei.com> wrote:
>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>> From: Bjorn Helgaas [mailto:bhelgaas@google.com]
>>>>>>>>>>> Sent: 30 July 2015 18:15
>>>>>>>>>>> On Thu, Jul 30, 2015 at 04:50:55PM +0000, Gabriele Paoloni
>>>> wrote:
>>>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>>>> From: linux-pci-owner@vger.kernel.org [mailto:linux-pci-
>>>>>>>>>>>>> owner@vger.kernel.org] On Behalf Of Bjorn Helgaas
>>>>>>>>>>>>> Sent: Thursday, July 30, 2015 5:15 PM
>>>>>>>>>>>>> On Thu, Jul 30, 2015 at 01:52:13PM +0000, Gabriele Paoloni
>>>>>> wrote:
>>>>>>>>>
>>>>>>>>> [...]
>>>>>>>>>
>>>>>>>>>>>>>> I don’t think we should rely on [CPU] addresses...what if
>>>> the
>>>>>>>>>>>>> intermediate
>>>>>>>>>>>>>> translation layer changes the lower significant bits of
>>> the
>>>>>>>>> "bus
>>>>>>>>>>>>> address"
>>>>>>>>>>>>>> to translate into a cpu address?
>>>>>>>>>>>>>
>>>>>>>>>>>>> Is it really a possiblity that the lower bits could be
>>>> changed?
>>>>>>>>>>>>
>>>>>>>>>>>> I've checked all the current deignware users DTs except
>>> "pci-
>>>>>>>>>>> layerscape"
>>>>>>>>>>>> that I could not find:
>>>>>>>>>>>> spear1310.dtsi
>>>>>>>>>>>> spear1340.dtsi
>>>>>>>>>>>> dra7.dtsi
>>>>>>>>>>>> imx6qdl.dtsi
>>>>>>>>>>>> imx6sx.dtsi
>>>>>>>>>>>> keystone.dtsi
>>>>>>>>>>>> exynos5440.dtsi
>>>>>>>>>>>>
>>>>>>>>>>>> None of them modifies the lower bits. To be more precise the
>>>>>> only
>>>>>>>>> guy
>>>>>>>>>>>> that provides another translation layer is "dra7.dtsi":
>>>>>>>>>>>> axi0
>>>>>>>>>>>> http://lxr.free-
>>>>>>>>> electrons.com/source/arch/arm/boot/dts/dra7.dtsi#L207
>>>>>>>>>>>>
>>>>>>>>>>>> axi1
>>>>>>>>>>>> http://lxr.free-
>>>>>>>>> electrons.com/source/arch/arm/boot/dts/dra7.dtsi#L241
>>>>>>>>>>>>
>>>>>>>>>>>> For this case masking the top 4bits (bits28 to 31) should
>>> make
>>>>>> the
>>>>>>>>> job.
>>>>>>>>>
>>>>>>>>> IMO, we should just fix this case. After further study, I don't
>>>>>> think
>>>>>>>>> this is a DW issue, but rather an SOC integration issue.
>>>>>>>>>
>>>>>>>>> I believe you can just fixup the address in the pp->ops-
>>>>> host_init
>>>>>> hook.
>>>>>>>>
>>>>>>>> Yes I guess that I could just assign pp->(*)_mod_base to the CPU
>>>>>> address
>>>>>>>> in DW and mask it out in dra7xx_pcie_host_init()...
>>>>>>>>
>>>>>>>> Kishon, would you be ok with that?
>>>>>>>
>>>>>>> Initially I was using *base-mask* property from dt. Me and Arnd
>>>>>> (cc'ed) had
>>>>>>> this discussion [1] before we decided the current approach. It'll
>>>> be
>>>>>> good to
>>>>>>> check with Arnd too.
>>>>>>>
>>>>>>> [1] ->  http://lists.infradead.org/pipermail/linux-arm-
>>> kernel/2014-
>>>>>> May/253528.html
>>>>>>
>>>>>> The problem I have here is the use of ranges does not necessarily
>>>> mean
>>>>>> fewer address bits are available. It can be used just for
>>>> convenience
>>>>>> of not putting the full address into every node's reg property.
>>> And
>>>>>> vice versa, there are probably plenty of cases where we have the
>>>> full
>>>>>> address in the nodes, but really only some of the address bits are
>>>>>> decoded at the IP block. Whether the address bits are present is
>>>>>> rarely cared about or known by s/w folks until you hit a problem
>>>> like
>>>>>> this. Given this is an isolated case ATM, I would fix it in an
>>>>>> isolated way. It does not affect the binding and could be changed
>>> in
>>>>>> the kernel later if this becomes a common need.
>>>>>>
>>>>>> Rob
>>>>>> --
>>>>>> To unsubscribe from this list: send the line "unsubscribe linux-
>>> pci"
>>>> in
>>>>>> the body of a message to majordomo@vger.kernel.org
>>>>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 
> 
> .
> 



^ permalink raw reply	[flat|nested] 73+ messages in thread
* [PATCH v6] PCI: Store PCIe bus address in struct of_pci_range
@ 2015-08-07  5:46                                           ` Zhou Wang
  0 siblings, 0 replies; 73+ messages in thread
From: Zhou Wang @ 2015-08-07  5:46 UTC (permalink / raw)
  To: linux-arm-kernel

On 2015/8/6 23:06, Jingoo Han wrote:
> On Thursday, August 06, 2015 10:53 PM, Gabriele Paoloni wrote:
>>
>> Hi all
>>
>> This patch has now been moved in "[PATCH v6 1/6] PCI: designware: move calculation of bus addresses to
>> DRA7xx"
> 
> To Zhou Wang,
> 
> Please send your patches to 'jingoohan1 at gmail.com', instead of 'jg1.han at samsung.com'.
> Even though, I have done mainline works for Samsung SoCs, Samsung does not support me
> as a maintainer. So, please don't send your patches to my Samsung e-mail.
> 
> Best regards,
> Jingoo Han
>

Hi Jingoo,

Sorry for that, I will add jingoohan1 at gmail.com to v6 thread.

Thanks for reminding,
Zhou

>>
>> commit "f4c55c5a3f7f (PCI: designware: Program ATU with untranslated address)" has been reverted in
>> "[PATCH v6 3/6] PCI: designware: Add ARM64 support"
>>
>> Gab
>>
>>> -----Original Message-----
>>> From: linux-pci-owner at vger.kernel.org [mailto:linux-pci-
>>> owner at vger.kernel.org] On Behalf Of Gabriele Paoloni
>>> Sent: Tuesday, August 04, 2015 11:12 AM
>>> To: Jingoo Han
>>> Cc: Rob Herring; Kishon Vijay Abraham I; Bjorn Helgaas; arnd at arndb.de;
>>> lorenzo.pieralisi at arm.com; Wangzhou (B); robh+dt at kernel.org;
>>> james.morse at arm.com; Liviu.Dudau at arm.com; linux-pci at vger.kernel.org;
>>> linux-arm-kernel at lists.infradead.org; devicetree at vger.kernel.org;
>>> Yuanzhichang; Zhudacai; zhangjukuo; qiuzhenfa; Liguozhu (Kenneth);
>>> Pratyush Anand; Arnd Bergmann
>>> Subject: RE: [PATCH v6] PCI: Store PCIe bus address in struct
>>> of_pci_range
>>>
>>>
>>>
>>>> -----Original Message-----
>>>> From: Jingoo Han [mailto:jingoohan1 at gmail.com]
>>>> Sent: Tuesday, August 04, 2015 5:20 AM
>>>> To: Gabriele Paoloni
>>>> Cc: Rob Herring; Kishon Vijay Abraham I; Bjorn Helgaas; arnd at arndb.de;
>>>> lorenzo.pieralisi at arm.com; Wangzhou (B); robh+dt at kernel.org;
>>>> james.morse at arm.com; Liviu.Dudau at arm.com; linux-pci at vger.kernel.org;
>>>> linux-arm-kernel at lists.infradead.org; devicetree at vger.kernel.org;
>>>> Yuanzhichang; Zhudacai; zhangjukuo; qiuzhenfa; Liguozhu (Kenneth);
>>>> Pratyush Anand; Arnd Bergmann; jingoohan1 at gmail.com
>>>> Subject: Re: [PATCH v6] PCI: Store PCIe bus address in struct
>>>> of_pci_range
>>>>
>>>> On 2015. 8. 3., at PM 8:18, Gabriele Paoloni
>>>> <gabriele.paoloni@huawei.com> wrote:
>>>>>
>>>>> Rob, Kishon what about the following solution?....
>>>>>
>>>>> ---
>>>>> drivers/pci/host/pci-dra7xx.c      | 12 ++++++++++++
>>>>> drivers/pci/host/pcie-designware.c |  9 +++------
>>>>
>>>> Hi Paoloni,
>>>>
>>>> OK, it looks good.
>>>> I want you to revert the following patch.
>>>>
>>>> commit "f4c55c5a3f7f (PCI: designware: Program ATU with untranslated
>>>> address)"
>>>>
>>>> Would you remove all '*_mode_base's?
>>>
>>> Hi Jingoo Han,
>>>
>>> If I reverted the commit, in order to get designware working, dra7
>>> should mask directly the CPU addresses stored in pp->cfg0_base,
>>> pp->cfg1_base, pp->mem_base and pp->io_base.
>>>
>>> The masking would happen at this point:
>>> http://lxr.free-electrons.com/source/drivers/pci/host/pcie-
>>> designware.c#L493
>>>
>>> Now:
>>> - I see that pp->cfg<0/1>_base are used in devm_ioremap before that
>>>   point and nowhere else, so they should be ok.
>>> - pp->mem_base is used in dw_pcie_setup_rc(); dw_pcie_setup_rc() is
>>> called
>>>   in dra7xx_pcie_host_init()...so here I should move the masking after
>>>   dw_pcie_setup() retuned, but I think it should be ok
>>> - pp->io_base is used in pci_ioremap_io() in dw_pcie_setup(). Now
>>> currently
>>>   in designware this is called by pci_bios_init_hw(); this is after the
>>>   masking....so here we would have a the wrong value passed
>>>
>>> Said that if you look at "[PATCH v5 2/5] PCI: designware: Add ARM64
>>> support",
>>> that is the patchset where this patch is needed, you can see that
>>> dw_pcie_setup() is removed and pp->io_base is used in
>>> pci_remap_iospace() before
>>> the masking would happen in dra7xx_pcie_host_init()...so for this
>>> patchset we
>>> should be good to revert the commit.
>>>
>>> I propose to add a new patch in the patchset "[PATCH v5 0/5] PCI: hisi:
>>> Add PCIe
>>> host support for HiSilicon SoC Hip05" as the one I just posted in this
>>> thread (this would not revert the commit but just moves the masking to
>>> dra7xx)
>>>
>>> I would revert the commit in "[PATCH v5 2/5] PCI: designware: Add ARM64
>>> support"
>>> together with the rest of designware rework.
>>>
>>> So we would have always a working version of designware...
>>>
>>> Are you ok with this?
>>>
>>>>
>>>> Best regards,
>>>> Jingoo Han
>>>>
>>>>> 2 files changed, 15 insertions(+), 6 deletions(-)
>>>>>
>>>>> diff --git a/drivers/pci/host/pci-dra7xx.c b/drivers/pci/host/pci-
>>>> dra7xx.c
>>>>> index 80db09e..bb2635f 100644
>>>>> --- a/drivers/pci/host/pci-dra7xx.c
>>>>> +++ b/drivers/pci/host/pci-dra7xx.c
>>>>> @@ -61,6 +61,7 @@
>>>>>
>>>>> #define    PCIECTRL_DRA7XX_CONF_PHY_CS            0x010C
>>>>> #define    LINK_UP                        BIT(16)
>>>>> +#define CPU_TO_BUS_ADDR             0x0FFFFFFF
>>>>>
>>>>> struct dra7xx_pcie {
>>>>>    void __iomem        *base;
>>>>> @@ -138,6 +139,17 @@ static void
>>> dra7xx_pcie_enable_interrupts(struct
>>>> pcie_port *pp)
>>>>>
>>>>> static void dra7xx_pcie_host_init(struct pcie_port *pp)
>>>>> {
>>>>> +    if (pp->io_mod_base)
>>>>> +        pp->io_mod_base &= CPU_TO_BUS_ADDR;
>>>>> +
>>>>> +    if (pp->mem_mod_base)
>>>>> +        pp->mem_mod_base &= CPU_TO_BUS_ADDR;
>>>>> +
>>>>> +    if (pp->cfg0_mod_base) {
>>>>> +        pp->cfg0_mod_base &= CPU_TO_BUS_ADDR;
>>>>> +        pp->cfg1_mod_base &= CPU_TO_BUS_ADDR;
>>>>> +    }
>>>>> +
>>>>>    dw_pcie_setup_rc(pp);
>>>>>    dra7xx_pcie_establish_link(pp);
>>>>>    if (IS_ENABLED(CONFIG_PCI_MSI))
>>>>> diff --git a/drivers/pci/host/pcie-designware.c
>>>> b/drivers/pci/host/pcie-designware.c
>>>>> index 69486be..06c682b 100644
>>>>> --- a/drivers/pci/host/pcie-designware.c
>>>>> +++ b/drivers/pci/host/pcie-designware.c
>>>>> @@ -416,8 +416,7 @@ int dw_pcie_host_init(struct pcie_port *pp)
>>>>>            pp->io_base = range.cpu_addr;
>>>>>
>>>>>            /* Find the untranslated IO space address */
>>>>> -            pp->io_mod_base = of_read_number(parser.range -
>>>>> -                             parser.np + na, ns);
>>>>> +            pp->io_mod_base = range.cpu_addr;;
>>>>>        }
>>>>>        if (restype == IORESOURCE_MEM) {
>>>>>            of_pci_range_to_resource(&range, np, &pp->mem);
>>>>> @@ -426,8 +425,7 @@ int dw_pcie_host_init(struct pcie_port *pp)
>>>>>            pp->mem_bus_addr = range.pci_addr;
>>>>>
>>>>>            /* Find the untranslated MEM space address */
>>>>> -            pp->mem_mod_base = of_read_number(parser.range -
>>>>> -                              parser.np + na, ns);
>>>>> +            pp->mem_mod_base = range.cpu_addr;
>>>>>        }
>>>>>        if (restype == 0) {
>>>>>            of_pci_range_to_resource(&range, np, &pp->cfg);
>>>>> @@ -437,8 +435,7 @@ int dw_pcie_host_init(struct pcie_port *pp)
>>>>>            pp->cfg1_base = pp->cfg.start + pp->cfg0_size;
>>>>>
>>>>>            /* Find the untranslated configuration space address */
>>>>> -            pp->cfg0_mod_base = of_read_number(parser.range -
>>>>> -                               parser.np + na, ns);
>>>>> +            pp->cfg0_mod_base = range.cpu_addr;
>>>>>            pp->cfg1_mod_base = pp->cfg0_mod_base +
>>>>>                        pp->cfg0_size;
>>>>>        }
>>>>> --
>>>>> 1.9.1
>>>>>
>>>>>
>>>>>> -----Original Message-----
>>>>>> From: linux-pci-owner at vger.kernel.org [mailto:linux-pci-
>>>>>> owner at vger.kernel.org] On Behalf Of Rob Herring
>>>>>> Sent: Friday, July 31, 2015 5:53 PM
>>>>>> To: Kishon Vijay Abraham I
>>>>>> Cc: Gabriele Paoloni; Bjorn Helgaas; arnd at arndb.de;
>>>>>> lorenzo.pieralisi at arm.com; Wangzhou (B); robh+dt at kernel.org;
>>>>>> james.morse at arm.com; Liviu.Dudau at arm.com; linux-
>>> pci at vger.kernel.org;
>>>>>> linux-arm-kernel at lists.infradead.org; devicetree at vger.kernel.org;
>>>>>> Yuanzhichang; Zhudacai; zhangjukuo; qiuzhenfa; Liguozhu (Kenneth);
>>>>>> Jingoo Han; Pratyush Anand; Arnd Bergmann
>>>>>> Subject: Re: [PATCH v6] PCI: Store PCIe bus address in struct
>>>>>> of_pci_range
>>>>>>
>>>>>> On Fri, Jul 31, 2015 at 9:57 AM, Kishon Vijay Abraham I
>>>> <kishon@ti.com>
>>>>>> wrote:
>>>>>>> +Arnd
>>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>>> On Friday 31 July 2015 07:55 PM, Gabriele Paoloni wrote:
>>>>>>>> [+cc Kishon]
>>>>>>>>
>>>>>>>>> -----Original Message-----
>>>>>>>>> From: linux-pci-owner at vger.kernel.org [mailto:linux-pci-
>>>>>>>>> owner at vger.kernel.org] On Behalf Of Rob Herring
>>>>>>>>> Sent: Thursday, July 30, 2015 9:42 PM
>>>>>>>>> To: Gabriele Paoloni
>>>>>>>>> Cc: Bjorn Helgaas; arnd at arndb.de; lorenzo.pieralisi at arm.com;
>>>>>> Wangzhou
>>>>>>>>> (B); robh+dt at kernel.org; james.morse at arm.com;
>>> Liviu.Dudau at arm.com;
>>>>>>>>> linux-pci at vger.kernel.org; linux-arm-kernel at lists.infradead.org;
>>>>>>>>> devicetree at vger.kernel.org; Yuanzhichang; Zhudacai; zhangjukuo;
>>>>>>>>> qiuzhenfa; Liguozhu (Kenneth); Jingoo Han; Pratyush Anand
>>>>>>>>> Subject: Re: [PATCH v6] PCI: Store PCIe bus address in struct
>>>>>>>>> of_pci_range
>>>>>>>>>
>>>>>>>>> On Thu, Jul 30, 2015 at 12:34 PM, Gabriele Paoloni
>>>>>>>>> <gabriele.paoloni@huawei.com> wrote:
>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>> From: Bjorn Helgaas [mailto:bhelgaas at google.com]
>>>>>>>>>>> Sent: 30 July 2015 18:15
>>>>>>>>>>> On Thu, Jul 30, 2015 at 04:50:55PM +0000, Gabriele Paoloni
>>>> wrote:
>>>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>>>> From: linux-pci-owner at vger.kernel.org [mailto:linux-pci-
>>>>>>>>>>>>> owner at vger.kernel.org] On Behalf Of Bjorn Helgaas
>>>>>>>>>>>>> Sent: Thursday, July 30, 2015 5:15 PM
>>>>>>>>>>>>> On Thu, Jul 30, 2015 at 01:52:13PM +0000, Gabriele Paoloni
>>>>>> wrote:
>>>>>>>>>
>>>>>>>>> [...]
>>>>>>>>>
>>>>>>>>>>>>>> I don?t think we should rely on [CPU] addresses...what if
>>>> the
>>>>>>>>>>>>> intermediate
>>>>>>>>>>>>>> translation layer changes the lower significant bits of
>>> the
>>>>>>>>> "bus
>>>>>>>>>>>>> address"
>>>>>>>>>>>>>> to translate into a cpu address?
>>>>>>>>>>>>>
>>>>>>>>>>>>> Is it really a possiblity that the lower bits could be
>>>> changed?
>>>>>>>>>>>>
>>>>>>>>>>>> I've checked all the current deignware users DTs except
>>> "pci-
>>>>>>>>>>> layerscape"
>>>>>>>>>>>> that I could not find:
>>>>>>>>>>>> spear1310.dtsi
>>>>>>>>>>>> spear1340.dtsi
>>>>>>>>>>>> dra7.dtsi
>>>>>>>>>>>> imx6qdl.dtsi
>>>>>>>>>>>> imx6sx.dtsi
>>>>>>>>>>>> keystone.dtsi
>>>>>>>>>>>> exynos5440.dtsi
>>>>>>>>>>>>
>>>>>>>>>>>> None of them modifies the lower bits. To be more precise the
>>>>>> only
>>>>>>>>> guy
>>>>>>>>>>>> that provides another translation layer is "dra7.dtsi":
>>>>>>>>>>>> axi0
>>>>>>>>>>>> http://lxr.free-
>>>>>>>>> electrons.com/source/arch/arm/boot/dts/dra7.dtsi#L207
>>>>>>>>>>>>
>>>>>>>>>>>> axi1
>>>>>>>>>>>> http://lxr.free-
>>>>>>>>> electrons.com/source/arch/arm/boot/dts/dra7.dtsi#L241
>>>>>>>>>>>>
>>>>>>>>>>>> For this case masking the top 4bits (bits28 to 31) should
>>> make
>>>>>> the
>>>>>>>>> job.
>>>>>>>>>
>>>>>>>>> IMO, we should just fix this case. After further study, I don't
>>>>>> think
>>>>>>>>> this is a DW issue, but rather an SOC integration issue.
>>>>>>>>>
>>>>>>>>> I believe you can just fixup the address in the pp->ops-
>>>>> host_init
>>>>>> hook.
>>>>>>>>
>>>>>>>> Yes I guess that I could just assign pp->(*)_mod_base to the CPU
>>>>>> address
>>>>>>>> in DW and mask it out in dra7xx_pcie_host_init()...
>>>>>>>>
>>>>>>>> Kishon, would you be ok with that?
>>>>>>>
>>>>>>> Initially I was using *base-mask* property from dt. Me and Arnd
>>>>>> (cc'ed) had
>>>>>>> this discussion [1] before we decided the current approach. It'll
>>>> be
>>>>>> good to
>>>>>>> check with Arnd too.
>>>>>>>
>>>>>>> [1] ->  http://lists.infradead.org/pipermail/linux-arm-
>>> kernel/2014-
>>>>>> May/253528.html
>>>>>>
>>>>>> The problem I have here is the use of ranges does not necessarily
>>>> mean
>>>>>> fewer address bits are available. It can be used just for
>>>> convenience
>>>>>> of not putting the full address into every node's reg property.
>>> And
>>>>>> vice versa, there are probably plenty of cases where we have the
>>>> full
>>>>>> address in the nodes, but really only some of the address bits are
>>>>>> decoded at the IP block. Whether the address bits are present is
>>>>>> rarely cared about or known by s/w folks until you hit a problem
>>>> like
>>>>>> this. Given this is an isolated case ATM, I would fix it in an
>>>>>> isolated way. It does not affect the binding and could be changed
>>> in
>>>>>> the kernel later if this becomes a common need.
>>>>>>
>>>>>> Rob
>>>>>> --
>>>>>> To unsubscribe from this list: send the line "unsubscribe linux-
>>> pci"
>>>> in
>>>>>> the body of a message to majordomo at vger.kernel.org
>>>>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 
> 
> .
> 

^ permalink raw reply	[flat|nested] 73+ messages in thread
end of thread, other threads:[~2015-08-07  5:47 UTC | newest]

Thread overview: 73+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2015-07-27 15:17 [PATCH v6] PCI: Store PCIe bus address in struct of_pci_range Gabriele Paoloni
2015-07-27 15:17 ` Gabriele Paoloni
2015-07-27 15:17 ` Gabriele Paoloni
2015-07-29 16:04 ` Gabriele Paoloni
2015-07-29 16:04   ` Gabriele Paoloni
2015-07-29 16:04   ` Gabriele Paoloni
     [not found] ` <1438010223-124422-1-git-send-email-gabriele.paoloni-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
2015-07-29 17:20   ` Bjorn Helgaas
2015-07-29 17:20     ` Bjorn Helgaas
2015-07-29 17:20     ` Bjorn Helgaas
     [not found]     ` <20150729172053.GE31170-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org>
2015-07-29 19:44       ` Gabriele Paoloni
2015-07-29 19:44         ` Gabriele Paoloni
2015-07-29 19:44         ` Gabriele Paoloni
2015-07-29 21:47         ` Bjorn Helgaas
2015-07-29 21:47           ` Bjorn Helgaas
2015-07-30  8:30           ` Gabriele Paoloni
2015-07-30  8:30             ` Gabriele Paoloni
2015-07-30 11:20             ` Liviu Dudau
2015-07-30 11:20               ` Liviu Dudau
2015-07-30  7:16         ` Zhou Wang
2015-07-30  7:16           ` Zhou Wang
2015-07-30 13:42         ` Rob Herring
2015-07-30 13:42           ` Rob Herring
     [not found]           ` <CAL_JsqKs11-sXQTdBD+ZjbVYFWUHBARTW9q5MF1hCXUhZ+nWLQ-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2015-07-30 13:52             ` Gabriele Paoloni
2015-07-30 13:52               ` Gabriele Paoloni
2015-07-30 13:52               ` Gabriele Paoloni
2015-07-30 14:15               ` Gabriele Paoloni
2015-07-30 14:15                 ` Gabriele Paoloni
2015-07-30 14:15                 ` Gabriele Paoloni
2015-07-30 16:14               ` Bjorn Helgaas
2015-07-30 16:14                 ` Bjorn Helgaas
2015-07-30 16:50                 ` Gabriele Paoloni
2015-07-30 16:50                   ` Gabriele Paoloni
2015-07-30 16:50                   ` Gabriele Paoloni
2015-07-30 17:14                   ` Bjorn Helgaas
2015-07-30 17:14                     ` Bjorn Helgaas
2015-07-30 17:34                     ` Gabriele Paoloni
2015-07-30 17:34                       ` Gabriele Paoloni
2015-07-30 17:34                       ` Gabriele Paoloni
2015-07-30 20:41                       ` Rob Herring
2015-07-30 20:41                         ` Rob Herring
2015-07-31 14:25                         ` Gabriele Paoloni
2015-07-31 14:25                           ` Gabriele Paoloni
2015-07-31 14:25                           ` Gabriele Paoloni
2015-07-31 14:57                           ` Kishon Vijay Abraham I
2015-07-31 14:57                             ` Kishon Vijay Abraham I
2015-07-31 14:57                             ` Kishon Vijay Abraham I
2015-07-31 15:09                             ` Gabriele Paoloni
2015-07-31 15:09                               ` Gabriele Paoloni
2015-07-31 15:09                               ` Gabriele Paoloni
2015-08-03 14:41                               ` Jingoo Han
2015-08-03 14:41                                 ` Jingoo Han
2015-08-03 14:41                                 ` Jingoo Han
2015-07-31 16:53                             ` Rob Herring
2015-07-31 16:53                               ` Rob Herring
2015-08-03 11:18                               ` Gabriele Paoloni
2015-08-03 11:18                                 ` Gabriele Paoloni
2015-08-03 11:18                                 ` Gabriele Paoloni
2015-08-04  4:19                                 ` Jingoo Han
2015-08-04  4:19                                   ` Jingoo Han
2015-08-04 10:12                                   ` Gabriele Paoloni
2015-08-04 10:12                                     ` Gabriele Paoloni
2015-08-04 10:12                                     ` Gabriele Paoloni
2015-08-06 13:52                                     ` Gabriele Paoloni
2015-08-06 13:52                                       ` Gabriele Paoloni
2015-08-06 13:52                                       ` Gabriele Paoloni
2015-08-06 15:06                                       ` Jingoo Han
2015-08-06 15:06                                         ` Jingoo Han
2015-08-06 15:06                                         ` Jingoo Han
2015-08-07  5:46                                         ` Zhou Wang
2015-08-07  5:46                                           ` Zhou Wang
2015-08-07  5:46                                           ` Zhou Wang
2015-07-30 16:06           ` Bjorn Helgaas
2015-07-30 16:06             ` Bjorn Helgaas

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.