[PATCH 0/3] MIPS: ralink: properly handle pci IO resources

[PATCH 0/3] MIPS: ralink: properly handle pci IO resources

[Sergio Paracuellos]

Hi all,

Defining PCI_IOBASE for MIPS ralink platform results in resource handling working
but the addresses generated for IO access being wrong, because the iomap tries to
ioremap it to a fixed virtual address (PCI_IOBASE), which can't work for KSEG1 
addresses. To get it working this way, we would need to put PCI_IOBASE somewhere
into KSEG2, which will create TLB entries for IO addresses, which most of the
time isn't needed on MIPS because of access via KSEG1. Instead of doing that and
taking into account that we need to get a valid IO address from 'pci_address_to_pio'
and ralink platforms have IO addresses higher than 0xffff, the following approach
will be preferred to get expected working behaviour from PCI core APIs and pci 
drivers working together:
 
1) Avoid to define PCI_IOBASE.
2) Set IO_SPACE_LIMIT to 0x1fffffff which is a valid range for this SoCs.
3) Avoid to ioremap IO resource if PCI_IOBASE is not defined. 
3) Set ioport_resource end limit to this new IO_SPACE_LIMIT.

Doing in this way we end up with a properly working PCI IO in ralink SoCs.
These changes metioned above are in the three patches included in this series.

Thanks in advance for your time and comments.

Best regards,
    Sergio Paracuellos

Sergio Paracuellos (3):
  MIPS: ralink: don't define PC_IOBASE but increase IO_SPACE_LIMIT
  PCI: of: avoid 'devm_pci_remap_iospace' if PCI_IOBASE is not defined
  staging: mt7621-pci: set end limit for 'ioport_resource'

 arch/mips/include/asm/mach-ralink/spaces.h | 4 +---
 drivers/pci/of.c                           | 2 ++
 drivers/staging/mt7621-pci/pci-mt7621.c    | 2 ++
 3 files changed, 5 insertions(+), 3 deletions(-)

-- 
2.25.1

[PATCH 1/3] MIPS: ralink: don't define PC_IOBASE but increase IO_SPACE_LIMIT

[Sergio Paracuellos]

Defining PCI_IOBASE results in pci resource handling working but the
addresses generated for IO accesses are wrong since the ioremap in the pci
core function 'pci_parse_request_of_pci_ranges' tries to remap to a fixed
virtual address (PC_IOBASE) which can't work for KSEG1 addresses. To get
it working this way, we would need to put PCI_IOBASE somewhere into KSEG2
which will result in creating TLB entries for IO addresses, which most of
the time isn't needed on MIPS because of access via KSEG1. So avoid to
define PCI_IOBASE and increase IO_SPACE_LIMIT resource for ralink MIPS
platform instead, to get valid IO addresses for resources from pci core
'pci_address_to_pio' function.

Fixes: 222b27713d7f ("MIPS: ralink: Define PCI_IOBASE)
Signed-off-by: Sergio Paracuellos <sergio.paracuellos@gmail.com>
---
 arch/mips/include/asm/mach-ralink/spaces.h | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)

diff --git a/arch/mips/include/asm/mach-ralink/spaces.h b/arch/mips/include/asm/mach-ralink/spaces.h
index 87d085c9ad61..31a3525213cf 100644
--- a/arch/mips/include/asm/mach-ralink/spaces.h
+++ b/arch/mips/include/asm/mach-ralink/spaces.h
@@ -2,9 +2,7 @@
 #ifndef __ASM_MACH_RALINK_SPACES_H_
 #define __ASM_MACH_RALINK_SPACES_H_
 
-#define PCI_IOBASE	_AC(0xa0000000, UL)
-#define PCI_IOSIZE	SZ_16M
-#define IO_SPACE_LIMIT	(PCI_IOSIZE - 1)
+#define IO_SPACE_LIMIT	0x1fffffff
 
 #include <asm/mach-generic/spaces.h>
 #endif
-- 
2.25.1

[PATCH 2/3] PCI: of: avoid 'devm_pci_remap_iospace' if PCI_IOBASE is not defined

[Sergio Paracuellos]

Defining PCI_IOBASE for MIPS ralink in expected addresses results in PCI IO
resources being assigned but the addresses generated for IO accesses are wrong
since the ioremap in the PCI core function 'pci_parse_request_of_pci_ranges'
tries to remap to a fixed virtual address (PC_IOBASE) which can't work for KSEG1
addresses. To get it working this way, we would need to put PCI_IOBASE somewhere
into KSEG2 which will result in creating TLB entries for IO addresses, which most
of the time isn't needed on MIPS because of access via KSEG1. To allow MIPS PCI
drivers to properly use the PCI generic core we need to increase IO_SPACE_LIMIT
since IO addresses are in addresses higher that 0xffff. We also need to avoid
the call 'devm_pci_remap_iospace' when 'pci_parse_request_of_pci_ranges' is
called to avoid the following problem:

------------[ cut here ]------------
WARNING: CPU: 2 PID: 1 at ../drivers/pci/pci.c:4066 pci_remap_iospace+0x3c/0x54
This architecture does not support memory mapped I/O
Modules linked in:
CPU: 2 PID: 1 Comm: swapper/0 Not tainted 5.10.1+ #1228
Stack : 00000000 00000000 807fa974 00000000 827ffa80 80066b48 80710000 0000000b
        00000000 00000000 81c59aac 7d06ddec 80850000 00000001 81c59a40 7d06ddec
        00000000 00000000 807c909c 81c598f0 00000001 81c59904 00000000 0000000a
        203a6d6d 80708880 0000000f 70617773 80850000 00000000 00000000 807d0000
        807ffecc 1e160000 00000001 00000200 00000000 8054e920 00000008 815e0008
        ...
Call Trace:
[<80008efc>] show_stack+0x8c/0x130
[<806e1674>] dump_stack+0x9c/0xc8
[<80024a3c>] __warn+0xc0/0xe8
[<80024ad0>] warn_slowpath_fmt+0x6c/0xbc
[<80410ca8>] pci_remap_iospace+0x3c/0x54
[<80410d20>] devm_pci_remap_iospace+0x58/0xa4
[<8042019c>] devm_of_pci_bridge_init+0x4dc/0x55c
[<80408de8>] devm_pci_alloc_host_bridge+0x78/0x88
[<80424e44>] mt7621_pci_probe+0x68/0x9a4
[<80464804>] platform_drv_probe+0x40/0x7c
[<804628bc>] really_probe+0x2fc/0x4e4
[<80463214>] device_driver_attach+0x4c/0x74
[<80463384>] __driver_attach+0x148/0x150
[<8046047c>] bus_for_each_dev+0x6c/0xb0
[<804614dc>] bus_add_driver+0x1b4/0x1fc
[<80463aa0>] driver_register+0xd0/0x110
[<80001714>] do_one_initcall+0x84/0x1c0
[<808e7fd0>] kernel_init_freeable+0x214/0x24c
[<806e4164>] kernel_init+0x14/0x118
[<80003358>] ret_from_kernel_thread+0x14/0x1c

---[ end trace 1c9d4412bd51b53c ]---
mt7621-pci 1e140000.pcie: error -19: failed to map resource [io  0x1e160000-0x1e16ffff]

Hence don't call 'devm_pci_remap_iospace' if PCI_IOBASE is not defined to get
a working PCI core APIs for MIPS ralink platforms.

Signed-off-by: Sergio Paracuellos <sergio.paracuellos@gmail.com>
---
 drivers/pci/of.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/drivers/pci/of.c b/drivers/pci/of.c
index a143b02b2dcd..657aef39bf63 100644
--- a/drivers/pci/of.c
+++ b/drivers/pci/of.c
@@ -564,12 +564,14 @@ static int pci_parse_request_of_pci_ranges(struct device *dev,
 
 		switch (resource_type(res)) {
 		case IORESOURCE_IO:
+#ifdef PCI_IOBASE
 			err = devm_pci_remap_iospace(dev, res, iobase);
 			if (err) {
 				dev_warn(dev, "error %d: failed to map resource %pR\n",
 					 err, res);
 				resource_list_destroy_entry(win);
 			}
+#endif
 			break;
 		case IORESOURCE_MEM:
 			res_valid |= !(res->flags & IORESOURCE_PREFETCH);
-- 
2.25.1

[PATCH 3/3] staging: mt7621-pci: set end limit for 'ioport_resource'

[Sergio Paracuellos]

We have increase IO_SPACE_LIMIT for ralink platform to get PCI IO resources
properly handled using PCI core APIs. To align those changes with driver
code we have to set 'ioport_resource' end limit to IO_SPACE_LIMIT to avoid
errors.

Signed-off-by: Sergio Paracuellos <sergio.paracuellos@gmail.com>
---
 drivers/staging/mt7621-pci/pci-mt7621.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/drivers/staging/mt7621-pci/pci-mt7621.c b/drivers/staging/mt7621-pci/pci-mt7621.c
index 691030e1a5ed..6301397c3987 100644
--- a/drivers/staging/mt7621-pci/pci-mt7621.c
+++ b/drivers/staging/mt7621-pci/pci-mt7621.c
@@ -522,6 +522,8 @@ static int mt7621_pci_probe(struct platform_device *pdev)
 	if (!dev->of_node)
 		return -ENODEV;
 
+	ioport_resource.end = IO_SPACE_LIMIT;
+
 	bridge = devm_pci_alloc_host_bridge(dev, sizeof(*pcie));
 	if (!bridge)
 		return -ENOMEM;
-- 
2.25.1

Re: [PATCH 2/3] PCI: of: avoid 'devm_pci_remap_iospace' if PCI_IOBASE is not defined

[Jiaxun Yang]

在2021年8月7日八月 下午3:24，Sergio Paracuellos写道：
> Defining PCI_IOBASE for MIPS ralink in expected addresses results in 
> PCI IO
> resources being assigned but the addresses generated for IO accesses 
> are wrong
> since the ioremap in the PCI core function 
> 'pci_parse_request_of_pci_ranges'
> tries to remap to a fixed virtual address (PC_IOBASE) which can't work 
> for KSEG1
> addresses. To get it working this way, we would need to put PCI_IOBASE 
> somewhere
> into KSEG2 which will result in creating TLB entries for IO addresses, 
> which most
> of the time isn't needed on MIPS because of access via KSEG1. To allow 

It was designed to allow multiple PCI bridge with sparse IO space pattern.
So for ralink it's not going to happen?

Thanks.
- Jiaxun


> MIPS PCI
> drivers to properly use the PCI generic core we need to increase 
> IO_SPACE_LIMIT
> since IO addresses are in addresses higher that 0xffff. We also need to 
> avoid
> the call 'devm_pci_remap_iospace' when 
> 'pci_parse_request_of_pci_ranges' is
> called to avoid the following problem:
> 
> ------------[ cut here ]------------
> WARNING: CPU: 2 PID: 1 at ../drivers/pci/pci.c:4066 pci_remap_iospace+0x3c/0x54
> This architecture does not support memory mapped I/O
> Modules linked in:
> CPU: 2 PID: 1 Comm: swapper/0 Not tainted 5.10.1+ #1228
> Stack : 00000000 00000000 807fa974 00000000 827ffa80 80066b48 80710000 0000000b
>         00000000 00000000 81c59aac 7d06ddec 80850000 00000001 81c59a40 7d06ddec
>         00000000 00000000 807c909c 81c598f0 00000001 81c59904 00000000 0000000a
>         203a6d6d 80708880 0000000f 70617773 80850000 00000000 00000000 807d0000
>         807ffecc 1e160000 00000001 00000200 00000000 8054e920 00000008 815e0008
>         ...
> Call Trace:
> [<80008efc>] show_stack+0x8c/0x130
> [<806e1674>] dump_stack+0x9c/0xc8
> [<80024a3c>] __warn+0xc0/0xe8
> [<80024ad0>] warn_slowpath_fmt+0x6c/0xbc
> [<80410ca8>] pci_remap_iospace+0x3c/0x54
> [<80410d20>] devm_pci_remap_iospace+0x58/0xa4
> [<8042019c>] devm_of_pci_bridge_init+0x4dc/0x55c
> [<80408de8>] devm_pci_alloc_host_bridge+0x78/0x88
> [<80424e44>] mt7621_pci_probe+0x68/0x9a4
> [<80464804>] platform_drv_probe+0x40/0x7c
> [<804628bc>] really_probe+0x2fc/0x4e4
> [<80463214>] device_driver_attach+0x4c/0x74
> [<80463384>] __driver_attach+0x148/0x150
> [<8046047c>] bus_for_each_dev+0x6c/0xb0
> [<804614dc>] bus_add_driver+0x1b4/0x1fc
> [<80463aa0>] driver_register+0xd0/0x110
> [<80001714>] do_one_initcall+0x84/0x1c0
> [<808e7fd0>] kernel_init_freeable+0x214/0x24c
> [<806e4164>] kernel_init+0x14/0x118
> [<80003358>] ret_from_kernel_thread+0x14/0x1c
> 
> ---[ end trace 1c9d4412bd51b53c ]---
> mt7621-pci 1e140000.pcie: error -19: failed to map resource [io  
> 0x1e160000-0x1e16ffff]
> 
> Hence don't call 'devm_pci_remap_iospace' if PCI_IOBASE is not defined to get
> a working PCI core APIs for MIPS ralink platforms.
> 
> Signed-off-by: Sergio Paracuellos <sergio.paracuellos@gmail.com>
> ---
>  drivers/pci/of.c | 2 ++
>  1 file changed, 2 insertions(+)
> 
> diff --git a/drivers/pci/of.c b/drivers/pci/of.c
> index a143b02b2dcd..657aef39bf63 100644
> --- a/drivers/pci/of.c
> +++ b/drivers/pci/of.c
> @@ -564,12 +564,14 @@ static int pci_parse_request_of_pci_ranges(struct 
> device *dev,
>  
>  		switch (resource_type(res)) {
>  		case IORESOURCE_IO:
> +#ifdef PCI_IOBASE
>  			err = devm_pci_remap_iospace(dev, res, iobase);
>  			if (err) {
>  				dev_warn(dev, "error %d: failed to map resource %pR\n",
>  					 err, res);
>  				resource_list_destroy_entry(win);
>  			}
> +#endif
>  			break;
>  		case IORESOURCE_MEM:
>  			res_valid |= !(res->flags & IORESOURCE_PREFETCH);
> -- 
> 2.25.1
> 
> 


-- 
- Jiaxun

Re: [PATCH 2/3] PCI: of: avoid 'devm_pci_remap_iospace' if PCI_IOBASE is not defined

[Sergio Paracuellos]

Hi Jiaxun,

On Sun, Aug 8, 2021 at 3:41 AM Jiaxun Yang <jiaxun.yang@flygoat.com> wrote:
>
>
>
> 在2021年8月7日八月 下午3:24，Sergio Paracuellos写道：
> > Defining PCI_IOBASE for MIPS ralink in expected addresses results in
> > PCI IO
> > resources being assigned but the addresses generated for IO accesses
> > are wrong
> > since the ioremap in the PCI core function
> > 'pci_parse_request_of_pci_ranges'
> > tries to remap to a fixed virtual address (PC_IOBASE) which can't work
> > for KSEG1
> > addresses. To get it working this way, we would need to put PCI_IOBASE
> > somewhere
> > into KSEG2 which will result in creating TLB entries for IO addresses,
> > which most
> > of the time isn't needed on MIPS because of access via KSEG1. To allow
>
> It was designed to allow multiple PCI bridge with sparse IO space pattern.
> So for ralink it's not going to happen?

Sorry, I don't understand your question. Can you please clarify it for me?

Thanks,
    Sergio Paracuellos
>
> Thanks.
> - Jiaxun
>
>
> > MIPS PCI
> > drivers to properly use the PCI generic core we need to increase
> > IO_SPACE_LIMIT
> > since IO addresses are in addresses higher that 0xffff. We also need to
> > avoid
> > the call 'devm_pci_remap_iospace' when
> > 'pci_parse_request_of_pci_ranges' is
> > called to avoid the following problem:
> >
> > ------------[ cut here ]------------
> > WARNING: CPU: 2 PID: 1 at ../drivers/pci/pci.c:4066 pci_remap_iospace+0x3c/0x54
> > This architecture does not support memory mapped I/O
> > Modules linked in:
> > CPU: 2 PID: 1 Comm: swapper/0 Not tainted 5.10.1+ #1228
> > Stack : 00000000 00000000 807fa974 00000000 827ffa80 80066b48 80710000 0000000b
> >         00000000 00000000 81c59aac 7d06ddec 80850000 00000001 81c59a40 7d06ddec
> >         00000000 00000000 807c909c 81c598f0 00000001 81c59904 00000000 0000000a
> >         203a6d6d 80708880 0000000f 70617773 80850000 00000000 00000000 807d0000
> >         807ffecc 1e160000 00000001 00000200 00000000 8054e920 00000008 815e0008
> >         ...
> > Call Trace:
> > [<80008efc>] show_stack+0x8c/0x130
> > [<806e1674>] dump_stack+0x9c/0xc8
> > [<80024a3c>] __warn+0xc0/0xe8
> > [<80024ad0>] warn_slowpath_fmt+0x6c/0xbc
> > [<80410ca8>] pci_remap_iospace+0x3c/0x54
> > [<80410d20>] devm_pci_remap_iospace+0x58/0xa4
> > [<8042019c>] devm_of_pci_bridge_init+0x4dc/0x55c
> > [<80408de8>] devm_pci_alloc_host_bridge+0x78/0x88
> > [<80424e44>] mt7621_pci_probe+0x68/0x9a4
> > [<80464804>] platform_drv_probe+0x40/0x7c
> > [<804628bc>] really_probe+0x2fc/0x4e4
> > [<80463214>] device_driver_attach+0x4c/0x74
> > [<80463384>] __driver_attach+0x148/0x150
> > [<8046047c>] bus_for_each_dev+0x6c/0xb0
> > [<804614dc>] bus_add_driver+0x1b4/0x1fc
> > [<80463aa0>] driver_register+0xd0/0x110
> > [<80001714>] do_one_initcall+0x84/0x1c0
> > [<808e7fd0>] kernel_init_freeable+0x214/0x24c
> > [<806e4164>] kernel_init+0x14/0x118
> > [<80003358>] ret_from_kernel_thread+0x14/0x1c
> >
> > ---[ end trace 1c9d4412bd51b53c ]---
> > mt7621-pci 1e140000.pcie: error -19: failed to map resource [io
> > 0x1e160000-0x1e16ffff]
> >
> > Hence don't call 'devm_pci_remap_iospace' if PCI_IOBASE is not defined to get
> > a working PCI core APIs for MIPS ralink platforms.
> >
> > Signed-off-by: Sergio Paracuellos <sergio.paracuellos@gmail.com>
> > ---
> >  drivers/pci/of.c | 2 ++
> >  1 file changed, 2 insertions(+)
> >
> > diff --git a/drivers/pci/of.c b/drivers/pci/of.c
> > index a143b02b2dcd..657aef39bf63 100644
> > --- a/drivers/pci/of.c
> > +++ b/drivers/pci/of.c
> > @@ -564,12 +564,14 @@ static int pci_parse_request_of_pci_ranges(struct
> > device *dev,
> >
> >               switch (resource_type(res)) {
> >               case IORESOURCE_IO:
> > +#ifdef PCI_IOBASE
> >                       err = devm_pci_remap_iospace(dev, res, iobase);
> >                       if (err) {
> >                               dev_warn(dev, "error %d: failed to map resource %pR\n",
> >                                        err, res);
> >                               resource_list_destroy_entry(win);
> >                       }
> > +#endif
> >                       break;
> >               case IORESOURCE_MEM:
> >                       res_valid |= !(res->flags & IORESOURCE_PREFETCH);
> > --
> > 2.25.1
> >
> >
>
>
> --
> - Jiaxun

Re: [PATCH 2/3] PCI: of: avoid 'devm_pci_remap_iospace' if PCI_IOBASE is not defined

[Bjorn Helgaas]

On Sat, Aug 07, 2021 at 09:24:08AM +0200, Sergio Paracuellos wrote:
> Defining PCI_IOBASE for MIPS ralink in expected addresses results in PCI IO
> resources being assigned but the addresses generated for IO accesses are wrong
> since the ioremap in the PCI core function 'pci_parse_request_of_pci_ranges'
> tries to remap to a fixed virtual address (PC_IOBASE) which can't work for KSEG1
> addresses. To get it working this way, we would need to put PCI_IOBASE somewhere
> into KSEG2 which will result in creating TLB entries for IO addresses, which most
> of the time isn't needed on MIPS because of access via KSEG1. To allow MIPS PCI
> drivers to properly use the PCI generic core we need to increase IO_SPACE_LIMIT
> since IO addresses are in addresses higher that 0xffff. We also need to avoid
> the call 'devm_pci_remap_iospace' when 'pci_parse_request_of_pci_ranges' is
> called to avoid the following problem:

Rewrap to fit in ~75 columns.

This is a generic change so the commit log needs to be generic as
well.  The MIPS/KSEG1/KSEG2 information is not really useful here
because most readers won't understand it (and I don't :)).

devm_pci_remap_iospace() calls pci_remap_iospace(), which already
contains #ifdef PCI_IOBASE.  When PCI_IOBASE is not defined (as on 
MIPS ralink), it emits the warning below and returns failure.

This patch avoids that failure, but it still leaves
devm_pci_remap_iospace() and pci_remap_iospace() broken on MIPS
ralink.  It's true that on MIPS ralink, they are currently only called
via pci_parse_request_of_pci_ranges(), but I think it would be better
if we could fix pci_remap_iospace() to handle this case so all these
interfaces work consistently.

This patch doesn't do anything with IO_SPACE_LIMIT, so I don't know
what that part of the commit log is telling me.

> ------------[ cut here ]------------
> WARNING: CPU: 2 PID: 1 at ../drivers/pci/pci.c:4066 pci_remap_iospace+0x3c/0x54
> This architecture does not support memory mapped I/O
> Modules linked in:
> CPU: 2 PID: 1 Comm: swapper/0 Not tainted 5.10.1+ #1228
> Stack : 00000000 00000000 807fa974 00000000 827ffa80 80066b48 80710000 0000000b
>         00000000 00000000 81c59aac 7d06ddec 80850000 00000001 81c59a40 7d06ddec
>         00000000 00000000 807c909c 81c598f0 00000001 81c59904 00000000 0000000a
>         203a6d6d 80708880 0000000f 70617773 80850000 00000000 00000000 807d0000
>         807ffecc 1e160000 00000001 00000200 00000000 8054e920 00000008 815e0008
>         ...
> Call Trace:
> [<80008efc>] show_stack+0x8c/0x130
> [<806e1674>] dump_stack+0x9c/0xc8
> [<80024a3c>] __warn+0xc0/0xe8
> [<80024ad0>] warn_slowpath_fmt+0x6c/0xbc
> [<80410ca8>] pci_remap_iospace+0x3c/0x54
> [<80410d20>] devm_pci_remap_iospace+0x58/0xa4
> [<8042019c>] devm_of_pci_bridge_init+0x4dc/0x55c
> [<80408de8>] devm_pci_alloc_host_bridge+0x78/0x88
> [<80424e44>] mt7621_pci_probe+0x68/0x9a4
> [<80464804>] platform_drv_probe+0x40/0x7c
> [<804628bc>] really_probe+0x2fc/0x4e4
> [<80463214>] device_driver_attach+0x4c/0x74
> [<80463384>] __driver_attach+0x148/0x150
> [<8046047c>] bus_for_each_dev+0x6c/0xb0
> [<804614dc>] bus_add_driver+0x1b4/0x1fc
> [<80463aa0>] driver_register+0xd0/0x110
> [<80001714>] do_one_initcall+0x84/0x1c0
> [<808e7fd0>] kernel_init_freeable+0x214/0x24c
> [<806e4164>] kernel_init+0x14/0x118
> [<80003358>] ret_from_kernel_thread+0x14/0x1c
> 
> ---[ end trace 1c9d4412bd51b53c ]---
> mt7621-pci 1e140000.pcie: error -19: failed to map resource [io  0x1e160000-0x1e16ffff]
> 
> Hence don't call 'devm_pci_remap_iospace' if PCI_IOBASE is not defined to get
> a working PCI core APIs for MIPS ralink platforms.
> 
> Signed-off-by: Sergio Paracuellos <sergio.paracuellos@gmail.com>
> ---
>  drivers/pci/of.c | 2 ++
>  1 file changed, 2 insertions(+)
> 
> diff --git a/drivers/pci/of.c b/drivers/pci/of.c
> index a143b02b2dcd..657aef39bf63 100644
> --- a/drivers/pci/of.c
> +++ b/drivers/pci/of.c
> @@ -564,12 +564,14 @@ static int pci_parse_request_of_pci_ranges(struct device *dev,
>  
>  		switch (resource_type(res)) {
>  		case IORESOURCE_IO:
> +#ifdef PCI_IOBASE
>  			err = devm_pci_remap_iospace(dev, res, iobase);
>  			if (err) {
>  				dev_warn(dev, "error %d: failed to map resource %pR\n",
>  					 err, res);
>  				resource_list_destroy_entry(win);
>  			}
> +#endif
>  			break;
>  		case IORESOURCE_MEM:
>  			res_valid |= !(res->flags & IORESOURCE_PREFETCH);
> -- 
> 2.25.1
> 

Re: [PATCH 2/3] PCI: of: avoid 'devm_pci_remap_iospace' if PCI_IOBASE is not defined

[Sergio Paracuellos]

Hi Bjorn,

On Wed, Aug 11, 2021 at 12:32 AM Bjorn Helgaas <helgaas@kernel.org> wrote:
>
> On Sat, Aug 07, 2021 at 09:24:08AM +0200, Sergio Paracuellos wrote:
> > Defining PCI_IOBASE for MIPS ralink in expected addresses results in PCI IO
> > resources being assigned but the addresses generated for IO accesses are wrong
> > since the ioremap in the PCI core function 'pci_parse_request_of_pci_ranges'
> > tries to remap to a fixed virtual address (PC_IOBASE) which can't work for KSEG1
> > addresses. To get it working this way, we would need to put PCI_IOBASE somewhere
> > into KSEG2 which will result in creating TLB entries for IO addresses, which most
> > of the time isn't needed on MIPS because of access via KSEG1. To allow MIPS PCI
> > drivers to properly use the PCI generic core we need to increase IO_SPACE_LIMIT
> > since IO addresses are in addresses higher that 0xffff. We also need to avoid
> > the call 'devm_pci_remap_iospace' when 'pci_parse_request_of_pci_ranges' is
> > called to avoid the following problem:
>
> Rewrap to fit in ~75 columns.

I thought it was already around 75 columns :). For sure, I will
carefully check this for the next version.

>
> This is a generic change so the commit log needs to be generic as
> well.  The MIPS/KSEG1/KSEG2 information is not really useful here
> because most readers won't understand it (and I don't :)).

Yes, you are right, my bad here. MIPS specific address segment names
should be out of generic commit messages like this.

>
> devm_pci_remap_iospace() calls pci_remap_iospace(), which already
> contains #ifdef PCI_IOBASE.  When PCI_IOBASE is not defined (as on
> MIPS ralink), it emits the warning below and returns failure.
>
> This patch avoids that failure, but it still leaves
> devm_pci_remap_iospace() and pci_remap_iospace() broken on MIPS
> ralink.  It's true that on MIPS ralink, they are currently only called
> via pci_parse_request_of_pci_ranges(), but I think it would be better
> if we could fix pci_remap_iospace() to handle this case so all these
> interfaces work consistently.

Ok, agred. So... what should be the correct approach here? Change the
core 'pci_remap_iospace' WARN_ON to a maybe a normal 'pr_warn' message
and silently return 0 and delete function comment there saying that
architectures that don't define PCI_IOBASE should not call that
function? Or should we just redefine this weak 'pci_remap_iospace'
symbol for mips ralink?? I prefer to always do changes in normal core
functions but if this is going to be a mips ralink adjustment because
of architecture specific stuff maybe is better to go for the other
option... Thoughts?


>
> This patch doesn't do anything with IO_SPACE_LIMIT, so I don't know
> what that part of the commit log is telling me.

True, I was introducing the whole thing to have all pci io stuff
working but it is out of scope for this commit in particular. Thanks
for pointing out this also.

Thanks in advance for your time.

Best regards,
    Sergio Paracuellos
>
> > ------------[ cut here ]------------
> > WARNING: CPU: 2 PID: 1 at ../drivers/pci/pci.c:4066 pci_remap_iospace+0x3c/0x54
> > This architecture does not support memory mapped I/O
> > Modules linked in:
> > CPU: 2 PID: 1 Comm: swapper/0 Not tainted 5.10.1+ #1228
> > Stack : 00000000 00000000 807fa974 00000000 827ffa80 80066b48 80710000 0000000b
> >         00000000 00000000 81c59aac 7d06ddec 80850000 00000001 81c59a40 7d06ddec
> >         00000000 00000000 807c909c 81c598f0 00000001 81c59904 00000000 0000000a
> >         203a6d6d 80708880 0000000f 70617773 80850000 00000000 00000000 807d0000
> >         807ffecc 1e160000 00000001 00000200 00000000 8054e920 00000008 815e0008
> >         ...
> > Call Trace:
> > [<80008efc>] show_stack+0x8c/0x130
> > [<806e1674>] dump_stack+0x9c/0xc8
> > [<80024a3c>] __warn+0xc0/0xe8
> > [<80024ad0>] warn_slowpath_fmt+0x6c/0xbc
> > [<80410ca8>] pci_remap_iospace+0x3c/0x54
> > [<80410d20>] devm_pci_remap_iospace+0x58/0xa4
> > [<8042019c>] devm_of_pci_bridge_init+0x4dc/0x55c
> > [<80408de8>] devm_pci_alloc_host_bridge+0x78/0x88
> > [<80424e44>] mt7621_pci_probe+0x68/0x9a4
> > [<80464804>] platform_drv_probe+0x40/0x7c
> > [<804628bc>] really_probe+0x2fc/0x4e4
> > [<80463214>] device_driver_attach+0x4c/0x74
> > [<80463384>] __driver_attach+0x148/0x150
> > [<8046047c>] bus_for_each_dev+0x6c/0xb0
> > [<804614dc>] bus_add_driver+0x1b4/0x1fc
> > [<80463aa0>] driver_register+0xd0/0x110
> > [<80001714>] do_one_initcall+0x84/0x1c0
> > [<808e7fd0>] kernel_init_freeable+0x214/0x24c
> > [<806e4164>] kernel_init+0x14/0x118
> > [<80003358>] ret_from_kernel_thread+0x14/0x1c
> >
> > ---[ end trace 1c9d4412bd51b53c ]---
> > mt7621-pci 1e140000.pcie: error -19: failed to map resource [io  0x1e160000-0x1e16ffff]
> >
> > Hence don't call 'devm_pci_remap_iospace' if PCI_IOBASE is not defined to get
> > a working PCI core APIs for MIPS ralink platforms.
> >
> > Signed-off-by: Sergio Paracuellos <sergio.paracuellos@gmail.com>
> > ---
> >  drivers/pci/of.c | 2 ++
> >  1 file changed, 2 insertions(+)
> >
> > diff --git a/drivers/pci/of.c b/drivers/pci/of.c
> > index a143b02b2dcd..657aef39bf63 100644
> > --- a/drivers/pci/of.c
> > +++ b/drivers/pci/of.c
> > @@ -564,12 +564,14 @@ static int pci_parse_request_of_pci_ranges(struct device *dev,
> >
> >               switch (resource_type(res)) {
> >               case IORESOURCE_IO:
> > +#ifdef PCI_IOBASE
> >                       err = devm_pci_remap_iospace(dev, res, iobase);
> >                       if (err) {
> >                               dev_warn(dev, "error %d: failed to map resource %pR\n",
> >                                        err, res);
> >                               resource_list_destroy_entry(win);
> >                       }
> > +#endif
> >                       break;
> >               case IORESOURCE_MEM:
> >                       res_valid |= !(res->flags & IORESOURCE_PREFETCH);
> > --
> > 2.25.1
> >