From: "Luís Mendes" <luis.p.mendes@gmail.com>
To: Bjorn Helgaas <helgaas@kernel.org>
Cc: Nicholas Johnson <nicholas.johnson-opensource@outlook.com.au>,
Linux PCI <linux-pci@vger.kernel.org>,
Thomas Petazzoni <thomas.petazzoni@bootlin.com>,
Jason Cooper <jason@lakedaemon.net>,
Benjamin Herrenschmidt <benh@kernel.crashing.org>
Subject: Re: Problem with PCIe enumeration of Google/Coral TPU Edge module on Linux
Date: Thu, 9 Apr 2020 00:05:25 +0100 [thread overview]
Message-ID: <CAEzXK1oU4KwatuXAjUwj5arZtdmDSgFsm2vn8DrAWCJu-v-r0g@mail.gmail.com> (raw)
In-Reply-To: <CAEzXK1pNiy_pjDh_=2XKpKUwEfO39rFubkviZyT_7DqEaQ7z5w@mail.gmail.com>
Hi Bjorn,
I have successfully setup a JTAG remote debug environment.
And I found this:
First call to __pci_bus_assign_resources visits 11ab:6828 -> SLOT 1,
which in turn calls __pci_bus_assign_resources which visits device
1ac1:089a on that slot and calls:
/*
* Try to assign any resources marked as IORESOURCE_PCI_FIXED, as they are
* skipped by pbus_assign_resources_sorted().
*/
static void pdev_assign_fixed_resources(struct pci_dev *dev)
{
int i;
pci_info(dev, "%s\n", __func__);
for (i = 0; i < PCI_NUM_RESOURCES; i++) {
struct pci_bus *b;
struct resource *r = &dev->resource[i];
if (r->parent || !(r->flags & IORESOURCE_PCI_FIXED) ||
!(r->flags & (IORESOURCE_IO | IORESOURCE_MEM)))
continue;
b = dev->bus;
while (b && !r->parent) {
assign_fixed_resource_on_bus(b, r);
b = b->parent;
}
}
}
where dev has the following data before calling
pdev_assign_fixed_resources, for some reason BAR0 and BAR2 are skipped
or assign_fixed_resource_on_bus is not called :
dev->vendorID 6849(0x1ac1)
->deviceID 2202(0x089a)
->resource={ {start = 0
end=16383
name="0000:01:00.0"
flags=1319436(0x14220C)
desc=0
parent=0x0
sibling=0x0
child=0x0},
{start = 0
end=0
name=0x0
desc=0
parent=0x0
sibling=0x0
child=0x0},
{start = 0,
end = 1048575,
name="0000:01:00.0"
flags=1319436(0x14220C)
desc=0,
parent=0x0,
sibling=0x0
child=0x0},
{start = 0
end=0
name="0000:01:00.0"
flags=0
desc=0
parent=0x0
sibling=0x0
child=0x0},
{start = 0
end=0
name="0000:01:00.0"
flags=0
desc=0
parent=0x0
sibling=0x0
child=0x0},
{start = 0
end=0
name="0000:01:00.0"
flags=0
desc=0
parent=0x0
sibling=0x0
child=0x0},
{start = 0
end=0
name=0x0
desc=0
parent=0x0
sibling=0x0
child=0x0},
{start = 0
end=0
name=0x0
desc=0
parent=0x0
sibling=0x0
child=0x0},
{start = 0
end=0
name=0x0
desc=0
parent=0x0
sibling=0x0
child=0x0},
{start = 0
end=0
name=0x0
desc=0
parent=0x0
sibling=0x0
child=0x0}}
Luís
On Sat, Apr 4, 2020 at 10:39 PM Luís Mendes <luis.p.mendes@gmail.com> wrote:
>
> Hi Bjorn,
>
> Thanks again for your help.
>
> Ok... I've tested your theory on this system and still no changes. The
> BARs 0 and 1 are still not assigned. I should note that this issue
> does not occur only on this particular armhf system, but also on a
> Toradex Apalis IMX8QM, which in this case is an arm64 device and
> doesn't make use of the mvebu infrastructure.
>
> So I did issue the following commands:
> # echo 1 > /sys/bus/pci/devices/0000\:00\:01.0/0000\:01\:00.0/remove
> # echo 1 > /sys/bus/pci/devices/0000\:00\:01.0/dev_rescan"
>
> And the dmesg update after the last command is:
> [ 61.124696] pci 0000:01:00.0: [1ac1:089a] type 00 class 0x0000ff
> [ 61.124732] pci 0000:01:00.0: reg 0x10: [mem 0x00000000-0x00003fff
> 64bit pref]
> [ 61.124743] pci 0000:01:00.0: reg 0x18: [mem 0x00000000-0x000fffff
> 64bit pref]
> [ 61.161258] pci_bus 0000:01: __pci_bus_size_bridges
> [ 61.161270] pci_bus 0000:01: pbus_size_mem: mask 0x2200 type 0x2200
> 0x2200 0x2200 min 0x0 add 0x0 b_res (null) parent (null)
> [ 61.161277] pci_bus 0000:01: pbus_size_mem: mask 0x200 type 0x200
> 0x200 0x200 min 0x0 add 0x0 b_res [mem 0xd0000000-0xd01fffff] parent
> [mem 0xd0000000-0xefffffff]
> [ 61.161281] pci_bus 0000:02: __pci_bus_size_bridges
> [ 61.161286] pci_bus 0000:02: pbus_size_mem: mask 0x2200 type 0x2200
> 0x2200 0x2200 min 0x0 add 0x0 b_res (null) parent (null)
> [ 61.161291] pci_bus 0000:02: pbus_size_mem: mask 0x200 type 0x200
> 0x200 0x200 min 0x0 add 0x0 b_res [mem 0x00000000] parent (null)
> [ 61.161295] pci_bus 0000:00: __pci_bus_assign_resources
> [ 61.161298] pci_bus 0000:00: pbus_assign_resources_sorted
> [ 61.161302] pci 0000:00:01.0: __dev_sort_resources
> [ 61.161305] pci 0000:00:02.0: __dev_sort_resources
> [ 61.161308] __assign_resources_sorted
> [ 61.161311] pci 0000:00:01.0: __pci_bus_assign_resources
> [ 61.161314] pci 0000:00:01.0: pdev_assign_fixed_resources
> [ 61.161317] pci_bus 0000:01: __pci_bus_assign_resources
> [ 61.161319] pci_bus 0000:01: pbus_assign_resources_sorted
> [ 61.161323] pci 0000:01:00.0: __dev_sort_resources
> [ 61.161324] __assign_resources_sorted
> [ 61.161327] pci 0000:01:00.0: __pci_bus_assign_resources
> [ 61.161330] pci 0000:01:00.0: pdev_assign_fixed_resources
> [ 61.161333] pci 0000:00:01.0: PCI bridge to [bus 01]
> [ 61.161340] pci 0000:00:01.0: bridge window [mem 0xd0000000-0xd01fffff]
> [ 61.161344] pci 0000:00:02.0: __pci_bus_assign_resources
> [ 61.161347] pci 0000:00:02.0: pdev_assign_fixed_resources
> [ 61.161350] pci_bus 0000:02: __pci_bus_assign_resources
> [ 61.161353] pci_bus 0000:02: pbus_assign_resources_sorted
> [ 61.161354] __assign_resources_sorted
> [ 61.161357] pci 0000:00:02.0: PCI bridge to [bus 02]
>
> Luís
>
> On Sat, Apr 4, 2020 at 2:32 AM Bjorn Helgaas <helgaas@kernel.org> wrote:
>
> > pci 0000:02:00.0: [10ec:525a] type 00 class 0xff0000
> > pci 0000:02:00.0: reg 0x14: [mem 0x00000000-0x00000fff]
> > pci 0000:02:00.0: BAR 1: assigned [mem 0xf1100000-0xf1100fff]
> > pci 0000:02:00.0: BAR 1: error updating (0xf1100000 != 0xffffffff)
> >
> > So we correctly detected the device, read the cleared BAR, and
> > allocated space for it, and tried to update the BAR. On my system the
> > update failed, I think because of a power management issue (all config
> > reads now return 0xffffffff). But there have been a lot of power
> > management fixes since the v5.2 kernel I'm running, so it's possible
> > you'd have better luck.
> >
> > On your system, I think you would want something like:
> >
> > # echo 1 > /sys/devices/pci0000:00/0000:00:01.0/0000:01:00.0/remove
> > # echo 1 > /sys/devices/pci0000:00/0000:00:01.0/rescan
> >
next prev parent reply other threads:[~2020-04-08 23:05 UTC|newest]
Thread overview: 25+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-03-06 14:32 Problem with PCIe enumeration of Google/Coral TPU Edge module on Linux Luís Mendes
2020-03-06 21:47 ` Bjorn Helgaas
2020-03-07 12:11 ` Luís Mendes
2020-03-07 15:26 ` Luís Mendes
2020-03-07 21:38 ` Bjorn Helgaas
2020-03-08 5:51 ` Nicholas Johnson
2020-03-09 11:21 ` Luís Mendes
2020-03-11 14:20 ` Luís Mendes
2020-03-29 22:11 ` Luís Mendes
2020-03-30 19:49 ` Bjorn Helgaas
2020-03-31 21:28 ` Luís Mendes
2020-04-01 18:16 ` Bjorn Helgaas
2020-04-01 21:20 ` Luís Mendes
2020-04-01 21:55 ` Luís Mendes
2020-04-01 23:31 ` Bjorn Helgaas
2020-04-02 14:13 ` Luís Mendes
2020-04-04 1:32 ` Bjorn Helgaas
2020-04-04 21:39 ` Luís Mendes
2020-04-08 23:05 ` Luís Mendes [this message]
2020-04-09 15:25 ` Luís Mendes
2020-04-09 15:29 ` Luís Mendes
2020-04-09 16:30 ` Bjorn Helgaas
2020-04-09 17:32 ` Luís Mendes
2020-04-09 18:08 ` Bjorn Helgaas
2020-04-09 20:07 ` Luís Mendes
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=CAEzXK1oU4KwatuXAjUwj5arZtdmDSgFsm2vn8DrAWCJu-v-r0g@mail.gmail.com \
--to=luis.p.mendes@gmail.com \
--cc=benh@kernel.crashing.org \
--cc=helgaas@kernel.org \
--cc=jason@lakedaemon.net \
--cc=linux-pci@vger.kernel.org \
--cc=nicholas.johnson-opensource@outlook.com.au \
--cc=thomas.petazzoni@bootlin.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).