From: Jim Quinlan <james.quinlan@broadcom.com> To: linux-pci@vger.kernel.org, Nicolas Saenz Julienne <nsaenz@kernel.org>, Bjorn Helgaas <bhelgaas@google.com>, Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>, Cyril Brulebois <kibi@debian.org>, Phil Elwell <phil@raspberrypi.com>, bcm-kernel-feedback-list@broadcom.com, james.quinlan@broadcom.com Cc: "Florian Fainelli" <florian.fainelli@broadcom.com>, "Jim Quinlan" <jim2101024@gmail.com>, "Lorenzo Pieralisi" <lpieralisi@kernel.org>, "Krzysztof Wilczyński" <kw@linux.com>, "Rob Herring" <robh@kernel.org>, linux-rpi-kernel@lists.infradead.org (moderated list:BROADCOM BCM2711/BCM2835 ARM ARCHITECTURE), linux-arm-kernel@lists.infradead.org (moderated list:BROADCOM BCM2711/BCM2835 ARM ARCHITECTURE), linux-kernel@vger.kernel.org (open list) Subject: [PATCH v9 3/4] PCI: brcmstb: Set downstream maximum {no-}snoop LTR values Date: Wed, 3 Apr 2024 17:39:00 -0400 [thread overview] Message-ID: <20240403213902.26391-4-james.quinlan@broadcom.com> (raw) In-Reply-To: <20240403213902.26391-1-james.quinlan@broadcom.com> [-- Attachment #1: Type: text/plain, Size: 3929 bytes --] Most of our systems do not have FW or ACPI, so it is up to the RC driver to set the maximum LTR {no-}snoop latency values of downstream devices. We set them to a value that is slightly smaller than the value of our internal bus timeout register. Signed-off-by: Jim Quinlan <james.quinlan@broadcom.com> --- drivers/pci/controller/pcie-brcmstb.c | 68 ++++++++++++++++++++++++++- 1 file changed, 66 insertions(+), 2 deletions(-) diff --git a/drivers/pci/controller/pcie-brcmstb.c b/drivers/pci/controller/pcie-brcmstb.c index e3480ca4cd57..3d08b92d5bb8 100644 --- a/drivers/pci/controller/pcie-brcmstb.c +++ b/drivers/pci/controller/pcie-brcmstb.c @@ -182,6 +182,20 @@ #define DATA_ADDR(pcie) (pcie->reg_offsets[EXT_CFG_DATA]) #define PCIE_RGR1_SW_INIT_1(pcie) (pcie->reg_offsets[RGR1_SW_INIT_1]) +/* + * What we call "LTR_FMT" is the 16 bit latency field format: + * [15:15] Requirement bit + * [12:10] Latency scale + * [09:00] Latency value + */ +#define LTR_FMT_TO_NS(p) (FIELD_GET(GENMASK(15, 15), (p)) \ + * ((unsigned long long)FIELD_GET(GENMASK(9, 0), (p)) \ + << (FIELD_GET(GENMASK(12, 10), (p)) * 5))) +#define BRCM_LTR_MAX_SCALE 4 /* Scale==4 => Each unit is 1,048,576ns */ +#define BRCM_LTR_MAX_VALUE 9 /* Using the above scale, roughly 9.4 ms */ +#define BRCM_LTR_MAX_NS ((unsigned long long)(BRCM_LTR_MAX_VALUE \ + << (5 * BRCM_LTR_MAX_SCALE))) + /* Rescal registers */ #define PCIE_DVT_PMU_PCIE_PHY_CTRL 0xc700 #define PCIE_DVT_PMU_PCIE_PHY_CTRL_DAST_NFLDS 0x3 @@ -679,6 +693,49 @@ static void brcm_extend_internal_bus_timeout(struct brcm_pcie *pcie, u32 nsec) writel(216 * timeout_us, pcie->base + REG_OFFSET); } +/* Sets downstream device latency tolerance registers to max we can handle */ +static int brcm_set_dev_ltr_max(struct pci_dev *dev, void *data) +{ + u16 ltr_cap_offset = pci_find_ext_capability(dev, PCI_EXT_CAP_ID_LTR); + u16 ltr_fmt_cur, ltr_fmt = *(u16 *)data; + unsigned long long cur_nsec; + static const u16 reg_offsets[2] = { + PCI_LTR_MAX_SNOOP_LAT, + PCI_LTR_MAX_NOSNOOP_LAT, + }; + unsigned int i; + + if (!ltr_cap_offset || !dev->ltr_path) + return 0; + + /* + * FW may have already written a value so we want to respect that + * value if it is lower than ltr_fmt. Update the current value if + * it is 0 or if the new value is less than the current. + */ + for (i = 0; i < ARRAY_SIZE(reg_offsets); i++) { + pci_read_config_word(dev, ltr_cap_offset + reg_offsets[i], + <r_fmt_cur); + cur_nsec = LTR_FMT_TO_NS(ltr_fmt_cur); + if (cur_nsec == 0 || cur_nsec > BRCM_LTR_MAX_NS) + pci_write_config_word(dev, ltr_cap_offset + + reg_offsets[i], ltr_fmt); + } + + return 0; +} + +void brcm_set_downstream_devs_ltr_max(struct brcm_pcie *pcie) +{ + struct pci_host_bridge *bridge = pci_host_bridge_from_priv(pcie); + u16 ltr_fmt = FIELD_PREP(GENMASK(9, 0), BRCM_LTR_MAX_VALUE) + | FIELD_PREP(GENMASK(12, 10), BRCM_LTR_MAX_SCALE) + | GENMASK(15, 15); + + if (bridge->native_ltr) + pci_walk_bus(bridge->bus, brcm_set_dev_ltr_max, <r_fmt); +} + /* The controller is capable of serving in both RC and EP roles */ static bool brcm_pcie_rc_mode(struct brcm_pcie *pcie) { @@ -1074,8 +1131,12 @@ static int brcm_pcie_start_link(struct brcm_pcie *pcie) return -ENODEV; } - /* Extend internal bus timeout to 8ms or so */ - brcm_extend_internal_bus_timeout(pcie, SZ_8M); + /* + * Extend internal bus timeout to 8-10ms, specifically to a value + * that is slightly larger than what we are using for the max + * {no-}snoop latency we will set in downstream devices. + */ + brcm_extend_internal_bus_timeout(pcie, BRCM_LTR_MAX_NS + 1000); if (pcie->gen) brcm_pcie_set_gen(pcie, pcie->gen); @@ -1616,6 +1677,9 @@ static int brcm_pcie_probe(struct platform_device *pdev) return ret; } + if (IS_ENABLED(CONFIG_PCIEASPM)) + brcm_set_downstream_devs_ltr_max(pcie); + return 0; fail: -- 2.17.1 [-- Attachment #2: S/MIME Cryptographic Signature --] [-- Type: application/pkcs7-signature, Size: 4210 bytes --]
WARNING: multiple messages have this Message-ID (diff)
From: Jim Quinlan <james.quinlan@broadcom.com> To: linux-pci@vger.kernel.org, Nicolas Saenz Julienne <nsaenz@kernel.org>, Bjorn Helgaas <bhelgaas@google.com>, Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>, Cyril Brulebois <kibi@debian.org>, Phil Elwell <phil@raspberrypi.com>, bcm-kernel-feedback-list@broadcom.com, james.quinlan@broadcom.com Cc: "Florian Fainelli" <florian.fainelli@broadcom.com>, "Jim Quinlan" <jim2101024@gmail.com>, "Lorenzo Pieralisi" <lpieralisi@kernel.org>, "Krzysztof Wilczyński" <kw@linux.com>, "Rob Herring" <robh@kernel.org>, linux-rpi-kernel@lists.infradead.org (moderated list:BROADCOM BCM2711/BCM2835 ARM ARCHITECTURE), linux-arm-kernel@lists.infradead.org (moderated list:BROADCOM BCM2711/BCM2835 ARM ARCHITECTURE), linux-kernel@vger.kernel.org (open list) Subject: [PATCH v9 3/4] PCI: brcmstb: Set downstream maximum {no-}snoop LTR values Date: Wed, 3 Apr 2024 17:39:00 -0400 [thread overview] Message-ID: <20240403213902.26391-4-james.quinlan@broadcom.com> (raw) In-Reply-To: <20240403213902.26391-1-james.quinlan@broadcom.com> [-- Attachment #1.1: Type: text/plain, Size: 3929 bytes --] Most of our systems do not have FW or ACPI, so it is up to the RC driver to set the maximum LTR {no-}snoop latency values of downstream devices. We set them to a value that is slightly smaller than the value of our internal bus timeout register. Signed-off-by: Jim Quinlan <james.quinlan@broadcom.com> --- drivers/pci/controller/pcie-brcmstb.c | 68 ++++++++++++++++++++++++++- 1 file changed, 66 insertions(+), 2 deletions(-) diff --git a/drivers/pci/controller/pcie-brcmstb.c b/drivers/pci/controller/pcie-brcmstb.c index e3480ca4cd57..3d08b92d5bb8 100644 --- a/drivers/pci/controller/pcie-brcmstb.c +++ b/drivers/pci/controller/pcie-brcmstb.c @@ -182,6 +182,20 @@ #define DATA_ADDR(pcie) (pcie->reg_offsets[EXT_CFG_DATA]) #define PCIE_RGR1_SW_INIT_1(pcie) (pcie->reg_offsets[RGR1_SW_INIT_1]) +/* + * What we call "LTR_FMT" is the 16 bit latency field format: + * [15:15] Requirement bit + * [12:10] Latency scale + * [09:00] Latency value + */ +#define LTR_FMT_TO_NS(p) (FIELD_GET(GENMASK(15, 15), (p)) \ + * ((unsigned long long)FIELD_GET(GENMASK(9, 0), (p)) \ + << (FIELD_GET(GENMASK(12, 10), (p)) * 5))) +#define BRCM_LTR_MAX_SCALE 4 /* Scale==4 => Each unit is 1,048,576ns */ +#define BRCM_LTR_MAX_VALUE 9 /* Using the above scale, roughly 9.4 ms */ +#define BRCM_LTR_MAX_NS ((unsigned long long)(BRCM_LTR_MAX_VALUE \ + << (5 * BRCM_LTR_MAX_SCALE))) + /* Rescal registers */ #define PCIE_DVT_PMU_PCIE_PHY_CTRL 0xc700 #define PCIE_DVT_PMU_PCIE_PHY_CTRL_DAST_NFLDS 0x3 @@ -679,6 +693,49 @@ static void brcm_extend_internal_bus_timeout(struct brcm_pcie *pcie, u32 nsec) writel(216 * timeout_us, pcie->base + REG_OFFSET); } +/* Sets downstream device latency tolerance registers to max we can handle */ +static int brcm_set_dev_ltr_max(struct pci_dev *dev, void *data) +{ + u16 ltr_cap_offset = pci_find_ext_capability(dev, PCI_EXT_CAP_ID_LTR); + u16 ltr_fmt_cur, ltr_fmt = *(u16 *)data; + unsigned long long cur_nsec; + static const u16 reg_offsets[2] = { + PCI_LTR_MAX_SNOOP_LAT, + PCI_LTR_MAX_NOSNOOP_LAT, + }; + unsigned int i; + + if (!ltr_cap_offset || !dev->ltr_path) + return 0; + + /* + * FW may have already written a value so we want to respect that + * value if it is lower than ltr_fmt. Update the current value if + * it is 0 or if the new value is less than the current. + */ + for (i = 0; i < ARRAY_SIZE(reg_offsets); i++) { + pci_read_config_word(dev, ltr_cap_offset + reg_offsets[i], + <r_fmt_cur); + cur_nsec = LTR_FMT_TO_NS(ltr_fmt_cur); + if (cur_nsec == 0 || cur_nsec > BRCM_LTR_MAX_NS) + pci_write_config_word(dev, ltr_cap_offset + + reg_offsets[i], ltr_fmt); + } + + return 0; +} + +void brcm_set_downstream_devs_ltr_max(struct brcm_pcie *pcie) +{ + struct pci_host_bridge *bridge = pci_host_bridge_from_priv(pcie); + u16 ltr_fmt = FIELD_PREP(GENMASK(9, 0), BRCM_LTR_MAX_VALUE) + | FIELD_PREP(GENMASK(12, 10), BRCM_LTR_MAX_SCALE) + | GENMASK(15, 15); + + if (bridge->native_ltr) + pci_walk_bus(bridge->bus, brcm_set_dev_ltr_max, <r_fmt); +} + /* The controller is capable of serving in both RC and EP roles */ static bool brcm_pcie_rc_mode(struct brcm_pcie *pcie) { @@ -1074,8 +1131,12 @@ static int brcm_pcie_start_link(struct brcm_pcie *pcie) return -ENODEV; } - /* Extend internal bus timeout to 8ms or so */ - brcm_extend_internal_bus_timeout(pcie, SZ_8M); + /* + * Extend internal bus timeout to 8-10ms, specifically to a value + * that is slightly larger than what we are using for the max + * {no-}snoop latency we will set in downstream devices. + */ + brcm_extend_internal_bus_timeout(pcie, BRCM_LTR_MAX_NS + 1000); if (pcie->gen) brcm_pcie_set_gen(pcie, pcie->gen); @@ -1616,6 +1677,9 @@ static int brcm_pcie_probe(struct platform_device *pdev) return ret; } + if (IS_ENABLED(CONFIG_PCIEASPM)) + brcm_set_downstream_devs_ltr_max(pcie); + return 0; fail: -- 2.17.1 [-- Attachment #1.2: S/MIME Cryptographic Signature --] [-- Type: application/pkcs7-signature, Size: 4210 bytes --] [-- Attachment #2: Type: text/plain, Size: 176 bytes --] _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
next prev parent reply other threads:[~2024-04-03 21:39 UTC|newest] Thread overview: 26+ messages / expand[flat|nested] mbox.gz Atom feed top 2024-04-03 21:38 [PATCH v9 0/4] PCI: brcmstb: Configure appropriate HW CLKREQ# mode Jim Quinlan 2024-04-03 21:38 ` Jim Quinlan 2024-04-03 21:38 ` [PATCH v9 1/4] dt-bindings: PCI: brcmstb: Add property "brcm,clkreq-mode" Jim Quinlan 2024-04-03 21:38 ` Jim Quinlan 2024-04-03 21:38 ` [PATCH v9 2/4] PCI: brcmstb: Set reasonable value for internal bus timeout Jim Quinlan 2024-04-03 21:38 ` Jim Quinlan 2024-05-06 22:45 ` Bjorn Helgaas 2024-05-06 22:45 ` Bjorn Helgaas 2024-04-03 21:39 ` Jim Quinlan [this message] 2024-04-03 21:39 ` [PATCH v9 3/4] PCI: brcmstb: Set downstream maximum {no-}snoop LTR values Jim Quinlan 2024-04-04 16:47 ` kernel test robot 2024-04-04 16:48 ` kernel test robot 2024-04-03 21:39 ` [PATCH v9 4/4] PCI: brcmstb: Configure HW CLKREQ# mode appropriate for downstream device Jim Quinlan 2024-04-03 21:39 ` Jim Quinlan 2024-05-06 23:20 ` Bjorn Helgaas 2024-05-06 23:20 ` Bjorn Helgaas 2024-05-08 17:55 ` Jim Quinlan 2024-05-08 17:55 ` Jim Quinlan 2024-05-08 23:33 ` Bjorn Helgaas 2024-05-08 23:33 ` Bjorn Helgaas 2024-04-04 20:01 ` [PATCH v9 0/4] PCI: brcmstb: Configure appropriate HW CLKREQ# mode Cyril Brulebois 2024-04-04 20:01 ` Cyril Brulebois 2024-04-30 21:02 ` Jim Quinlan 2024-04-30 21:02 ` Jim Quinlan 2024-05-06 22:31 ` Bjorn Helgaas 2024-05-06 22:31 ` Bjorn Helgaas
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20240403213902.26391-4-james.quinlan@broadcom.com \ --to=james.quinlan@broadcom.com \ --cc=bcm-kernel-feedback-list@broadcom.com \ --cc=bhelgaas@google.com \ --cc=florian.fainelli@broadcom.com \ --cc=jim2101024@gmail.com \ --cc=kibi@debian.org \ --cc=kw@linux.com \ --cc=linux-arm-kernel@lists.infradead.org \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-pci@vger.kernel.org \ --cc=linux-rpi-kernel@lists.infradead.org \ --cc=lorenzo.pieralisi@arm.com \ --cc=lpieralisi@kernel.org \ --cc=nsaenz@kernel.org \ --cc=phil@raspberrypi.com \ --cc=robh@kernel.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.