Re: [PATCH v3] PCI/ASPM: Update LTR threshold based upon reported max latencies

From: Krishna Chaitanya Chundru <quic_krichai@quicinc.com>
To: <helgaas@kernel.org>
Cc: linux-pci@vger.kernel.org, linux-arm-msm@vger.kernel.org,
	linux-kernel@vger.kernel.org, quic_vbadigan@quicinc.com,
	quic_ramkri@quicinc.com, manivannan.sadhasivam@linaro.org,
	swboyd@chromium.org, "Bjorn Helgaas" <bhelgaas@google.com>,
	"Saheed O. Bolarinwa" <refactormyself@gmail.com>,
	"Krzysztof Wilczyński" <kw@linux.com>,
	"Rajat Jain" <rajatja@google.com>,
	vidyas@nvidia.com, kenny@panix.com
Subject: Re: [PATCH v3] PCI/ASPM: Update LTR threshold based upon reported max latencies
Date: Wed, 1 Jun 2022 17:57:53 +0530	[thread overview]
Message-ID: <91b75542-8e4c-5b91-bbfd-38ffc456c12e@quicinc.com> (raw)
In-Reply-To: <1654086232-17055-1-git-send-email-quic_krichai@quicinc.com>

[+cc kenny, vidya]

On 6/1/2022 5:53 PM, Krishna chaitanya chundru wrote:
> In ASPM driver, LTR threshold scale and value is updating based on
> tcommon_mode and t_poweron values. In kioxia NVMe L1.2 is failing due to
> LTR threshold scale and value is greater values than max snoop/non-snoop
> value.
>
> Based on PCIe r4.1, sec 5.5.1, L1.2 substate must be entered when
> reported snoop/no-snoop values is greather than or equal to
> LTR_L1.2_THRESHOLD value.
>
> Suggested-by: Prasad Malisetty  <quic_pmaliset@quicinc.com>
> Signed-off-by: Krishna chaitanya chundru <quic_krichai@quicinc.com>
> ---
>
> I am takking this patch forward as prasad is no more working with our org.
>
> Changes since v2:
> 	- Replaced LTRME logic with max snoop/no-snoop latencies check.
> Changes since v1:
> 	- Added missing variable declaration in v1 patch
> ---
>   drivers/pci/pcie/aspm.c | 22 +++++++++++++++++++++-
>   1 file changed, 21 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/pci/pcie/aspm.c b/drivers/pci/pcie/aspm.c
> index a96b742..4a15e50 100644
> --- a/drivers/pci/pcie/aspm.c
> +++ b/drivers/pci/pcie/aspm.c
> @@ -465,10 +465,19 @@ static void aspm_calc_l1ss_info(struct pcie_link_state *link,
>   	u32 ctl1 = 0, ctl2 = 0;
>   	u32 pctl1, pctl2, cctl1, cctl2;
>   	u32 pl1_2_enables, cl1_2_enables;
> +	int ltr;
> +	u16 max_snoop_lat = 0, max_nosnoop_lat = 0;
>   
>   	if (!(link->aspm_support & ASPM_STATE_L1_2_MASK))
>   		return;
>   
> +	ltr = pci_find_ext_capability(child, PCI_EXT_CAP_ID_LTR);
> +	if (!ltr)
> +		return;
> +
> +	pci_read_config_word(child, ltr + PCI_LTR_MAX_SNOOP_LAT, &max_snoop_lat);
> +	pci_read_config_word(child, ltr + PCI_LTR_MAX_NOSNOOP_LAT, &max_nosnoop_lat);
> +
>   	/* Choose the greater of the two Port Common_Mode_Restore_Times */
>   	val1 = (parent_l1ss_cap & PCI_L1SS_CAP_CM_RESTORE_TIME) >> 8;
>   	val2 = (child_l1ss_cap & PCI_L1SS_CAP_CM_RESTORE_TIME) >> 8;
> @@ -501,7 +510,18 @@ static void aspm_calc_l1ss_info(struct pcie_link_state *link,
>   	 */
>   	l1_2_threshold = 2 + 4 + t_common_mode + t_power_on;
>   	encode_l12_threshold(l1_2_threshold, &scale, &value);
> -	ctl1 |= t_common_mode << 8 | scale << 29 | value << 16;
> +
> +	/*
> +	 * If the max snoop and no snoop latencies are '0', then avoid updating scale
> +	 * and value.
> +	 *
> +	 * Based on PCIe r4.1, sec 5.5.1, L1.2 substate must be entered when reported
> +	 * snoop/no-snoop values is greather than or equal to LTR_L1.2_THRESHOLD value.
> +	 */
> +	if ((max_snoop_lat == 0) && (max_nosnoop_lat == 0))
> +		ctl1 |= t_common_mode << 8;
> +	else
> +		ctl1 |= t_common_mode << 8 | scale << 29 | value << 16;
>   
>   	/* Some broken devices only support dword access to L1 SS */
>   	pci_read_config_dword(parent, parent->l1ss + PCI_L1SS_CTL1, &pctl1);