[PATCH] ath10k: support PCIe enter L1 state

[PATCH] ath10k: support PCIe enter L1 state

[Wen Gong]

QCA6174A/QCA9377 PCIe chips support PCIe L1 and L1SS, and indicate the
L1/L1SS capabilities in PCI configuration space. Currently ath10k driver
write target PCIe config flags to disallow HW enter into L1, this leads
some HW modules are still powered up even when both system PCIe RC and
QCA6174A/QCA9377 endpoint decides to enter into L1 or L1SS.

This cause ~12 mA power drain of bottom power consumption for all scenarios.
Fix this issue by removing the drive code to write PCIe config flags.

Tested with QCA6174 PCI with firmware
WLAN.RM.4.4.1-00109-QCARMSWPZ-1, but this will also affect QCA9377 PCI.
It's not a regression with new firmware releases.

Signed-off-by: Wen Gong <wgong@codeaurora.org>
---
 drivers/net/wireless/ath/ath10k/pci.c | 21 ---------------------
 1 file changed, 21 deletions(-)

diff --git a/drivers/net/wireless/ath/ath10k/pci.c b/drivers/net/wireless/ath/ath10k/pci.c
index af2cf55..549da7a 100644
--- a/drivers/net/wireless/ath/ath10k/pci.c
+++ b/drivers/net/wireless/ath/ath10k/pci.c
@@ -2301,7 +2301,6 @@ int ath10k_pci_init_config(struct ath10k *ar)
 	u32 pcie_state_targ_addr = 0;
 	u32 pipe_cfg_targ_addr = 0;
 	u32 svc_to_pipe_map = 0;
-	u32 pcie_config_flags = 0;
 	u32 ealloc_value;
 	u32 ealloc_targ_addr;
 	u32 flag2_value;
@@ -2374,26 +2373,6 @@ int ath10k_pci_init_config(struct ath10k *ar)
 		return ret;
 	}
 
-	ret = ath10k_pci_diag_read32(ar, (pcie_state_targ_addr +
-					  offsetof(struct pcie_state,
-						   config_flags)),
-				     &pcie_config_flags);
-	if (ret != 0) {
-		ath10k_err(ar, "Failed to get pcie config_flags: %d\n", ret);
-		return ret;
-	}
-
-	pcie_config_flags &= ~PCIE_CONFIG_FLAG_ENABLE_L1;
-
-	ret = ath10k_pci_diag_write32(ar, (pcie_state_targ_addr +
-					   offsetof(struct pcie_state,
-						    config_flags)),
-				      pcie_config_flags);
-	if (ret != 0) {
-		ath10k_err(ar, "Failed to write pcie config_flags: %d\n", ret);
-		return ret;
-	}
-
 	/* configure early allocation */
 	ealloc_targ_addr = host_interest_item_address(HI_ITEM(hi_early_alloc));
 
-- 
1.9.1

Re: [PATCH] ath10k: support PCIe enter L1 state

[Brian Norris]

Hi Wen,

On Wed, Nov 14, 2018 at 10:50:48AM +0800, Wen Gong wrote:
> QCA6174A/QCA9377 PCIe chips support PCIe L1 and L1SS, and indicate the
> L1/L1SS capabilities in PCI configuration space. Currently ath10k driver
> write target PCIe config flags to disallow HW enter into L1, this leads
> some HW modules are still powered up even when both system PCIe RC and
> QCA6174A/QCA9377 endpoint decides to enter into L1 or L1SS.
> 
> This cause ~12 mA power drain of bottom power consumption for all scenarios.
> Fix this issue by removing the drive code to write PCIe config flags.
> 
> Tested with QCA6174 PCI with firmware
> WLAN.RM.4.4.1-00109-QCARMSWPZ-1, but this will also affect QCA9377 PCI.
> It's not a regression with new firmware releases.
> 
> Signed-off-by: Wen Gong <wgong@codeaurora.org>

Is there some reason L1 was disabled in the first place? Was it known to
be unreliable?

Brian

RE: [PATCH] ath10k: support PCIe enter L1 state

[Wen Gong]

> -----Original Message-----
> From: ath10k <ath10k-bounces@lists.infradead.org> On Behalf Of Brian
> Norris
> Sent: Thursday, November 15, 2018 8:29 AM
> To: Wen Gong <wgong@codeaurora.org>
> Cc: linux-wireless@vger.kernel.org; ath10k@lists.infradead.org
> Subject: [EXTERNAL] Re: [PATCH] ath10k: support PCIe enter L1 state
> 
> Hi Wen,
> 
> >
> > Signed-off-by: Wen Gong <wgong@codeaurora.org>
> 
> Is there some reason L1 was disabled in the first place? Wa
s it known to be
> unreliable?
> 
Hi Brian,
It is a BUG for power, and it is not considered this BUG before.
So this change will fix the bug.

> Brian
> 
> _______________________________________________
> ath10k mailing list
> ath10k@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/ath10k

Re: [PATCH] ath10k: support PCIe enter L1 state

[Brian Norris]

Hi,

On Thu, Nov 15, 2018 at 06:38:25AM +0000, Wen Gong wrote:
> > -----Original Message-----
> > From: ath10k <ath10k-bounces@lists.infradead.org> On Behalf Of Brian Norris
> > 
> > Is there some reason L1 was disabled in the first place? Was it known to be
> > unreliable?
>
> Hi Brian,
> It is a BUG for power, and it is not considered this BUG before.
> So this change will fix the bug.

I understand that the existing behavior is suboptimal for power, but on
the other hand, code that goes out of its way to *clear* the L1 flag
doesn't just pop up out of nowhere. Somebody clearly wrote that! If it
just meant "we didn't verify L1 at first", then maybe it's fine to
enable it unconditionally and see what happens, but if it meant "we
tried L1 on some old chip XXXX and it caused problems", then it would be
nice to know what those problems were.

Or maybe that is hard to figure out, given there's no public git history
tracking the original code, and we just need to try it out.

Anyway, I'm giving it a try here, but I just wanted to ask :)

Thanks,
Brian

Re: [PATCH] ath10k: support PCIe enter L1 state

[Kalle Valo]

Brian Norris <briannorris@chromium.org> writes:
> On Thu, Nov 15, 2018 at 06:38:25AM +0000, Wen Gong wrote:
>> > 
>> > Is there some reason L1 was disabled in the first place? Was it known to be
>> > unreliable?
>>
>> Hi Brian,
>> It is a BUG for power, and it is not considered this BUG before.
>> So this change will fix the bug.
>
> I understand that the existing behavior is suboptimal for power, but on
> the other hand, code that goes out of its way to *clear* the L1 flag
> doesn't just pop up out of nowhere. Somebody clearly wrote that! If it
> just meant "we didn't verify L1 at first", then maybe it's fine to
> enable it unconditionally and see what happens, but if it meant "we
> tried L1 on some old chip XXXX and it caused problems", then it would be
> nice to know what those problems were.
>
> Or maybe that is hard to figure out, given there's no public git history
> tracking the original code, and we just need to try it out.

Yeah, it seems L1 was disabled already on the first ath10k commit:

5e3dd157d7e70 (Kalle Valo                2013-06-12 20:52:10 +0300 2404)        pcie_config_flags &= ~PCIE_CONFIG_FLAG_ENABLE_L1;

I don't remember anymore why but my guess is that the proprietary driver
at the time didn't support it with QCA998X. Or maybe QCA988X doesn't
even support L1? Michal, do you remember?

Related to QCA988X supporting L1 state also the commit log is misleading
as it only talks QCA6174/QCA9377, and has been only tested on QCA6174,
but the actual change enables L1 on _all_ PCI and AHB devices. So this
patch needs a lot more testing so that we have confidence that no
existing setups break.

-- 
Kalle Valo

Re: [PATCH] ath10k: support PCIe enter L1 state

[Michał Kazior]

On Fri, 16 Nov 2018 at 08:00, Kalle Valo <kvalo@codeaurora.org> wrote:
>
> Brian Norris <briannorris@chromium.org> writes:
> > On Thu, Nov 15, 2018 at 06:38:25AM +0000, Wen Gong wrote:
> >> >
> >> > Is there some reason L1 was disabled in the first place? Was it known to be
> >> > unreliable?
> >>
> >> Hi Brian,
> >> It is a BUG for power, and it is not considered this BUG before.
> >> So this change will fix the bug.
> >
> > I understand that the existing behavior is suboptimal for power, but on
> > the other hand, code that goes out of its way to *clear* the L1 flag
> > doesn't just pop up out of nowhere. Somebody clearly wrote that! If it
> > just meant "we didn't verify L1 at first", then maybe it's fine to
> > enable it unconditionally and see what happens, but if it meant "we
> > tried L1 on some old chip XXXX and it caused problems", then it would be
> > nice to know what those problems were.
> >
> > Or maybe that is hard to figure out, given there's no public git history
> > tracking the original code, and we just need to try it out.
>
> Yeah, it seems L1 was disabled already on the first ath10k commit:
>
> 5e3dd157d7e70 (Kalle Valo                2013-06-12 20:52:10 +0300 2404)        pcie_config_flags &= ~PCIE_CONFIG_FLAG_ENABLE_L1;
>
> I don't remember anymore why but my guess is that the proprietary driver
> at the time didn't support it with QCA998X. Or maybe QCA988X doesn't
> even support L1? Michal, do you remember?

Proprietary driver has it ifdef-ed to enable/disable.

Older driver enabled it only for some station-only target/product so
by default QCA988X would build with L1 flag masked out. It made sense
to be conservative and change as little as possible to avoid random
bugs and breakage - so the logic was inherited minus the build-time
ifdef.

However 10.4 driver seems to enable it unconditionally. I'm not sure
if this depends on target firmware in any way or if some other host
driver or bus settings need to be pre-set before L1 can be expected to
work reliably.

I guess there's no way other than testing it out.


Michał

Re: [PATCH] ath10k: support PCIe enter L1 state

[Kalle Valo]

Michał Kazior <kazikcz@gmail.com> writes:

> On Fri, 16 Nov 2018 at 08:00, Kalle Valo <kvalo@codeaurora.org> wrote:
>>
>> Brian Norris <briannorris@chromium.org> writes:
>> > On Thu, Nov 15, 2018 at 06:38:25AM +0000, Wen Gong wrote:
>> >> >
>> >> > Is there some reason L1 was disabled in the first place? Was it known to be
>> >> > unreliable?
>> >>
>> >> Hi Brian,
>> >> It is a BUG for power, and it is not considered this BUG before.
>> >> So this change will fix the bug.
>> >
>> > I understand that the existing behavior is suboptimal for power, but on
>> > the other hand, code that goes out of its way to *clear* the L1 flag
>> > doesn't just pop up out of nowhere. Somebody clearly wrote that! If it
>> > just meant "we didn't verify L1 at first", then maybe it's fine to
>> > enable it unconditionally and see what happens, but if it meant "we
>> > tried L1 on some old chip XXXX and it caused problems", then it would be
>> > nice to know what those problems were.
>> >
>> > Or maybe that is hard to figure out, given there's no public git history
>> > tracking the original code, and we just need to try it out.
>>
>> Yeah, it seems L1 was disabled already on the first ath10k commit:
>>
>> 5e3dd157d7e70 (Kalle Valo 2013-06-12 20:52:10 +0300 2404)
>> pcie_config_flags &= ~PCIE_CONFIG_FLAG_ENABLE_L1;
>>
>> I don't remember anymore why but my guess is that the proprietary driver
>> at the time didn't support it with QCA998X. Or maybe QCA988X doesn't
>> even support L1? Michal, do you remember?
>
> Proprietary driver has it ifdef-ed to enable/disable.
>
> Older driver enabled it only for some station-only target/product so
> by default QCA988X would build with L1 flag masked out. It made sense
> to be conservative and change as little as possible to avoid random
> bugs and breakage - so the logic was inherited minus the build-time
> ifdef.
>
> However 10.4 driver seems to enable it unconditionally. I'm not sure
> if this depends on target firmware in any way or if some other host
> driver or bus settings need to be pre-set before L1 can be expected to
> work reliably.
>
> I guess there's no way other than testing it out.

No replies from anyone (including Wen) for 3 months about testing this
patch on anything else than QCA6174. So I'll drop this now, please
resubmit once test coverage is better.

-- 
Kalle Valo

Re: [PATCH] ath10k: support PCIe enter L1 state

[Brian Norris]

On Fri, Feb 8, 2019 at 5:42 AM Kalle Valo <kvalo@codeaurora.org> wrote:
> No replies from anyone (including Wen) for 3 months about testing this
> patch on anything else than QCA6174. So I'll drop this now, please
> resubmit once test coverage is better.

I know this isn't exactly what you're asking for, but FWIW we've been
using this since late November on all our QCA6174 products. No issues
seen as far as I know, and we have seen some power savings.

Regards,
Brian

Re: [PATCH] ath10k: support PCIe enter L1 state

[Kalle Valo]

Brian Norris <briannorris@chromium.org> writes:

> On Fri, Feb 8, 2019 at 5:42 AM Kalle Valo <kvalo@codeaurora.org> wrote:
>> No replies from anyone (including Wen) for 3 months about testing this
>> patch on anything else than QCA6174. So I'll drop this now, please
>> resubmit once test coverage is better.
>
> I know this isn't exactly what you're asking for, but FWIW we've been
> using this since late November on all our QCA6174 products. No issues
> seen as far as I know, and we have seen some power savings.

Thanks for the feedback, this is very good to know. I also would like to
apply this but not before we have some testing feedback from AP chipsets
like QCA988X or QCA9984. Wen, are you planning to test those and
resubmit?

-- 
Kalle Valo

Re: [PATCH] ath10k: support PCIe enter L1 state

[Brian Norris]

On Fri, Mar 8, 2019 at 1:42 AM Kalle Valo <kvalo@codeaurora.org> wrote:
> Brian Norris <briannorris@chromium.org> writes:
> > On Fri, Feb 8, 2019 at 5:42 AM Kalle Valo <kvalo@codeaurora.org> wrote:
> >> No replies from anyone (including Wen) for 3 months about testing this
> >> patch on anything else than QCA6174. So I'll drop this now, please
> >> resubmit once test coverage is better.
> >
> > I know this isn't exactly what you're asking for, but FWIW we've been
> > using this since late November on all our QCA6174 products. No issues
> > seen as far as I know, and we have seen some power savings.
>
> Thanks for the feedback, this is very good to know. I also would like to
> apply this but not before we have some testing feedback from AP chipsets
> like QCA988X or QCA9984. Wen, are you planning to test those and
> resubmit?

May I ping here? We (Chromium OS) are continuing to carry this patch,
and would love to come up with something that can land upstream. If
necessary, I can rework it to apply more specifically -- e.g., only to
QCA6174 PCI IDs.

Thanks,
Brian

Re: [PATCH] ath10k: support PCIe enter L1 state

[Kalle Valo]

Brian Norris <briannorris@chromium.org> writes:

> On Fri, Mar 8, 2019 at 1:42 AM Kalle Valo <kvalo@codeaurora.org> wrote:
>> Brian Norris <briannorris@chromium.org> writes:
>> > On Fri, Feb 8, 2019 at 5:42 AM Kalle Valo <kvalo@codeaurora.org> wrote:
>> >> No replies from anyone (including Wen) for 3 months about testing this
>> >> patch on anything else than QCA6174. So I'll drop this now, please
>> >> resubmit once test coverage is better.
>> >
>> > I know this isn't exactly what you're asking for, but FWIW we've been
>> > using this since late November on all our QCA6174 products. No issues
>> > seen as far as I know, and we have seen some power savings.
>>
>> Thanks for the feedback, this is very good to know. I also would like to
>> apply this but not before we have some testing feedback from AP chipsets
>> like QCA988X or QCA9984. Wen, are you planning to test those and
>> resubmit?
>
> May I ping here? We (Chromium OS) are continuing to carry this patch,
> and would love to come up with something that can land upstream. If
> necessary, I can rework it to apply more specifically -- e.g., only to
> QCA6174 PCI IDs.

Yeah, enabling only for QCA6174 sounds best.

-- 
https://wireless.wiki.kernel.org/en/developers/documentation/submittingpatches