[Intel-wired-lan] [PATCH 1/3] e1000e: Set HTHRESH when PTHRESH is used

[Intel-wired-lan] [PATCH 1/3] e1000e: Set HTHRESH when PTHRESH is used

[Matt Turner]

From: Matt Turner <matt.turner@intel.com>

According to section 12.0.3.4.13 "Receive Descriptor Control - RXDCTL"
of the Intel? 82579 Gigabit Ethernet PHY Datasheet v2.1:

    "HTHRESH should be given a non zero value when ever PTHRESH is
     used."

In RXDCTL(0), PTHRESH lives at bits 5:0, and HTHREST lives at bits 13:8.
Set only bit 8 of HTHREST as is done in e1000_flush_rx_ring(). Found by
inspection.

Signed-off-by: Matt Turner <matt.turner@intel.com>
---
 drivers/net/ethernet/intel/e1000e/netdev.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/net/ethernet/intel/e1000e/netdev.c b/drivers/net/ethernet/intel/e1000e/netdev.c
index 2dcb5463d9b8..286d0277284e 100644
--- a/drivers/net/ethernet/intel/e1000e/netdev.c
+++ b/drivers/net/ethernet/intel/e1000e/netdev.c
@@ -3285,7 +3285,7 @@ static void e1000_configure_rx(struct e1000_adapter *adapter)
 		if (adapter->flags & FLAG_IS_ICH) {
 			u32 rxdctl = er32(RXDCTL(0));
 
-			ew32(RXDCTL(0), rxdctl | 0x3);
+			ew32(RXDCTL(0), rxdctl | 0x3 | BIT(8));
 		}
 
 		pm_qos_update_request(&adapter->pm_qos_req, lat);
-- 
2.13.6

[Intel-wired-lan] [PATCH 2/3] e1000e: Alert the user that C-states will be disabled by enabling jumbo frames

[Matt Turner]

From: Matt Turner <matt.turner@intel.com>

I personally spent a long time trying to decypher why my CPU would not
reach deeper C-states. Let's just tell the next user what's going on.

Signed-off-by: Matt Turner <matt.turner@intel.com>
---
 drivers/net/ethernet/intel/e1000e/netdev.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/drivers/net/ethernet/intel/e1000e/netdev.c b/drivers/net/ethernet/intel/e1000e/netdev.c
index 286d0277284e..4dcff481c4b4 100644
--- a/drivers/net/ethernet/intel/e1000e/netdev.c
+++ b/drivers/net/ethernet/intel/e1000e/netdev.c
@@ -3288,6 +3288,9 @@ static void e1000_configure_rx(struct e1000_adapter *adapter)
 			ew32(RXDCTL(0), rxdctl | 0x3 | BIT(8));
 		}
 
+		dev_info(&adapter->pdev->dev,
+			 "Some CPU C-states have been disabled in order to "
+			 "enable jumbo frames\n");
 		pm_qos_update_request(&adapter->pm_qos_req, lat);
 	} else {
 		pm_qos_update_request(&adapter->pm_qos_req,
-- 
2.13.6

[Intel-wired-lan] [PATCH 2/3] e1000e: Alert the user that C-states will be disabled by enabling jumbo frames

[Neftin, Sasha]

On 11/8/2017 00:13, Matt Turner wrote:
> From: Matt Turner <matt.turner@intel.com>
>
> I personally spent a long time trying to decypher why my CPU would not
> reach deeper C-states. Let's just tell the next user what's going on.
>
> Signed-off-by: Matt Turner <matt.turner@intel.com>
> ---
>   drivers/net/ethernet/intel/e1000e/netdev.c | 3 +++
>   1 file changed, 3 insertions(+)
>
> diff --git a/drivers/net/ethernet/intel/e1000e/netdev.c b/drivers/net/ethernet/intel/e1000e/netdev.c
> index 286d0277284e..4dcff481c4b4 100644
> --- a/drivers/net/ethernet/intel/e1000e/netdev.c
> +++ b/drivers/net/ethernet/intel/e1000e/netdev.c
> @@ -3288,6 +3288,9 @@ static void e1000_configure_rx(struct e1000_adapter *adapter)
>   			ew32(RXDCTL(0), rxdctl | 0x3 | BIT(8));
>   		}
>   
> +		dev_info(&adapter->pdev->dev,
> +			 "Some CPU C-states have been disabled in order to "
> +			 "enable jumbo frames\n");
>   		pm_qos_update_request(&adapter->pm_qos_req, lat);
>   	} else {
>   		pm_qos_update_request(&adapter->pm_qos_req,

Ack. Let's accept this comments.

[Intel-wired-lan] [PATCH 2/3] e1000e: Alert the user that C-states will be disabled by enabling jumbo frames

[Shannon Nelson]

On 11/9/2017 1:36 AM, Neftin, Sasha wrote:
> On 11/8/2017 00:13, Matt Turner wrote:
>> From: Matt Turner <matt.turner@intel.com>
>>
>> I personally spent a long time trying to decypher why my CPU would not
>> reach deeper C-states. Let's just tell the next user what's going on.
>>
>> Signed-off-by: Matt Turner <matt.turner@intel.com>
>> ---
>> ? drivers/net/ethernet/intel/e1000e/netdev.c | 3 +++
>> ? 1 file changed, 3 insertions(+)
>>
>> diff --git a/drivers/net/ethernet/intel/e1000e/netdev.c 
>> b/drivers/net/ethernet/intel/e1000e/netdev.c
>> index 286d0277284e..4dcff481c4b4 100644
>> --- a/drivers/net/ethernet/intel/e1000e/netdev.c
>> +++ b/drivers/net/ethernet/intel/e1000e/netdev.c
>> @@ -3288,6 +3288,9 @@ static void e1000_configure_rx(struct 
>> e1000_adapter *adapter)
>> ????????????? ew32(RXDCTL(0), rxdctl | 0x3 | BIT(8));
>> ????????? }
>> +??????? dev_info(&adapter->pdev->dev,
>> +???????????? "Some CPU C-states have been disabled in order to "
>> +???????????? "enable jumbo frames\n");
>> ????????? pm_qos_update_request(&adapter->pm_qos_req, lat);
>> ????? } else {
>> ????????? pm_qos_update_request(&adapter->pm_qos_req,
> 
> Ack. Let's accept this comments.

But let's make that a single string, not split between lines.
sln


> 
> _______________________________________________
> Intel-wired-lan mailing list
> Intel-wired-lan at osuosl.org
> https://lists.osuosl.org/mailman/listinfo/intel-wired-lan

[Intel-wired-lan] [PATCH 2/3] e1000e: Alert the user that C-states will be disabled by enabling jumbo frames

[Matt Turner]

From: Matt Turner <matt.turner@intel.com>

I personally spent a long time trying to decypher why my CPU would not
reach deeper C-states. Let's just tell the next user what's going on.

Signed-off-by: Matt Turner <matt.turner@intel.com>
---
v2: Per Shannon, put string on one line for easier grepping

 drivers/net/ethernet/intel/e1000e/netdev.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/drivers/net/ethernet/intel/e1000e/netdev.c b/drivers/net/ethernet/intel/e1000e/netdev.c
index 286d0277284e..31577dad22cb 100644
--- a/drivers/net/ethernet/intel/e1000e/netdev.c
+++ b/drivers/net/ethernet/intel/e1000e/netdev.c
@@ -3288,6 +3288,8 @@ static void e1000_configure_rx(struct e1000_adapter *adapter)
 			ew32(RXDCTL(0), rxdctl | 0x3 | BIT(8));
 		}
 
+		dev_info(&adapter->pdev->dev,
+			 "Some CPU C-states have been disabled in order to enable jumbo frames\n");
 		pm_qos_update_request(&adapter->pm_qos_req, lat);
 	} else {
 		pm_qos_update_request(&adapter->pm_qos_req,
-- 
2.13.6

[Intel-wired-lan] [PATCH 2/3] e1000e: Alert the user that C-states will be disabled by enabling jumbo frames

[Shannon Nelson]

On 11/14/2017 3:51 PM, Matt Turner wrote:
> From: Matt Turner <matt.turner@intel.com>
> 
> I personally spent a long time trying to decypher why my CPU would not
> reach deeper C-states. Let's just tell the next user what's going on.
> 
> Signed-off-by: Matt Turner <matt.turner@intel.com>
> ---
> v2: Per Shannon, put string on one line for easier grepping

Acked-by: Shannon Nelson <shannon.nelson@oracle.com>

> 
>   drivers/net/ethernet/intel/e1000e/netdev.c | 2 ++
>   1 file changed, 2 insertions(+)
> 
> diff --git a/drivers/net/ethernet/intel/e1000e/netdev.c b/drivers/net/ethernet/intel/e1000e/netdev.c
> index 286d0277284e..31577dad22cb 100644
> --- a/drivers/net/ethernet/intel/e1000e/netdev.c
> +++ b/drivers/net/ethernet/intel/e1000e/netdev.c
> @@ -3288,6 +3288,8 @@ static void e1000_configure_rx(struct e1000_adapter *adapter)
>   			ew32(RXDCTL(0), rxdctl | 0x3 | BIT(8));
>   		}
>   
> +		dev_info(&adapter->pdev->dev,
> +			 "Some CPU C-states have been disabled in order to enable jumbo frames\n");
>   		pm_qos_update_request(&adapter->pm_qos_req, lat);
>   	} else {
>   		pm_qos_update_request(&adapter->pm_qos_req,
> 

[Intel-wired-lan] [PATCH 2/3] e1000e: Alert the user that C-states will be disabled by enabling jumbo frames

[Brown, Aaron F]

> From: Intel-wired-lan [mailto:intel-wired-lan-bounces at osuosl.org] On Behalf
> Of Matt Turner
> Sent: Tuesday, November 14, 2017 3:52 PM
> To: intel-wired-lan at lists.osuosl.org
> Cc: Turner, Matt <matt.turner@intel.com>
> Subject: [Intel-wired-lan] [PATCH 2/3] e1000e: Alert the user that C-states will
> be disabled by enabling jumbo frames
> 
> From: Matt Turner <matt.turner@intel.com>
> 
> I personally spent a long time trying to decypher why my CPU would not
> reach deeper C-states. Let's just tell the next user what's going on.
> 
> Signed-off-by: Matt Turner <matt.turner@intel.com>
> ---
> v2: Per Shannon, put string on one line for easier grepping
> 
>  drivers/net/ethernet/intel/e1000e/netdev.c | 2 ++
>  1 file changed, 2 insertions(+)

Tested-by: Aaron Brown <aaron.f.brown@intel.com>

[Intel-wired-lan] [PATCH 3/3] e1000e: Disable jumbo receive workaround on Lynx Point and newer

[Matt Turner]

From: Matt Turner <matt.turner@intel.com>

Commit 3e35d9918cbb ("e1000e: adjust PM QoS request") expanded the
application of what is evidently a hardware workaround to apply to all
e1000e devices. Prior to the commit, it applied only to e1000_pch2lan
(Sandy Bridge era) and the commit message notes that other earlier parts
such as ICH9 and ICH10 suffer from the problem as well.

The workaround works by preventing the CPU from entering deep sleep
states, which increases energy consumption significantly. My Skylake
CPU reaches the C10 state the PC8 package state with the MTU for its
I219-LM set to 1500. At an MTU of 9000, it can only reach the C1E state
and no low power package state at all. With this patch, the CPU again
reaches the C10 state and (only) the PC2 package state. Not ideal, but an
improvement nonetheless.

Signed-off-by: Matt Turner <matt.turner@intel.com>
---
This patch is speculative -- I'm an Intel employee (not in the networking
division), but I have no idea where to find documentation about the hardware
bug in question. I'm hoping that someone more in the know can check if the
hardware was fixed in subsequent generations and this workaround can be
disabled. I have chosen the pch2 cut off only because nothing further is
mentioned about the bug in git history.

 drivers/net/ethernet/intel/e1000e/netdev.c | 17 +++++++++--------
 1 file changed, 9 insertions(+), 8 deletions(-)

diff --git a/drivers/net/ethernet/intel/e1000e/netdev.c b/drivers/net/ethernet/intel/e1000e/netdev.c
index 4dcff481c4b4..553f8bd45eea 100644
--- a/drivers/net/ethernet/intel/e1000e/netdev.c
+++ b/drivers/net/ethernet/intel/e1000e/netdev.c
@@ -3277,7 +3277,8 @@ static void e1000_configure_rx(struct e1000_adapter *adapter)
 	/* With jumbo frames, excessive C-state transition latencies result
 	 * in dropped transactions.
 	 */
-	if (adapter->netdev->mtu > ETH_DATA_LEN) {
+	if (adapter->hw.mac.type <= e1000_pch2lan &&
+	    adapter->netdev->mtu > ETH_DATA_LEN) {
 		u32 lat =
 		    ((er32(PBA) & E1000_PBA_RXA_MASK) * 1024 -
 		     adapter->max_frame_size) * 8 / 1000;
@@ -3292,9 +3293,6 @@ static void e1000_configure_rx(struct e1000_adapter *adapter)
 			 "Some CPU C-states have been disabled in order to "
 			 "enable jumbo frames\n");
 		pm_qos_update_request(&adapter->pm_qos_req, lat);
-	} else {
-		pm_qos_update_request(&adapter->pm_qos_req,
-				      PM_QOS_DEFAULT_VALUE);
 	}
 
 	/* Enable Receives */
@@ -4625,8 +4623,9 @@ int e1000e_open(struct net_device *netdev)
 		e1000_update_mng_vlan(adapter);
 
 	/* DMA latency requirement to workaround jumbo issue */
-	pm_qos_add_request(&adapter->pm_qos_req, PM_QOS_CPU_DMA_LATENCY,
-			   PM_QOS_DEFAULT_VALUE);
+	if (adapter->hw.mac.type <= e1000_pch2lan)
+		pm_qos_add_request(&adapter->pm_qos_req, PM_QOS_CPU_DMA_LATENCY,
+				   PM_QOS_DEFAULT_VALUE);
 
 	/* before we allocate an interrupt, we must be ready to handle it.
 	 * Setting DEBUG_SHIRQ in the kernel makes it fire an interrupt
@@ -4669,7 +4668,8 @@ int e1000e_open(struct net_device *netdev)
 	return 0;
 
 err_req_irq:
-	pm_qos_remove_request(&adapter->pm_qos_req);
+	if (adapter->hw.mac.type <= e1000_pch2lan)
+		pm_qos_remove_request(&adapter->pm_qos_req);
 	e1000e_release_hw_control(adapter);
 	e1000_power_down_phy(adapter);
 	e1000e_free_rx_resources(adapter->rx_ring);
@@ -4733,7 +4733,8 @@ int e1000e_close(struct net_device *netdev)
 	    !test_bit(__E1000_TESTING, &adapter->state))
 		e1000e_release_hw_control(adapter);
 
-	pm_qos_remove_request(&adapter->pm_qos_req);
+	if (adapter->hw.mac.type <= e1000_pch2lan)
+		pm_qos_remove_request(&adapter->pm_qos_req);
 
 	pm_runtime_put_sync(&pdev->dev);
 
-- 
2.13.6

[Intel-wired-lan] [PATCH 3/3] e1000e: Disable jumbo receive workaround on Lynx Point and newer

[Neftin, Sasha]

On 11/8/2017 00:13, Matt Turner wrote:
> From: Matt Turner <matt.turner@intel.com>
>
> Commit 3e35d9918cbb ("e1000e: adjust PM QoS request") expanded the
> application of what is evidently a hardware workaround to apply to all
> e1000e devices. Prior to the commit, it applied only to e1000_pch2lan
> (Sandy Bridge era) and the commit message notes that other earlier parts
> such as ICH9 and ICH10 suffer from the problem as well.
>
> The workaround works by preventing the CPU from entering deep sleep
> states, which increases energy consumption significantly. My Skylake
> CPU reaches the C10 state the PC8 package state with the MTU for its
> I219-LM set to 1500. At an MTU of 9000, it can only reach the C1E state
> and no low power package state at all. With this patch, the CPU again
> reaches the C10 state and (only) the PC2 package state. Not ideal, but an
> improvement nonetheless.
>
> Signed-off-by: Matt Turner <matt.turner@intel.com>
> ---
> This patch is speculative -- I'm an Intel employee (not in the networking
> division), but I have no idea where to find documentation about the hardware
> bug in question. I'm hoping that someone more in the know can check if the
> hardware was fixed in subsequent generations and this workaround can be
> disabled. I have chosen the pch2 cut off only because nothing further is
> mentioned about the bug in git history.
>
>   drivers/net/ethernet/intel/e1000e/netdev.c | 17 +++++++++--------
>   1 file changed, 9 insertions(+), 8 deletions(-)
>
> diff --git a/drivers/net/ethernet/intel/e1000e/netdev.c b/drivers/net/ethernet/intel/e1000e/netdev.c
> index 4dcff481c4b4..553f8bd45eea 100644
> --- a/drivers/net/ethernet/intel/e1000e/netdev.c
> +++ b/drivers/net/ethernet/intel/e1000e/netdev.c
> @@ -3277,7 +3277,8 @@ static void e1000_configure_rx(struct e1000_adapter *adapter)
>   	/* With jumbo frames, excessive C-state transition latencies result
>   	 * in dropped transactions.
>   	 */
> -	if (adapter->netdev->mtu > ETH_DATA_LEN) {
> +	if (adapter->hw.mac.type <= e1000_pch2lan &&
> +	    adapter->netdev->mtu > ETH_DATA_LEN) {
>   		u32 lat =
>   		    ((er32(PBA) & E1000_PBA_RXA_MASK) * 1024 -
>   		     adapter->max_frame_size) * 8 / 1000;
> @@ -3292,9 +3293,6 @@ static void e1000_configure_rx(struct e1000_adapter *adapter)
>   			 "Some CPU C-states have been disabled in order to "
>   			 "enable jumbo frames\n");
>   		pm_qos_update_request(&adapter->pm_qos_req, lat);
> -	} else {
> -		pm_qos_update_request(&adapter->pm_qos_req,
> -				      PM_QOS_DEFAULT_VALUE);
>   	}
>   
>   	/* Enable Receives */
> @@ -4625,8 +4623,9 @@ int e1000e_open(struct net_device *netdev)
>   		e1000_update_mng_vlan(adapter);
>   
>   	/* DMA latency requirement to workaround jumbo issue */
> -	pm_qos_add_request(&adapter->pm_qos_req, PM_QOS_CPU_DMA_LATENCY,
> -			   PM_QOS_DEFAULT_VALUE);
> +	if (adapter->hw.mac.type <= e1000_pch2lan)
> +		pm_qos_add_request(&adapter->pm_qos_req, PM_QOS_CPU_DMA_LATENCY,
> +				   PM_QOS_DEFAULT_VALUE);
>   
>   	/* before we allocate an interrupt, we must be ready to handle it.
>   	 * Setting DEBUG_SHIRQ in the kernel makes it fire an interrupt
> @@ -4669,7 +4668,8 @@ int e1000e_open(struct net_device *netdev)
>   	return 0;
>   
>   err_req_irq:
> -	pm_qos_remove_request(&adapter->pm_qos_req);
> +	if (adapter->hw.mac.type <= e1000_pch2lan)
> +		pm_qos_remove_request(&adapter->pm_qos_req);
>   	e1000e_release_hw_control(adapter);
>   	e1000_power_down_phy(adapter);
>   	e1000e_free_rx_resources(adapter->rx_ring);
> @@ -4733,7 +4733,8 @@ int e1000e_close(struct net_device *netdev)
>   	    !test_bit(__E1000_TESTING, &adapter->state))
>   		e1000e_release_hw_control(adapter);
>   
> -	pm_qos_remove_request(&adapter->pm_qos_req);
> +	if (adapter->hw.mac.type <= e1000_pch2lan)
> +		pm_qos_remove_request(&adapter->pm_qos_req);
>   
>   	pm_runtime_put_sync(&pdev->dev);
>   

This patch should be rejected. The original patch 3e35d9918cbb ("e1000e: 
adjust PM QoS request") is from 2013, before Lynx Point and later 
devices were introduced. However, nothing was changed in this regard in 
the later devices and same limitation still apply. Enabling deep CPU Cx 
states with jumbo frame can lead to packet loss, which is worse than 
extra power consumption. Therefore we suggest to reject this patch. Note 
that on modern systems, the effect of jumbo frames on performance is 
negligible, as near-wire speed can be reached with standard MTU size.

[Intel-wired-lan] [PATCH 3/3] e1000e: Disable jumbo receive workaround on Lynx Point and newer

[Matt Turner]

On Thu, Nov 9, 2017 at 12:19 AM, Neftin, Sasha <sasha.neftin@intel.com> wrote:
> This patch should be rejected. The original patch 3e35d9918cbb ("e1000e:
> adjust PM QoS request") is from 2013, before Lynx Point and later devices
> were introduced. However, nothing was changed in this regard in the later
> devices and same limitation still apply.

Dang, that's unfortunate. Thanks for checking.

[Intel-wired-lan] [PATCH 1/3] e1000e: Set HTHRESH when PTHRESH is used

[Neftin, Sasha]

On 11/8/2017 00:13, Matt Turner wrote:
> From: Matt Turner <matt.turner@intel.com>
>
> According to section 12.0.3.4.13 "Receive Descriptor Control - RXDCTL"
> of the Intel? 82579 Gigabit Ethernet PHY Datasheet v2.1:
>
>      "HTHRESH should be given a non zero value when ever PTHRESH is
>       used."
>
> In RXDCTL(0), PTHRESH lives at bits 5:0, and HTHREST lives at bits 13:8.
> Set only bit 8 of HTHREST as is done in e1000_flush_rx_ring(). Found by
> inspection.
>
> Signed-off-by: Matt Turner <matt.turner@intel.com>
> ---
>   drivers/net/ethernet/intel/e1000e/netdev.c | 2 +-
>   1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/drivers/net/ethernet/intel/e1000e/netdev.c b/drivers/net/ethernet/intel/e1000e/netdev.c
> index 2dcb5463d9b8..286d0277284e 100644
> --- a/drivers/net/ethernet/intel/e1000e/netdev.c
> +++ b/drivers/net/ethernet/intel/e1000e/netdev.c
> @@ -3285,7 +3285,7 @@ static void e1000_configure_rx(struct e1000_adapter *adapter)
>   		if (adapter->flags & FLAG_IS_ICH) {
>   			u32 rxdctl = er32(RXDCTL(0));
>   
> -			ew32(RXDCTL(0), rxdctl | 0x3);
> +			ew32(RXDCTL(0), rxdctl | 0x3 | BIT(8));
>   		}
>   
>   		pm_qos_update_request(&adapter->pm_qos_req, lat);

Ack. Good point.

[Intel-wired-lan] [PATCH 1/3] e1000e: Set HTHRESH when PTHRESH is used

[Brown, Aaron F]

> From: Intel-wired-lan [mailto:intel-wired-lan-bounces at osuosl.org] On Behalf
> Of Matt Turner
> Sent: Tuesday, November 7, 2017 2:14 PM
> To: intel-wired-lan at lists.osuosl.org
> Cc: Turner, Matt <matt.turner@intel.com>
> Subject: [Intel-wired-lan] [PATCH 1/3] e1000e: Set HTHRESH when PTHRESH
> is used
> 
> From: Matt Turner <matt.turner@intel.com>
> 
> According to section 12.0.3.4.13 "Receive Descriptor Control - RXDCTL"
> of the Intel? 82579 Gigabit Ethernet PHY Datasheet v2.1:
> 
>     "HTHRESH should be given a non zero value when ever PTHRESH is
>      used."
> 
> In RXDCTL(0), PTHRESH lives at bits 5:0, and HTHREST lives at bits 13:8.
> Set only bit 8 of HTHREST as is done in e1000_flush_rx_ring(). Found by
> inspection.
> 
> Signed-off-by: Matt Turner <matt.turner@intel.com>
> ---
>  drivers/net/ethernet/intel/e1000e/netdev.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)

Tested-by: Aaron Brown <aaron.f.brown@intel.com>