All of lore.kernel.org
 help / color / mirror / Atom feed
From: Caleb Sander <csander@purestorage.com>
To: intel-wired-lan@lists.osuosl.org
Cc: Tony Nguyen <anthony.l.nguyen@intel.com>,
	Caleb Sander <csander@purestorage.com>,
	Joern Engel <joern@purestorage.com>,
	Tony Brelinski <tony.brelinski@intel.com>,
	davem@davemloft.net, Jakub Kicinski <kuba@kernel.org>,
	netdev@vger.kernel.org, sassmann@redhat.com
Subject: [PATCH v2] i40e: avoid spin loop in i40e_asq_send_command()
Date: Mon,  1 Nov 2021 11:38:08 -0600	[thread overview]
Message-ID: <20211101173808.1735144-1-csander@purestorage.com> (raw)
In-Reply-To: <YXq3M5XvOkpMgiOg@cork>

Previously, the kernel could spend up to 250 ms waiting for a command to
be submitted to an admin queue. This function is also called in a loop,
e.g., in i40e_get_module_eeprom() (through i40e_aq_get_phy_register()),
so the time spent in the kernel may be even higher. We observed
scheduling delays of over 2 seconds in production,
with stacktraces pointing to this code as the culprit.

Use usleep_range() instead of udelay() so the loop can yield the CPU.
Also compute the elapsed time using the jiffies counter rather than
assuming udelay() waits exactly the time interval requested.

Signed-off-by: Caleb Sander <csander@purestorage.com>
Reviewed-by: Joern Engel <joern@purestorage.com>
---
 drivers/net/ethernet/intel/i40e/i40e_adminq.c | 7 +++----
 1 file changed, 3 insertions(+), 4 deletions(-)

Changed from v1:
Use usleep_range() instead of udelay() + cond_resched(),
to avoid using the CPU while waiting.
Use 50 us as the max for the range since hrtimers schedules the sleep
for the max (unless another timer interrupt occurs after the min).
Since checking if the command is done too frequently would waste time
context-switching, use half of the max (25 us) as the min for the range.

diff --git a/drivers/net/ethernet/intel/i40e/i40e_adminq.c b/drivers/net/ethernet/intel/i40e/i40e_adminq.c
index 593912b17..b2c27ab3b 100644
--- a/drivers/net/ethernet/intel/i40e/i40e_adminq.c
+++ b/drivers/net/ethernet/intel/i40e/i40e_adminq.c
@@ -902,7 +902,7 @@ i40e_status i40e_asq_send_command(struct i40e_hw *hw,
 	 * we need to wait for desc write back
 	 */
 	if (!details->async && !details->postpone) {
-		u32 total_delay = 0;
+		unsigned long timeout_end = jiffies + usecs_to_jiffies(hw->aq.asq_cmd_timeout);
 
 		do {
 			/* AQ designers suggest use of head for better
@@ -910,9 +910,8 @@ i40e_status i40e_asq_send_command(struct i40e_hw *hw,
 			 */
 			if (i40e_asq_done(hw))
 				break;
-			udelay(50);
-			total_delay += 50;
-		} while (total_delay < hw->aq.asq_cmd_timeout);
+			usleep_range(25, 50);
+		} while (time_before(jiffies, timeout_end));
 	}
 
 	/* if ready, copy the desc back to temp */
-- 
2.25.1


WARNING: multiple messages have this Message-ID (diff)
From: Caleb Sander <csander@purestorage.com>
To: intel-wired-lan@osuosl.org
Subject: [Intel-wired-lan] [PATCH v2] i40e: avoid spin loop in i40e_asq_send_command()
Date: Mon,  1 Nov 2021 11:38:08 -0600	[thread overview]
Message-ID: <20211101173808.1735144-1-csander@purestorage.com> (raw)
In-Reply-To: <YXq3M5XvOkpMgiOg@cork>

Previously, the kernel could spend up to 250 ms waiting for a command to
be submitted to an admin queue. This function is also called in a loop,
e.g., in i40e_get_module_eeprom() (through i40e_aq_get_phy_register()),
so the time spent in the kernel may be even higher. We observed
scheduling delays of over 2 seconds in production,
with stacktraces pointing to this code as the culprit.

Use usleep_range() instead of udelay() so the loop can yield the CPU.
Also compute the elapsed time using the jiffies counter rather than
assuming udelay() waits exactly the time interval requested.

Signed-off-by: Caleb Sander <csander@purestorage.com>
Reviewed-by: Joern Engel <joern@purestorage.com>
---
 drivers/net/ethernet/intel/i40e/i40e_adminq.c | 7 +++----
 1 file changed, 3 insertions(+), 4 deletions(-)

Changed from v1:
Use usleep_range() instead of udelay() + cond_resched(),
to avoid using the CPU while waiting.
Use 50 us as the max for the range since hrtimers schedules the sleep
for the max (unless another timer interrupt occurs after the min).
Since checking if the command is done too frequently would waste time
context-switching, use half of the max (25 us) as the min for the range.

diff --git a/drivers/net/ethernet/intel/i40e/i40e_adminq.c b/drivers/net/ethernet/intel/i40e/i40e_adminq.c
index 593912b17..b2c27ab3b 100644
--- a/drivers/net/ethernet/intel/i40e/i40e_adminq.c
+++ b/drivers/net/ethernet/intel/i40e/i40e_adminq.c
@@ -902,7 +902,7 @@ i40e_status i40e_asq_send_command(struct i40e_hw *hw,
 	 * we need to wait for desc write back
 	 */
 	if (!details->async && !details->postpone) {
-		u32 total_delay = 0;
+		unsigned long timeout_end = jiffies + usecs_to_jiffies(hw->aq.asq_cmd_timeout);
 
 		do {
 			/* AQ designers suggest use of head for better
@@ -910,9 +910,8 @@ i40e_status i40e_asq_send_command(struct i40e_hw *hw,
 			 */
 			if (i40e_asq_done(hw))
 				break;
-			udelay(50);
-			total_delay += 50;
-		} while (total_delay < hw->aq.asq_cmd_timeout);
+			usleep_range(25, 50);
+		} while (time_before(jiffies, timeout_end));
 	}
 
 	/* if ready, copy the desc back to temp */
-- 
2.25.1


  reply	other threads:[~2021-11-01 17:39 UTC|newest]

Thread overview: 16+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-10-25 17:55 [PATCH net-next 0/4][pull request] 40GbE Intel Wired LAN Driver Updates 2021-10-25 Tony Nguyen
2021-10-25 17:55 ` [PATCH net-next 1/4] i40e: avoid spin loop in i40e_asq_send_command() Tony Nguyen
2021-10-27 16:01   ` Jakub Kicinski
2021-10-28 11:49     ` Jörn Engel
2021-10-28 14:26       ` Jakub Kicinski
2021-10-28 14:44         ` Jörn Engel
2021-11-01 17:38           ` Caleb Sander [this message]
2021-11-01 17:38             ` [Intel-wired-lan] [PATCH v2] " Caleb Sander
2021-10-25 17:55 ` [PATCH net-next 2/4] intel: Simplify bool conversion Tony Nguyen
2021-10-25 17:55 ` [PATCH net-next 3/4] igb: unbreak I2C bit-banging on i350 Tony Nguyen
2021-10-27 16:30   ` Jakub Kicinski
2021-10-28 15:25     ` Nguyen, Anthony L
2021-10-28 15:32       ` Jakub Kicinski
2021-10-25 17:55 ` [PATCH net-next 4/4] net: ixgbevf: Remove redundant initialization of variable ret_val Tony Nguyen
2021-10-27 15:51 ` [PATCH net-next 0/4][pull request] 40GbE Intel Wired LAN Driver Updates 2021-10-25 Nguyen, Anthony L
2021-10-27 15:59   ` Jakub Kicinski

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20211101173808.1735144-1-csander@purestorage.com \
    --to=csander@purestorage.com \
    --cc=anthony.l.nguyen@intel.com \
    --cc=davem@davemloft.net \
    --cc=intel-wired-lan@lists.osuosl.org \
    --cc=joern@purestorage.com \
    --cc=kuba@kernel.org \
    --cc=netdev@vger.kernel.org \
    --cc=sassmann@redhat.com \
    --cc=tony.brelinski@intel.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.