From: Nitesh Narayan Lal <email@example.com> To: firstname.lastname@example.org, email@example.com, firstname.lastname@example.org, email@example.com, firstname.lastname@example.org, email@example.com, firstname.lastname@example.org Subject: [PATCH v1 0/1] limit the i40e msix vectors based on housekeeping CPUs Date: Mon, 15 Jun 2020 16:21:24 -0400 [thread overview] Message-ID: <email@example.com> (raw) Issue ===== With the current implementation at the time of i40e_init_msix(), i40e creates vectors only based on the number of online CPUs. This would be problematic for RT setup that includes a large number of isolated but very few housekeeping CPUs. This is because in those setups an attempt to move all IRQs from isolated to housekeeping CPUs may easily fail due to per CPU vector limit. Setup For The Issue =================== I have triggered this issue on a setup that had a total of 72 cores among which 68 were isolated and only 4 were left for housekeeping tasks. I was using tuned's realtime-virtual-host profile to configure the system. However, Tuned reported the error message 'Failed to set SMP affinity of IRQ xxx to '00000040,00000010,00000005': [Errno 28] No space left on the device' for several IRQs in tuned.log. Note: There were other IRQs as well pinned to the housekeeping CPUs that were generated by other drivers. Fix === - In this proposed fix I have replaced num_online_cpus in i40e_init_msix() with the number of housekeeping CPUs. - The reason why I chose to include both HK_FLAG_DOMAIN & HK_FLAG_WQ is because we would also need IRQ isolation with something like systemd's CPU affinity. Testing ======= To test this change I had added a tracepoint in i40e_init_msix() to find the number of CPUs derived for vector creation with and without tuned's realtime-virtual-host profile. As per expectation with the profile applied I was only getting the number of housekeeping CPUs and all available CPUs without it. Nitesh Narayan Lal (1): i40e: limit the msix vectors based on housekeeping CPUs drivers/net/ethernet/intel/i40e/i40e_main.c | 16 +++++++++++----- 1 file changed, 11 insertions(+), 5 deletions(-) --
next reply other threads:[~2020-06-15 20:21 UTC|newest] Thread overview: 7+ messages / expand[flat|nested] mbox.gz Atom feed top 2020-06-15 20:21 Nitesh Narayan Lal [this message] 2020-06-15 20:21 ` [Patch v1] i40e: limit the " Nitesh Narayan Lal 2020-06-15 20:48 ` Keller, Jacob E 2020-06-15 20:55 ` Nitesh Narayan Lal 2020-06-16 8:03 ` Christoph Hellwig 2020-06-16 17:29 ` Nitesh Narayan Lal 2020-06-26 20:11 ` Nitesh Narayan Lal
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ --subject='Re: [PATCH v1 0/1] limit the i40e msix vectors based on housekeeping CPUs' \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).