* Why is NIC driver queue depth driver dependent when it allocates system memory?
@ 2020-01-30 13:02 Martin T
2020-01-30 16:56 ` Jakub Kicinski
2020-02-01 19:08 ` Cong Wang
0 siblings, 2 replies; 6+ messages in thread
From: Martin T @ 2020-01-30 13:02 UTC (permalink / raw)
To: netdev
Hi,
when I read the source code of for example tg3 driver or e1000e
driver, then looks like the driver queue is allocated from system
memory. For example, in e1000_ethtool.c kcalloc() is called to
allocate GFP_KERNEL memory.
If system memory is allocated, then why are there driver-dependent
limits? For example, in my workstation the maximum RX/TX queue for the
NIC using tg3 driver is 511 while maximum RX/TX queue for the NIC
using e1000e driver is 4096:
# Broadcom Limited NetXtreme BCM5722 Gigabit Ethernet; tg3 driver
# ethtool -g eth1
Ring parameters for eth1:
Pre-set maximums:
RX: 511
RX Mini: 0
RX Jumbo: 0
TX: 511
Current hardware settings:
RX: 200
RX Mini: 0
RX Jumbo: 0
TX: 511
# Intel Corporation Ethernet Connection (2) I219-LM; e1000e driver
# ethtool -g eth0
Ring parameters for eth0:
Pre-set maximums:
RX: 4096
RX Mini: 0
RX Jumbo: 0
TX: 4096
Current hardware settings:
RX: 256
RX Mini: 0
RX Jumbo: 0
TX: 256
#
thanks,
Martin
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Why is NIC driver queue depth driver dependent when it allocates system memory?
2020-01-30 13:02 Why is NIC driver queue depth driver dependent when it allocates system memory? Martin T
@ 2020-01-30 16:56 ` Jakub Kicinski
2020-01-31 14:45 ` Martin T
2020-02-01 19:08 ` Cong Wang
1 sibling, 1 reply; 6+ messages in thread
From: Jakub Kicinski @ 2020-01-30 16:56 UTC (permalink / raw)
To: Martin T; +Cc: netdev
On Thu, 30 Jan 2020 15:02:02 +0200, Martin T wrote:
> Hi,
>
> when I read the source code of for example tg3 driver or e1000e
> driver, then looks like the driver queue is allocated from system
> memory. For example, in e1000_ethtool.c kcalloc() is called to
> allocate GFP_KERNEL memory.
>
> If system memory is allocated, then why are there driver-dependent
> limits? For example, in my workstation the maximum RX/TX queue for the
> NIC using tg3 driver is 511 while maximum RX/TX queue for the NIC
> using e1000e driver is 4096:
Hi Martin,
the idea is that drivers can choose the initial setting which are
optimal for the implementation and the hardware. Whether they actually
are, or whether values chosen 10 years ago for tg3 and e1000e are
reasonable for modern uses of that hardware could perhaps be questioned.
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Why is NIC driver queue depth driver dependent when it allocates system memory?
2020-01-30 16:56 ` Jakub Kicinski
@ 2020-01-31 14:45 ` Martin T
0 siblings, 0 replies; 6+ messages in thread
From: Martin T @ 2020-01-31 14:45 UTC (permalink / raw)
To: Jakub Kicinski; +Cc: netdev
Hi Jakub,
> the idea is that drivers can choose the initial setting which are
> optimal for the implementation and the hardware. Whether they actually
> are, or whether values chosen 10 years ago for tg3 and e1000e are
> reasonable for modern uses of that hardware could perhaps be questioned.
Ok, thanks!
Martin
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Why is NIC driver queue depth driver dependent when it allocates system memory?
2020-01-30 13:02 Why is NIC driver queue depth driver dependent when it allocates system memory? Martin T
2020-01-30 16:56 ` Jakub Kicinski
@ 2020-02-01 19:08 ` Cong Wang
2020-02-01 19:14 ` Michal Kubecek
1 sibling, 1 reply; 6+ messages in thread
From: Cong Wang @ 2020-02-01 19:08 UTC (permalink / raw)
To: Martin T; +Cc: Linux Kernel Network Developers
On Thu, Jan 30, 2020 at 5:03 AM Martin T <m4rtntns@gmail.com> wrote:
>
> Hi,
>
> when I read the source code of for example tg3 driver or e1000e
> driver, then looks like the driver queue is allocated from system
> memory. For example, in e1000_ethtool.c kcalloc() is called to
> allocate GFP_KERNEL memory.
>
> If system memory is allocated, then why are there driver-dependent
> limits? For example, in my workstation the maximum RX/TX queue for the
> NIC using tg3 driver is 511 while maximum RX/TX queue for the NIC
> using e1000e driver is 4096:
I doubt memory is a consideration for driver to decide the number
of queues. How many CPU's do you have? At least mellanox driver
uses the number of CPU's to determine the default value. Anyway,
you can change it to whatever you prefer.
Thanks.
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Why is NIC driver queue depth driver dependent when it allocates system memory?
2020-02-01 19:08 ` Cong Wang
@ 2020-02-01 19:14 ` Michal Kubecek
2020-02-01 19:51 ` Cong Wang
0 siblings, 1 reply; 6+ messages in thread
From: Michal Kubecek @ 2020-02-01 19:14 UTC (permalink / raw)
To: netdev; +Cc: Cong Wang, Martin T
On Sat, Feb 01, 2020 at 11:08:37AM -0800, Cong Wang wrote:
> On Thu, Jan 30, 2020 at 5:03 AM Martin T <m4rtntns@gmail.com> wrote:
> >
> > Hi,
> >
> > when I read the source code of for example tg3 driver or e1000e
> > driver, then looks like the driver queue is allocated from system
> > memory. For example, in e1000_ethtool.c kcalloc() is called to
> > allocate GFP_KERNEL memory.
> >
> > If system memory is allocated, then why are there driver-dependent
> > limits? For example, in my workstation the maximum RX/TX queue for the
> > NIC using tg3 driver is 511 while maximum RX/TX queue for the NIC
> > using e1000e driver is 4096:
>
> I doubt memory is a consideration for driver to decide the number
> of queues. How many CPU's do you have? At least mellanox driver
> uses the number of CPU's to determine the default value. Anyway,
> you can change it to whatever you prefer.
Martin was asking about ring sizes, not about number of queues.
Michal
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Why is NIC driver queue depth driver dependent when it allocates system memory?
2020-02-01 19:14 ` Michal Kubecek
@ 2020-02-01 19:51 ` Cong Wang
0 siblings, 0 replies; 6+ messages in thread
From: Cong Wang @ 2020-02-01 19:51 UTC (permalink / raw)
To: Michal Kubecek; +Cc: Linux Kernel Network Developers, Martin T
On Sat, Feb 1, 2020 at 11:14 AM Michal Kubecek <mkubecek@suse.cz> wrote:
>
> On Sat, Feb 01, 2020 at 11:08:37AM -0800, Cong Wang wrote:
> > On Thu, Jan 30, 2020 at 5:03 AM Martin T <m4rtntns@gmail.com> wrote:
> > >
> > > Hi,
> > >
> > > when I read the source code of for example tg3 driver or e1000e
> > > driver, then looks like the driver queue is allocated from system
> > > memory. For example, in e1000_ethtool.c kcalloc() is called to
> > > allocate GFP_KERNEL memory.
> > >
> > > If system memory is allocated, then why are there driver-dependent
> > > limits? For example, in my workstation the maximum RX/TX queue for the
> > > NIC using tg3 driver is 511 while maximum RX/TX queue for the NIC
> > > using e1000e driver is 4096:
> >
> > I doubt memory is a consideration for driver to decide the number
> > of queues. How many CPU's do you have? At least mellanox driver
> > uses the number of CPU's to determine the default value. Anyway,
> > you can change it to whatever you prefer.
>
> Martin was asking about ring sizes, not about number of queues.
Ah, sorry for reading it too quickly.
Thanks.
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2020-02-01 19:52 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-01-30 13:02 Why is NIC driver queue depth driver dependent when it allocates system memory? Martin T
2020-01-30 16:56 ` Jakub Kicinski
2020-01-31 14:45 ` Martin T
2020-02-01 19:08 ` Cong Wang
2020-02-01 19:14 ` Michal Kubecek
2020-02-01 19:51 ` Cong Wang
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).