From: Jesper Dangaard Brouer <brouer@redhat.com>
To: Daniel Borkmann <daniel@iogearbox.net>
Cc: Alexei Starovoitov <ast@plumgrid.com>,
netdev@vger.kernel.org, Eric Dumazet <eric.dumazet@gmail.com>,
brouer@redhat.com
Subject: Multiqueue pktgen and ingress path (Was: [PATCH v5 2/2] pktgen: introduce xmit_mode '<start_xmit|netif_receive>')
Date: Fri, 8 May 2015 17:49:27 +0200 [thread overview]
Message-ID: <20150508174927.5b1ecdd1@redhat.com> (raw)
In-Reply-To: <20150508173900.3fcf78de@redhat.com>
[-- Attachment #1: Type: text/plain, Size: 2772 bytes --]
On Fri, 8 May 2015 17:39:00 +0200
Jesper Dangaard Brouer <brouer@redhat.com> wrote:
> On Thu, 07 May 2015 19:11:58 +0200 Daniel Borkmann <daniel@iogearbox.net> wrote:
>
> > On 05/07/2015 06:28 PM, Alexei Starovoitov wrote:
> > > On 5/7/15 7:35 AM, Jesper Dangaard Brouer wrote:
> > >> From: Alexei Starovoitov <ast@plumgrid.com>
> > >>
> [...snip...]
>
> > > btw, I've started to work on a patch on top of this one that allows
> > > multiple pktgen threads to submit into the same netdev.
> > > I've used it to stress test removal of spin_lock in ingress qdisc.
> > > The idea is to add another 'name' parameter to command:
> > > 'add_device name dev'
> > > 'name' will be used to identify this pktgen thread in /proc
> > > and 'dev' used as target net_device.
> > > I think it will be useful for start_xmit testing as well.
> > > I wonder why it wasn't done earlier? The queue configuration is
> > > already supported.
> >
> > You mean other than below commit (iow independant of queue mapping)?
> >
> > commit e6fce5b916cd7f7f79b2b3e53ba74bbfc1d7cf8b
> > Author: Robert Olsson <robert.olsson@its.uu.se>
> > Date: Thu Aug 7 02:23:01 2008 -0700
> >
> > pktgen: multiqueue etc.
>
> For completeness and others reading this threads...
>
> Pktgen multiqueue is already supported via mentioned commit, which adds
> the device naming scheme: "add_device dev@number"
>
> And yes, the documentation does not seem to mention this. I've been
> using it for years now... My scripts[1] take param "-t" for "threads".
> [1] https://github.com/netoptimizer/network-testing/tree/master/pktgen
>
> I've added a more plain version of a script, based on yours, below my
> signature.
Now attached.
> The funny thing now is that scaling does not "happen" as we stall on:
> atomic_long_inc(&skb->dev->rx_dropped);
More interesting observations with the mentioned script (now attached).
On my system the scaling stopped a 24Mpps, when I increased the number
of threads the collective scaling was stuck at 24Mpps.
Then I simply removed/compiled-out the:
atomic_long_inc(&skb->dev->rx_dropped);
And after that change, the scaling is basically infinite/perfect.
Single thread performance increased from 24.7Mpps to 31.1Mpps, which
corresponds perfectly with the cost of an atomic operation on this HW
(8.25ns).
Diff to before:
* (1/24700988*10^9)-(1/31170819*10^9) = 8.40292328196 ns
When increasing the threads now, they all basically run at 31Mpps.
Tried it upto 12 threads.
I'm quite puzzled why a single atomic op could "freeze" my system from
scaling beyond 24Mpps.
--
Best regards,
Jesper Dangaard Brouer
MSc.CS, Sr. Network Kernel Developer at Red Hat
Author of http://www.iptv-analyzer.org
LinkedIn: http://www.linkedin.com/in/brouer
[-- Attachment #2: pktgen_multiqueue01_xmit_mode.sh --]
[-- Type: application/x-shellscript, Size: 1217 bytes --]
next prev parent reply other threads:[~2015-05-08 15:49 UTC|newest]
Thread overview: 38+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-05-01 5:12 [PATCH v3 net-next] pktgen: introduce 'rx' mode Alexei Starovoitov
2015-05-01 16:54 ` Eric Dumazet
2015-05-02 8:46 ` Jesper Dangaard Brouer
2015-05-02 9:44 ` Daniel Borkmann
2015-05-02 9:54 ` Jesper Dangaard Brouer
2015-05-02 10:30 ` Daniel Borkmann
2015-05-02 16:01 ` Alexei Starovoitov
2015-05-02 16:46 ` Jesper Dangaard Brouer
2015-05-02 17:07 ` Alexei Starovoitov
2015-05-05 18:15 ` Jesper Dangaard Brouer
2015-05-05 18:30 ` Alexei Starovoitov
2015-05-05 20:29 ` [PATCH 0/2] pktgen changes Jesper Dangaard Brouer
2015-05-05 20:29 ` [PATCH 1/2] pktgen: adjust flag NO_TIMESTAMP to be more pktgen compliant Jesper Dangaard Brouer
2015-05-05 20:30 ` [PATCH 2/2] pktgen: introduce xmit_mode 'rx_inject' Jesper Dangaard Brouer
2015-05-06 4:33 ` Alexei Starovoitov
2015-05-06 5:24 ` Jesper Dangaard Brouer
2015-05-06 10:17 ` Daniel Borkmann
2015-05-06 11:22 ` Jesper Dangaard Brouer
2015-05-06 5:32 ` Alexander Duyck
2015-05-06 8:44 ` Jesper Dangaard Brouer
2015-05-06 14:35 ` Alexei Starovoitov
2015-05-07 14:34 ` [PATCH v5 0/2] pktgen changes Jesper Dangaard Brouer
2015-05-07 14:34 ` [PATCH v5 1/2] pktgen: adjust flag NO_TIMESTAMP to be more pktgen compliant Jesper Dangaard Brouer
2015-05-07 14:35 ` [PATCH v5 2/2] pktgen: introduce xmit_mode '<start_xmit|netif_receive>' Jesper Dangaard Brouer
2015-05-07 16:28 ` Alexei Starovoitov
2015-05-07 17:11 ` Daniel Borkmann
2015-05-07 17:16 ` Alexei Starovoitov
2015-05-07 17:20 ` Daniel Borkmann
2015-05-08 13:40 ` Multiqueue pktgen (was: [PATCH v5 2/2] pktgen: introduce xmit_mode '<start_xmit|netif_receive>') Jesper Dangaard Brouer
2015-05-08 15:39 ` [PATCH v5 2/2] pktgen: introduce xmit_mode '<start_xmit|netif_receive>' Jesper Dangaard Brouer
2015-05-08 15:49 ` Jesper Dangaard Brouer [this message]
2015-05-08 15:56 ` Multiqueue pktgen and ingress path (Was: [PATCH v5 2/2] pktgen: introduce xmit_mode '<start_xmit|netif_receive>') Eric Dumazet
2015-05-08 16:53 ` Alexander Duyck
2015-05-08 17:00 ` Alexei Starovoitov
2015-05-08 18:21 ` Alexander Duyck
2015-05-08 15:57 ` [PATCH v5 2/2] pktgen: introduce xmit_mode '<start_xmit|netif_receive>' Eric Dumazet
2015-05-08 16:50 ` Alexei Starovoitov
2015-05-10 2:26 ` [PATCH v5 0/2] pktgen changes David Miller
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20150508174927.5b1ecdd1@redhat.com \
--to=brouer@redhat.com \
--cc=ast@plumgrid.com \
--cc=daniel@iogearbox.net \
--cc=eric.dumazet@gmail.com \
--cc=netdev@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).