linux-trace-devel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Alexander Aring <aahringo@redhat.com>
To: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Cc: linux-trace-devel@vger.kernel.org, linux-trace-users@vger.kernel.org
Subject: Re: RFC: using NIC DMA ring as tracing buffer?
Date: Wed, 17 Apr 2024 19:05:25 -0400	[thread overview]
Message-ID: <CAK-6q+jAQFco6zMS8EKr3RgwmZ+bP4pg-aD_j7+Qi-oEoPXJuQ@mail.gmail.com> (raw)
In-Reply-To: <CAK-6q+h=5LcjtTxJ0i3BcaVZRGfyWpQJy6Htxjh_zFfOgUpwfA@mail.gmail.com>

Hi,

On Wed, Apr 17, 2024 at 6:51 PM Alexander Aring <aahringo@redhat.com> wrote:
>
> Hi Mathieu,
>
> On Tue, Apr 16, 2024 at 10:49 AM Mathieu Desnoyers
> <mathieu.desnoyers@efficios.com> wrote:
> >
> > On 2024-04-16 10:41, Alexander Aring wrote:
> > > Hi all,
> > >
> > > I have an idea about using the DMA rings of a networking card as a
> > > tracing buffer.
> >
> > The first thing I'd be worried about is overhead caused by
> > false-sharing if you trace a large SMP/many-core system all
> > into the same global NIC buffer. The cache lines will bounce
> > left and right which is bad performance-wise.
> >
>
> modern NICs have several tx and rx dma rings. There is an ethtool
> command to query the number:
>
> sudo ethtool -g $IFACE
>
> so it depends on the hardware if you can pin one or more rings on a
> per-cpu buffer basis.
>
> I hope that somehow helps to make this idea less worrisome.
>

What I can think about that makes it worrisome is NUMA. In case of an
E-PCI connected NIC, the NICs buffer associated with a core needs to
be in the same NUMA node. You need to set up your system right, even
with 2 nics or more, to make it work "better".

- Alex


      reply	other threads:[~2024-04-17 23:05 UTC|newest]

Thread overview: 4+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2024-04-16 14:41 RFC: using NIC DMA ring as tracing buffer? Alexander Aring
2024-04-16 14:48 ` Mathieu Desnoyers
2024-04-17 22:51   ` Alexander Aring
2024-04-17 23:05     ` Alexander Aring [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=CAK-6q+jAQFco6zMS8EKr3RgwmZ+bP4pg-aD_j7+Qi-oEoPXJuQ@mail.gmail.com \
    --to=aahringo@redhat.com \
    --cc=linux-trace-devel@vger.kernel.org \
    --cc=linux-trace-users@vger.kernel.org \
    --cc=mathieu.desnoyers@efficios.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).