From: Duncan Roe <duncan_roe@optusnet.com.au>
To: Netfilter Development <netfilter-devel@vger.kernel.org>,
Pablo Neira Ayuso <pablo@netfilter.org>
Cc: Pablo Neira Ayuso <pablo@netfilter.org>
Subject: Re: [PATCH libnetfilter_queue 0/3] pktbuff API updates
Date: Sat, 2 May 2020 22:50:06 +1000 [thread overview]
Message-ID: <20200502125006.GH3833@dimstar.local.net> (raw)
In-Reply-To: <20200430063404.GF3833@dimstar.local.net>
On Thu, Apr 30, 2020 at 04:34:04PM +1000, Duncan Roe wrote:
> On Wed, Apr 29, 2020 at 11:05:12PM +0200, Pablo Neira Ayuso wrote:
> > On Thu, Apr 30, 2020 at 06:30:29AM +1000, Duncan Roe wrote:
> > > Hi Pablo,
> > >
> > > I sent this email (explanation of how the system works) before I saw your email
> > > asking for that explanation.
> > >
> > > Then I replied to that email of yours before I saw this one.
> > >
> > > On Wed, Apr 29, 2020 at 09:16:43PM +0200, Pablo Neira Ayuso wrote:
> > > > On Thu, Apr 30, 2020 at 05:10:47AM +1000, Duncan Roe wrote:
> > > > [...]
> > > > > Sorry, I should have explained a bit more how the system would work:
> > > > >
> > > > > struct pkt_buff has 3 new members:
> > > > >
> > > > > bool copy_done;
> > > > > uint32_t extra;
> > > > > uint8_t *copy_buf;
> > > > >
> > > > > When extra > 0, pktb_alloc2 verifies that buflen is >= len + extra. It then
> > > > > stores extra and copy_buf in pktb, ready for use by pktb_mangle() (all the other
> > > > > manglers call this eventually).
> > > > >
> > > > > So that's how pktb_mangle() doesn't need to allocate a buffer.
> > > >
> > > > Thanks for the explaining. Given this is in userspace, it is easier if
> > >
> > > Tiny nit - this could be userspace on an embedded device where memory is really
> > > tight. So perhaps document with "If memory is at a premium, you really only need
> > > len + extra" otherwise a big buf is fine.
> >
> > This buffer is still relatively small, the reallocation also forces
> > the user to refetch pointers. Skipping all that complexity for a bit
> > of memory in userspace is fine.
>
> Oh well in that case, how about:
>
> > struct pkt_buff *pktb_alloc2(int family, void *buf, size_t buf_size, void *data, size_t len, size_t extra);
>
> I.e. exactly as you suggested in
> https://www.spinics.net/lists/netfilter-devel/msg65830.html except s/head/buf/
>
> And we tell users to dimension buf to NFQ_BUFFER_SIZE. We don't even need to
> expose pktb_head_size().
> >
> > > > the user allocates the maximum packet length that is possible:
> > > >
> > > > 0xffff + (MNL_SOCKET_BUFFER_SIZE/2);
> > > >
> > > > We can probably expose this to the header so they can pre-allocate a
> > > > buffer that is large enough and, hence, _mangle() is guaranteed to
> > > > have always enough room to add extra bytes.
> > >
> > > Yes I saw that expression in examples/nf-queue.c. How about
> > >
> > > #define COPY_BUF_SIZE (0xffff + (MNL_SOCKET_BUFFER_SIZE/2))
> > >
> > > or what other name would you like?
> >
> > I'd suggest to add the NFQ_ prefix, probably NFQ_BUFFER_SIZE is fine.
> >
> > > --- Off-topic
> > >
> > > I'm intrigued that you ccan use MNL_SOCKET_BUFFER_SIZE when dimensioning static
> > > variables. The expansion is
> > >
> > > #define MNL_SOCKET_BUFFER_SIZE (sysconf(_SC_PAGESIZE) < 8192L ?
> > > sysconf(_SC_PAGESIZE) : 8192L)
> > >
> > > and sysconf is an actual function:
> > >
> > > unistd.h:622:extern long int sysconf (int __name) __THROW;
> > >
> > > If I try to dimension a static variable using pktb_head_size(), the compiler
> > > throws an error. Why is sysconf() ok but not pktb_head_size()?
> >
> > I'm fine to expose if you would like to add pktb_head_size() if you
> > prefer to store the pkt_buff in the stack. You will have to use
> > pktb_build_data() [1] to initialize the pkt_buff. Would that work for you?
> >
> > [1] https://patchwork.ozlabs.org/project/netfilter-devel/patch/20200426132356.8346-2-pablo@netfilter.org/
To be clear, the email reference was to
https://www.spinics.net/lists/netfilter-devel/msg65830.html
I should have put a comma between that and "except s/head/buf/"
Sorry about that,
Cheers ... Duncan.
next prev parent reply other threads:[~2020-05-02 12:50 UTC|newest]
Thread overview: 28+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-04-26 13:23 [PATCH libnetfilter_queue 0/3] pktbuff API updates Pablo Neira Ayuso
2020-04-26 13:23 ` [PATCH libnetfilter_queue 1/3] pktbuff: add pktb_alloc_head() and pktb_build_data() Pablo Neira Ayuso
2020-04-30 5:41 ` Duncan Roe
2020-04-26 13:23 ` [PATCH libnetfilter_queue 2/3] example: nf-queue: use pkt_buff Pablo Neira Ayuso
2020-05-14 4:35 ` Duncan Roe
2020-05-14 4:35 ` [PATCH libnetfilter_queue 1/1] example: nf-queue: use pkt_buff (updated) Duncan Roe
2020-04-26 13:23 ` [PATCH libnetfilter_queue 3/3] pktbuff: add pktb_reset_network_header() and pktb_set_network_header() Pablo Neira Ayuso
2020-04-27 11:06 ` [PATCH libnetfilter_queue 0/3] pktbuff API updates Duncan Roe
2020-04-27 17:06 ` Pablo Neira Ayuso
2020-04-28 4:33 ` Duncan Roe
2020-04-28 10:34 ` Pablo Neira Ayuso
2020-04-28 21:14 ` Duncan Roe
2020-04-28 22:55 ` Pablo Neira Ayuso
2020-04-29 13:28 ` Duncan Roe
2020-04-29 19:00 ` Pablo Neira Ayuso
2020-04-29 19:54 ` Duncan Roe
2020-04-29 21:12 ` Pablo Neira Ayuso
2020-04-29 19:10 ` Duncan Roe
2020-04-29 19:16 ` Pablo Neira Ayuso
2020-04-29 20:30 ` Duncan Roe
2020-04-29 21:05 ` Pablo Neira Ayuso
2020-04-30 6:34 ` Duncan Roe
2020-05-02 12:50 ` Duncan Roe [this message]
2020-05-05 12:30 ` Pablo Neira Ayuso
2020-05-06 0:57 ` Duncan Roe
2020-05-06 2:39 ` Duncan Roe
2020-05-08 1:13 ` Duncan Roe
2020-05-09 8:26 ` Duncan Roe
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20200502125006.GH3833@dimstar.local.net \
--to=duncan_roe@optusnet.com.au \
--cc=netfilter-devel@vger.kernel.org \
--cc=pablo@netfilter.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).