* NFQUEUE/iptables and kernel warning messages for net/ipv4/tcp_output.c @ 2020-02-17 17:30 Vieri Di Paola 2020-02-18 12:39 ` Florian Westphal 0 siblings, 1 reply; 5+ messages in thread From: Vieri Di Paola @ 2020-02-17 17:30 UTC (permalink / raw) To: netfilter Hi, Whenever I use NFQUEUE/iptables to send traffic to an IDS/IPS (eg. Suricata), I get an ugly kernel warning which can sometimes and on the long run turn into a system freeze. I'm using NFQUEUE 0:5, and I'm running Suricata with -q 0 -q 1 -q 2 -q 3 -q 4 -q 5 as arguments. I've already reported the issue on the LKML here: https://lkml.org/lkml/2020/2/13/1255 However, I've been told by the Suricata ML to try and post here too. The message "WARNING: CPU: * at net/ipv4/tcp_output.c:915" does not appear when I stop using Suricata with NFQUEUE. Regards, Vieri ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: NFQUEUE/iptables and kernel warning messages for net/ipv4/tcp_output.c 2020-02-17 17:30 NFQUEUE/iptables and kernel warning messages for net/ipv4/tcp_output.c Vieri Di Paola @ 2020-02-18 12:39 ` Florian Westphal 2020-02-18 12:59 ` Vieri Di Paola 0 siblings, 1 reply; 5+ messages in thread From: Florian Westphal @ 2020-02-18 12:39 UTC (permalink / raw) To: Vieri Di Paola; +Cc: netfilter Vieri Di Paola <vieridipaola@gmail.com> wrote: > Hi, > > Whenever I use NFQUEUE/iptables to send traffic to an IDS/IPS (eg. > Suricata), I get an ugly kernel warning which can sometimes and on the > long run turn into a system freeze. > > I'm using NFQUEUE 0:5, and I'm running Suricata with -q 0 -q 1 -q 2 -q > 3 -q 4 -q 5 as arguments. > > I've already reported the issue on the LKML here: > > https://lkml.org/lkml/2020/2/13/1255 No idea. Suricata forces software-side segmentation for each packet, could be related. Can you post to suricata ML and get this patch working (untested): If the problem doesn't occur with segmentation off we've at least narrowed it down: diff --git a/src/source-nfq.c b/src/source-nfq.c --- a/src/source-nfq.c +++ b/src/source-nfq.c @@ -154,6 +154,7 @@ typedef enum NFQMode_ { } NFQMode; #define NFQ_FLAG_FAIL_OPEN (1 << 0) +#define NFQ_FLAG_GSO (1 << 2) typedef struct NFQCnf_ { NFQMode mode; @@ -242,6 +243,10 @@ void NFQInitConfig(char quiet) #endif } +#ifdef HAVE_NFQ_SET_QUEUE_FLAGS + nfq_config.flags |= NFQ_FLAG_GSO; +#endif + if ((ConfGetInt("nfq.repeat-mark", &value)) == 1) { nfq_config.mark = (uint32_t)value; } @@ -389,6 +394,16 @@ static inline void NFQMutexInit(NFQQueueVars *nq) } } +/* Ugly Hack */ +struct nfq_data { + void **data; +}; + +static uint32_t nfq_get_pktinfo(struct nfq_data *nfad) +{ + return ntohl(nfnl_get_data(nfad->data, NFQA_SKB_INFO, uint32_t)); +} + #define NFQMutexLock(nq) do { \ if ((nq)->use_mutex) \ SCMutexLock(&(nq)->mutex_qh); \ @@ -412,6 +427,7 @@ static int NFQSetupPkt (Packet *p, struct nfq_q_handle *qh, void *data) int ret; char *pktdata; struct nfqnl_msg_packet_hdr *ph; + uint32_t pktinfo; ph = nfq_get_msg_packet_hdr(tb); if (ph != NULL) { @@ -474,6 +490,11 @@ static int NFQSetupPkt (Packet *p, struct nfq_q_handle *qh, void *data) gettimeofday(&p->ts, NULL); } + pktinfo = nfq_get_pktinfo(tb); + /* kernel/nic will compute checksum on output */ + if (pktinfo & NFQA_SKB_CSUMNOTREADY) + p->flags |= PKT_IGNORE_CHECKSUM; + p->datalink = DLT_RAW; return 0; } @@ -674,16 +695,14 @@ static TmEcode NFQInitThread(NFQThreadVars *t, uint32_t queue_maxlen) #endif #ifdef HAVE_NFQ_SET_QUEUE_FLAGS - if (nfq_config.flags & NFQ_FLAG_FAIL_OPEN) { - uint32_t flags = NFQA_CFG_F_FAIL_OPEN; - uint32_t mask = NFQA_CFG_F_FAIL_OPEN; - int r = nfq_set_queue_flags(q->qh, mask, flags); + if (nfq_config.flags) { + int r = nfq_set_queue_flags(q->qh, nfq_config.flags, nfq_config.flags); if (r == -1) { - SCLogWarning(SC_ERR_NFQ_SET_MODE, "can't set fail-open mode: %s", - strerror(errno)); + SCLogWarning(SC_ERR_NFQ_SET_MODE, "can't set nfq flags 0x%x: %s", + nfq_config.flags, strerror(errno)); } else { - SCLogInfo("fail-open mode should be set on queue"); + SCLogInfo("Set flag modes 0x%x on queue", nfq_config.flags); } } #endif ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: NFQUEUE/iptables and kernel warning messages for net/ipv4/tcp_output.c 2020-02-18 12:39 ` Florian Westphal @ 2020-02-18 12:59 ` Vieri Di Paola 2020-02-18 13:21 ` Florian Westphal 0 siblings, 1 reply; 5+ messages in thread From: Vieri Di Paola @ 2020-02-18 12:59 UTC (permalink / raw) To: Florian Westphal; +Cc: netfilter On Tue, Feb 18, 2020 at 1:39 PM Florian Westphal <fw@strlen.de> wrote: > > get this patch working (untested) > +static uint32_t nfq_get_pktinfo(struct nfq_data *nfad) > +{ > + return ntohl(nfnl_get_data(nfad->data, NFQA_SKB_INFO, uint32_t)); I applied the patch, but I get this compilation error: In file included from suricata-common.h:180, from source-nfq.c:28: source-nfq.c: In function 'nfq_get_pktinfo': source-nfq.c:404:48: error: 'NFQA_SKB_INFO' undeclared (first use in this function) 404 | return ntohl(nfnl_get_data(nfad->data, NFQA_SKB_INFO, uint32_t)); | ^~~~~~~~~~~~~ I'll post to the suricata ML asap. Thanks, Vieri ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: NFQUEUE/iptables and kernel warning messages for net/ipv4/tcp_output.c 2020-02-18 12:59 ` Vieri Di Paola @ 2020-02-18 13:21 ` Florian Westphal 2020-02-19 19:37 ` Vieri Di Paola 0 siblings, 1 reply; 5+ messages in thread From: Florian Westphal @ 2020-02-18 13:21 UTC (permalink / raw) To: Vieri Di Paola; +Cc: Florian Westphal, netfilter Vieri Di Paola <vieridipaola@gmail.com> wrote: > On Tue, Feb 18, 2020 at 1:39 PM Florian Westphal <fw@strlen.de> wrote: > > > > get this patch working (untested) > > > +static uint32_t nfq_get_pktinfo(struct nfq_data *nfad) > > +{ > > + return ntohl(nfnl_get_data(nfad->data, NFQA_SKB_INFO, uint32_t)); > > I applied the patch, but I get this compilation error: > > In file included from suricata-common.h:180, > from source-nfq.c:28: > source-nfq.c: In function 'nfq_get_pktinfo': > source-nfq.c:404:48: error: 'NFQA_SKB_INFO' undeclared (first use in > this function) > 404 | return ntohl(nfnl_get_data(nfad->data, NFQA_SKB_INFO, > uint32_t)); > | ^~~~~~~~~~~~~ This means your kernel headers are older than 3.10. It should be part of /usr/include/linux/netfilter/nfnetlink_queue.h . You can substitute 14 instead, or add #define NFQA_SKB_INFO 14 > I'll post to the suricata ML asap. Thanks. ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: NFQUEUE/iptables and kernel warning messages for net/ipv4/tcp_output.c 2020-02-18 13:21 ` Florian Westphal @ 2020-02-19 19:37 ` Vieri Di Paola 0 siblings, 0 replies; 5+ messages in thread From: Vieri Di Paola @ 2020-02-19 19:37 UTC (permalink / raw) To: Florian Westphal; +Cc: netfilter On Tue, Feb 18, 2020 at 2:21 PM Florian Westphal <fw@strlen.de> wrote: > > > This means your kernel headers are older than 3.10. > It should be part of /usr/include/linux/netfilter/nfnetlink_queue.h . Actually I have 4.19. I also had to define another constant. In any case, it compiled OK, and the problem was not seen again for a test period of at least 15 hours (I usually had several of these kernel warnings almost each hour, so 15 hours in a weekday is significant). I then removed your patch, recompiled and ran suricata in nfq "accept" mode (default). It was set as "repeat mode" before. It hasn't been running long enough yet (7 hours), but for now I haven't seen any kernel warnings in "accept mode". So all I can say for now is that it seems I'm getting these kernel warnings when using Suricata in nfq repeat mode. I'll ask the Suricata ML what they think about that. Thanks, Vieri ^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2020-02-19 19:37 UTC | newest] Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2020-02-17 17:30 NFQUEUE/iptables and kernel warning messages for net/ipv4/tcp_output.c Vieri Di Paola 2020-02-18 12:39 ` Florian Westphal 2020-02-18 12:59 ` Vieri Di Paola 2020-02-18 13:21 ` Florian Westphal 2020-02-19 19:37 ` Vieri Di Paola
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.