* Re: [REGRESSION] 5.0-rc2: iptables -nvL consumes 100% of CPU and hogs memory with kernel 5.0-rc2
[not found] <1608957.xtEoauR4Ya@merkaba>
@ 2019-01-15 10:10 ` Michal Kubecek
2019-01-15 10:10 ` Michal Kubecek
2019-01-15 10:15 ` Florian Westphal
0 siblings, 2 replies; 6+ messages in thread
From: Michal Kubecek @ 2019-01-15 10:10 UTC (permalink / raw)
To: netfilter-devel; +Cc: netdev, linux-kernel, Martin Steigerwald
(CC netfilter-devel and netdev)
On Mon, Jan 14, 2019 at 11:53:10PM +0100, Martin Steigerwald wrote:
> Hi!
>
> Does that ring a bell with someone? For now I just downgraded, no time
> for detailed analysis.
>
> Debian bug report at:
>
> iptables -nvL consumes 100% of CPU and hogs memory with kernel 5.0-rc2
> https://bugs.debian.org/919325
>
> 4.20 works, 5.0-rc2 showed this issue with iptables. Configurations attached.
>
> Excerpt from Debian bug report follows:
>
> I upgraded to self-compiled 5.0-rc2 today and found the machine to be slow
> after startup. I saw iptables consuming 100% CPU, it only responded to
> SIGKILL. It got restarted several times, probably by some systemd service.
>
> Then I started 'iptables -nvL' manually. And I got this:
>
> % strace -p 5748
> [… tons more, in what appeared an endless loop …]
> recvmsg(3, {msg_name={sa_family=AF_NETLINK, nl_pid=0, nl_groups=00000000}, msg_namelen=12, msg_iov=[{iov_base={{len=372, type=0xa06 /* NLMSG_??? */, flags=NLM_F_MULTI|0x800, seq=0, pid=5748}, "\x02\x00\x00\x11\x0b\x00\x01\x00\x66\x69\x6c\x74\x65\x72\x00\x00\x0b\x00\x02\x00\x4f\x55\x54\x50\x55\x54\x00\x00\x0c\x00\x03\x00"...}, iov_len=16536}], msg_iovlen=1, msg_controllen=0, msg_flags=0}, 0) = 372
> recvmsg(3, {msg_name={sa_family=AF_NETLINK, nl_pid=0, nl_groups=00000000}, msg_namelen=12, msg_iov=[{iov_base={{len=372, type=0xa06 /* NLMSG_??? */, flags=NLM_F_MULTI|0x800, seq=0, pid=5748}, "\x02\x00\x00\x11\x0b\x00\x01\x00\x66\x69\x6c\x74\x65\x72\x00\x00\x0b\x00\x02\x00\x4f\x55\x54\x50\x55\x54\x00\x00\x0c\x00\x03\x00"...}, iov_len=16536}], msg_iovlen=1, msg_controllen=0, msg_flags=0}, 0) = 372
> recvmsg(3, {msg_name={sa_family=AF_NETLINK, nl_pid=0, nl_groups=00000000}, msg_namelen=12, msg_iov=[{iov_base={{len=372, type=0xa06 /* NLMSG_??? */, flags=NLM_F_MULTI|0x800, seq=0, pid=5748}, "\x02\x00\x00\x11\x0b\x00\x01\x00\x66\x69\x6c\x74\x65\x72\x00\x00\x0b\x00\x02\x00\x4f\x55\x54\x50\x55\x54\x00\x00\x0c\x00\x03\x00"...}, iov_len=16536}], msg_iovlen=1, msg_controllen=0, msg_flags=0}, 0) = 372
> recvmsg(3, ^C{msg_name={sa_family=AF_NETLINK, nl_pid=0, nl_groups=00000000}, msg_namelen=12, msg_iov=[{iov_base={{len=372, type=0xa06 /* NLMSG_??? */, flags=NLM_F_MULTI|0x800, seq=0, pid=5748}, "\x02\x00\x00\x11\x0b\x00\x01\x00\x66\x69\x6c\x74\x65\x72\x00\x00\x0b\x00\x02\x00\x4f\x55\x54\x50\x55\x54\x00\x00\x0c\x00\x03\x00"...}, iov_len=16536}], msg_iovlen=1, msg_controllen=0, msg_flags=0}, 0) = 372
> strace: Process 5748 detached
>
> and this (output from atop):
>
> PID TID MINFLT MAJFLT VSTEXT VSLIBS VDATA VSTACK VSIZE RSIZE PSIZE VGROW RGROW SWAPSZ RUID EUID MEM CMD 1/16
> 11575 - 61552 0 152K 2324K 5.0G 132K 5.1G 5.1G 0K 240.4M 240.5M 0K root root 33% iptables
>
> I had it growing till 10 GiB before I stopped it by SIGKILL to prevent
> excessive swapping.
>
> I will attach kernel configuration.
>
> That is all I am willing to spend time on for now before going to sleep.
> I will however reboot with older 4.20 kernel to see whether it is kernel
> related.
>
> […]
>
> -- System Information:
> Debian Release: buster/sid
> […]
> Kernel: Linux 5.0.0-rc2-tp520 (SMP w/4 CPU cores; PREEMPT)
>
> Thanks,
> --
> Martin
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [REGRESSION] 5.0-rc2: iptables -nvL consumes 100% of CPU and hogs memory with kernel 5.0-rc2
2019-01-15 10:10 ` [REGRESSION] 5.0-rc2: iptables -nvL consumes 100% of CPU and hogs memory with kernel 5.0-rc2 Michal Kubecek
@ 2019-01-15 10:10 ` Michal Kubecek
2019-01-15 10:15 ` Florian Westphal
1 sibling, 0 replies; 6+ messages in thread
From: Michal Kubecek @ 2019-01-15 10:10 UTC (permalink / raw)
To: netfilter-devel; +Cc: netdev, linux-kernel, Martin Steigerwald
(CC netfilter-devel and netdev)
On Mon, Jan 14, 2019 at 11:53:10PM +0100, Martin Steigerwald wrote:
> Hi!
>
> Does that ring a bell with someone? For now I just downgraded, no time
> for detailed analysis.
>
> Debian bug report at:
>
> iptables -nvL consumes 100% of CPU and hogs memory with kernel 5.0-rc2
> https://bugs.debian.org/919325
>
> 4.20 works, 5.0-rc2 showed this issue with iptables. Configurations attached.
>
> Excerpt from Debian bug report follows:
>
> I upgraded to self-compiled 5.0-rc2 today and found the machine to be slow
> after startup. I saw iptables consuming 100% CPU, it only responded to
> SIGKILL. It got restarted several times, probably by some systemd service.
>
> Then I started 'iptables -nvL' manually. And I got this:
>
> % strace -p 5748
> [… tons more, in what appeared an endless loop …]
> recvmsg(3, {msg_name={sa_family=AF_NETLINK, nl_pid=0, nl_groups=00000000}, msg_namelen=12, msg_iov=[{iov_base={{len=372, type=0xa06 /* NLMSG_??? */, flags=NLM_F_MULTI|0x800, seq=0, pid=5748}, "\x02\x00\x00\x11\x0b\x00\x01\x00\x66\x69\x6c\x74\x65\x72\x00\x00\x0b\x00\x02\x00\x4f\x55\x54\x50\x55\x54\x00\x00\x0c\x00\x03\x00"...}, iov_len=16536}], msg_iovlen=1, msg_controllen=0, msg_flags=0}, 0) = 372
> recvmsg(3, {msg_name={sa_family=AF_NETLINK, nl_pid=0, nl_groups=00000000}, msg_namelen=12, msg_iov=[{iov_base={{len=372, type=0xa06 /* NLMSG_??? */, flags=NLM_F_MULTI|0x800, seq=0, pid=5748}, "\x02\x00\x00\x11\x0b\x00\x01\x00\x66\x69\x6c\x74\x65\x72\x00\x00\x0b\x00\x02\x00\x4f\x55\x54\x50\x55\x54\x00\x00\x0c\x00\x03\x00"...}, iov_len=16536}], msg_iovlen=1, msg_controllen=0, msg_flags=0}, 0) = 372
> recvmsg(3, {msg_name={sa_family=AF_NETLINK, nl_pid=0, nl_groups=00000000}, msg_namelen=12, msg_iov=[{iov_base={{len=372, type=0xa06 /* NLMSG_??? */, flags=NLM_F_MULTI|0x800, seq=0, pid=5748}, "\x02\x00\x00\x11\x0b\x00\x01\x00\x66\x69\x6c\x74\x65\x72\x00\x00\x0b\x00\x02\x00\x4f\x55\x54\x50\x55\x54\x00\x00\x0c\x00\x03\x00"...}, iov_len=16536}], msg_iovlen=1, msg_controllen=0, msg_flags=0}, 0) = 372
> recvmsg(3, ^C{msg_name={sa_family=AF_NETLINK, nl_pid=0, nl_groups=00000000}, msg_namelen=12, msg_iov=[{iov_base={{len=372, type=0xa06 /* NLMSG_??? */, flags=NLM_F_MULTI|0x800, seq=0, pid=5748}, "\x02\x00\x00\x11\x0b\x00\x01\x00\x66\x69\x6c\x74\x65\x72\x00\x00\x0b\x00\x02\x00\x4f\x55\x54\x50\x55\x54\x00\x00\x0c\x00\x03\x00"...}, iov_len=16536}], msg_iovlen=1, msg_controllen=0, msg_flags=0}, 0) = 372
> strace: Process 5748 detached
>
> and this (output from atop):
>
> PID TID MINFLT MAJFLT VSTEXT VSLIBS VDATA VSTACK VSIZE RSIZE PSIZE VGROW RGROW SWAPSZ RUID EUID MEM CMD 1/16
> 11575 - 61552 0 152K 2324K 5.0G 132K 5.1G 5.1G 0K 240.4M 240.5M 0K root root 33% iptables
>
> I had it growing till 10 GiB before I stopped it by SIGKILL to prevent
> excessive swapping.
>
> I will attach kernel configuration.
>
> That is all I am willing to spend time on for now before going to sleep.
> I will however reboot with older 4.20 kernel to see whether it is kernel
> related.
>
> […]
>
> -- System Information:
> Debian Release: buster/sid
> […]
> Kernel: Linux 5.0.0-rc2-tp520 (SMP w/4 CPU cores; PREEMPT)
>
> Thanks,
> --
> Martin
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [REGRESSION] 5.0-rc2: iptables -nvL consumes 100% of CPU and hogs memory with kernel 5.0-rc2
2019-01-15 10:10 ` [REGRESSION] 5.0-rc2: iptables -nvL consumes 100% of CPU and hogs memory with kernel 5.0-rc2 Michal Kubecek
2019-01-15 10:10 ` Michal Kubecek
@ 2019-01-15 10:15 ` Florian Westphal
2019-01-15 10:15 ` Florian Westphal
2019-01-15 11:52 ` Martin Steigerwald
1 sibling, 2 replies; 6+ messages in thread
From: Florian Westphal @ 2019-01-15 10:15 UTC (permalink / raw)
To: Michal Kubecek; +Cc: netfilter-devel, netdev, linux-kernel, Martin Steigerwald
Michal Kubecek <mkubecek@suse.cz> wrote:
> > I upgraded to self-compiled 5.0-rc2 today and found the machine to be slow
> > after startup. I saw iptables consuming 100% CPU, it only responded to
> > SIGKILL. It got restarted several times, probably by some systemd service.
> >
> > Then I started 'iptables -nvL' manually. And I got this:
> >
> > % strace -p 5748
> > [… tons more, in what appeared an endless loop …]
This is fixed by:
http://patchwork.ozlabs.org/patch/1024772/
("netfilter: nf_tables: Fix for endless loop when dumping ruleset").
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [REGRESSION] 5.0-rc2: iptables -nvL consumes 100% of CPU and hogs memory with kernel 5.0-rc2
2019-01-15 10:15 ` Florian Westphal
@ 2019-01-15 10:15 ` Florian Westphal
2019-01-15 11:52 ` Martin Steigerwald
1 sibling, 0 replies; 6+ messages in thread
From: Florian Westphal @ 2019-01-15 10:15 UTC (permalink / raw)
To: Michal Kubecek; +Cc: netfilter-devel, netdev, linux-kernel, Martin Steigerwald
Michal Kubecek <mkubecek@suse.cz> wrote:
> > I upgraded to self-compiled 5.0-rc2 today and found the machine to be slow
> > after startup. I saw iptables consuming 100% CPU, it only responded to
> > SIGKILL. It got restarted several times, probably by some systemd service.
> >
> > Then I started 'iptables -nvL' manually. And I got this:
> >
> > % strace -p 5748
> > [… tons more, in what appeared an endless loop …]
This is fixed by:
http://patchwork.ozlabs.org/patch/1024772/
("netfilter: nf_tables: Fix for endless loop when dumping ruleset").
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [REGRESSION] 5.0-rc2: iptables -nvL consumes 100% of CPU and hogs memory with kernel 5.0-rc2
2019-01-15 10:15 ` Florian Westphal
2019-01-15 10:15 ` Florian Westphal
@ 2019-01-15 11:52 ` Martin Steigerwald
2019-01-15 11:52 ` Martin Steigerwald
1 sibling, 1 reply; 6+ messages in thread
From: Martin Steigerwald @ 2019-01-15 11:52 UTC (permalink / raw)
To: Florian Westphal; +Cc: Michal Kubecek, netfilter-devel, netdev, linux-kernel
Florian Westphal - 15.01.19, 11:15:
> Michal Kubecek <mkubecek@suse.cz> wrote:
> > > I upgraded to self-compiled 5.0-rc2 today and found the machine to
> > > be slow after startup. I saw iptables consuming 100% CPU, it only
> > > responded to SIGKILL. It got restarted several times, probably by
> > > some systemd service.
> > >
> > > Then I started 'iptables -nvL' manually. And I got this:
> > >
> > > % strace -p 5748
> > > [… tons more, in what appeared an endless loop …]
>
> This is fixed by:
>
> http://patchwork.ozlabs.org/patch/1024772/
> ("netfilter: nf_tables: Fix for endless loop when dumping ruleset").
Thanks, Florian.
Will wait for first 5.0-rcx with x=>2 that contains the fix. Bug closed on
Debian side already, was premature to report it there.
Ciao,
--
Martin
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [REGRESSION] 5.0-rc2: iptables -nvL consumes 100% of CPU and hogs memory with kernel 5.0-rc2
2019-01-15 11:52 ` Martin Steigerwald
@ 2019-01-15 11:52 ` Martin Steigerwald
0 siblings, 0 replies; 6+ messages in thread
From: Martin Steigerwald @ 2019-01-15 11:52 UTC (permalink / raw)
To: Florian Westphal; +Cc: Michal Kubecek, netfilter-devel, netdev, linux-kernel
Florian Westphal - 15.01.19, 11:15:
> Michal Kubecek <mkubecek@suse.cz> wrote:
> > > I upgraded to self-compiled 5.0-rc2 today and found the machine to
> > > be slow after startup. I saw iptables consuming 100% CPU, it only
> > > responded to SIGKILL. It got restarted several times, probably by
> > > some systemd service.
> > >
> > > Then I started 'iptables -nvL' manually. And I got this:
> > >
> > > % strace -p 5748
> > > [… tons more, in what appeared an endless loop …]
>
> This is fixed by:
>
> http://patchwork.ozlabs.org/patch/1024772/
> ("netfilter: nf_tables: Fix for endless loop when dumping ruleset").
Thanks, Florian.
Will wait for first 5.0-rcx with x=>2 that contains the fix. Bug closed on
Debian side already, was premature to report it there.
Ciao,
--
Martin
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2019-01-15 12:00 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
[not found] <1608957.xtEoauR4Ya@merkaba>
2019-01-15 10:10 ` [REGRESSION] 5.0-rc2: iptables -nvL consumes 100% of CPU and hogs memory with kernel 5.0-rc2 Michal Kubecek
2019-01-15 10:10 ` Michal Kubecek
2019-01-15 10:15 ` Florian Westphal
2019-01-15 10:15 ` Florian Westphal
2019-01-15 11:52 ` Martin Steigerwald
2019-01-15 11:52 ` Martin Steigerwald
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).