* Does anybody experience kernel crush when 'ebtable -t nat -L'
@ 2020-02-11 11:57 양유석
2020-02-11 12:03 ` G.W. Haywood
0 siblings, 1 reply; 11+ messages in thread
From: 양유석 @ 2020-02-11 11:57 UTC (permalink / raw)
To: netfilter
Hi netfilter,
I encountered strange kernel crush using ebtables, and did not find any clue.
Our cluster keep crushing so please shed light on me if somebody has
some exprience.
Current our environment is
- ubuntu xenial kernel 4.13
- openstack stein
- ebtables version 3.10
From today, we even does not know what makes crush since there was no
any log (literally kernel, application or something) Only that we
encounter is when linux bridge agent in openstack restarted, server
reboot without any log. After reboot, we see strange null string (@)
in syslog. (Google says it kernel failed to write file because of
unknown crush but no detail)
We tracked what excutable makes crush using bcc execsnoop, and finally
find ebtables!
To be sure, I just ran simply 'ebtables -t nat -L' , surprisingly it
do make panic just like before.
More hints to consider
- We run the command over 400 computes and only 10% (about 40 servers) crushed
- After crush, server never crushed anymore with same command
- We never used ebtables command, so there were no rules. Ebtables
module itself insmoded but not ebtables_nat .
I dig ebtables client and kernel module codebases, but from my
knowledges I could not fund which can make panic possibly.
Does anybody can guide how netfilter make crush? We are so desperate
to find clues so any help will be really appreciated.
Thanks
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Does anybody experience kernel crush when 'ebtable -t nat -L'
2020-02-11 11:57 Does anybody experience kernel crush when 'ebtable -t nat -L' 양유석
@ 2020-02-11 12:03 ` G.W. Haywood
2020-02-11 12:07 ` 양유석
0 siblings, 1 reply; 11+ messages in thread
From: G.W. Haywood @ 2020-02-11 12:03 UTC (permalink / raw)
To: 양유석; +Cc: netfilter
[-- Attachment #1: Type: text/plain, Size: 501 bytes --]
Hi there,
On Tue, 11 Feb 2020, 양유석 wrote:
> From today, we even does not know what makes crush since there was no
> any log (literally kernel, application or something) Only that we
> encounter is when linux bridge agent in openstack restarted, server
> reboot without any log. After reboot, we see strange null string (@)
> in syslog. ...
I cannot help you with the kernel crash but I suggest that you try
remote logging to another server, it might help provide more log info.
--
73,
Ged.
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Does anybody experience kernel crush when 'ebtable -t nat -L'
2020-02-11 12:03 ` G.W. Haywood
@ 2020-02-11 12:07 ` 양유석
2020-02-11 12:23 ` kfm
2020-02-11 12:47 ` G.W. Haywood
0 siblings, 2 replies; 11+ messages in thread
From: 양유석 @ 2020-02-11 12:07 UTC (permalink / raw)
To: G.W. Haywood, netfilter
Haywood,
Do you mean the remote logging like rsyslog? We did not try since
syslog does not emit at all. Is it worth to try though?
Anyway thanks for the advice, I will try to do anything to fix it.
2020년 2월 11일 (화) 오후 9:03, G.W. Haywood <ged@jubileegroup.co.uk>님이 작성:
>
> Hi there,
>
> On Tue, 11 Feb 2020, 양유석 wrote:
>
> > From today, we even does not know what makes crush since there was no
> > any log (literally kernel, application or something) Only that we
> > encounter is when linux bridge agent in openstack restarted, server
> > reboot without any log. After reboot, we see strange null string (@)
> > in syslog. ...
>
> I cannot help you with the kernel crash but I suggest that you try
> remote logging to another server, it might help provide more log info.
>
> --
>
> 73,
> Ged.
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Does anybody experience kernel crush when 'ebtable -t nat -L'
2020-02-11 12:07 ` 양유석
@ 2020-02-11 12:23 ` kfm
2020-02-11 12:47 ` G.W. Haywood
1 sibling, 0 replies; 11+ messages in thread
From: kfm @ 2020-02-11 12:23 UTC (permalink / raw)
To: netfilter
On 11/02/2020 12:07, 양유석 wrote:
> Haywood,
>
> Do you mean the remote logging like rsyslog? We did not try since
> syslog does not emit at all. Is it worth to try though?
I would suggest trying to use netconsole:
https://www.kernel.org/doc/Documentation/networking/netconsole.txt
https://wiki.archlinux.org/index.php/Netconsole
This has helped me on a few prior occasions in capturing information
that has proven vital to the subsequent debugging of unexpected kernel
crashes, one of which was kindly resolved by a Netfilter developer.
The other thing to consider is that 4.13 is not a supported kernel,
beyond the efforts of your distribution vendor. If at all possible, I
would suggest trying to reproduce this issue in (at least) one of the
supported longterm releases, as designated by the kernel.org homepage.
That may entail the use of a backported package, or even the compilation
of a kernel on your own part.
--
Kerin Millar
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Does anybody experience kernel crush when 'ebtable -t nat -L'
2020-02-11 12:07 ` 양유석
2020-02-11 12:23 ` kfm
@ 2020-02-11 12:47 ` G.W. Haywood
2020-02-11 13:29 ` 양유석
2020-02-11 14:00 ` 양유석
1 sibling, 2 replies; 11+ messages in thread
From: G.W. Haywood @ 2020-02-11 12:47 UTC (permalink / raw)
To: 양유석; +Cc: netfilter
[-- Attachment #1: Type: text/plain, Size: 645 bytes --]
Hi there,
On Tue, 11 Feb 2020, 양유석 wrote:
> Do you mean the remote logging like rsyslog?
Yes.
> We did not try since syslog does not emit at all. Is it worth to try
> though?
Like you I have occasionally seen strings of NULL bytes in the logs.
It has appeared to me that something has tried to write a message but
failed to get it to the disc because of kernel disc write caching,
which can be many seconds of data. If the messages go out over the
network, to a server which does not crash, they can have a better
chance of reaching some permanent storage. There are other tricks,
Mr. Millar's suggestion is good too.
--
73,
Ged.
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Does anybody experience kernel crush when 'ebtable -t nat -L'
2020-02-11 12:47 ` G.W. Haywood
@ 2020-02-11 13:29 ` 양유석
2020-02-11 14:15 ` G.W. Haywood
2020-02-11 14:00 ` 양유석
1 sibling, 1 reply; 11+ messages in thread
From: 양유석 @ 2020-02-11 13:29 UTC (permalink / raw)
To: G.W. Haywood, netfilter
2020년 2월 11일 (화) 오후 9:47, G.W. Haywood <ged@jubileegroup.co.uk>님이 작성:
>
> Hi there,
>
> On Tue, 11 Feb 2020, 양유석 wrote:
>
> > Do you mean the remote logging like rsyslog?
>
> Yes.
>
> > We did not try since syslog does not emit at all. Is it worth to try
> > though?
>
> Like you I have occasionally seen strings of NULL bytes in the logs.
> It has appeared to me that something has tried to write a message but
> failed to get it to the disc because of kernel disc write caching,
> which can be many seconds of data. If the messages go out over the
> network, to a server which does not crash, they can have a better
> chance of reaching some permanent storage. There are other tricks,
> Mr. Millar's suggestion is good too.
Oh, I did not realize difference between network and disc. It's
reasonable so surely worth to try. Thanks.
Also, where is Mr.Millar's suggestion? I could receive his e-mail.
Regards,
>
> --
>
> 73,
> Ged.
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Does anybody experience kernel crush when 'ebtable -t nat -L'
2020-02-11 12:47 ` G.W. Haywood
2020-02-11 13:29 ` 양유석
@ 2020-02-11 14:00 ` 양유석
1 sibling, 0 replies; 11+ messages in thread
From: 양유석 @ 2020-02-11 14:00 UTC (permalink / raw)
To: G.W. Haywood; +Cc: netfilter
Oh, I did not realize difference between network and disc. It's
reasonable so surely worth to try. Thanks.
Also, where is Mr.Millar's suggestion? I could not receive his e-mail.
(Sadly G-mail also keep preventing me to send mail..)
Regards,
2020년 2월 11일 (화) 오후 9:47, G.W. Haywood <ged@jubileegroup.co.uk>님이 작성:
>
> Hi there,
>
> On Tue, 11 Feb 2020, 양유석 wrote:
>
> > Do you mean the remote logging like rsyslog?
>
> Yes.
>
> > We did not try since syslog does not emit at all. Is it worth to try
> > though?
>
> Like you I have occasionally seen strings of NULL bytes in the logs.
> It has appeared to me that something has tried to write a message but
> failed to get it to the disc because of kernel disc write caching,
> which can be many seconds of data. If the messages go out over the
> network, to a server which does not crash, they can have a better
> chance of reaching some permanent storage. There are other tricks,
> Mr. Millar's suggestion is good too.
>
> --
>
> 73,
> Ged.
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Does anybody experience kernel crush when 'ebtable -t nat -L'
2020-02-11 13:29 ` 양유석
@ 2020-02-11 14:15 ` G.W. Haywood
2020-02-12 2:37 ` 양유석
0 siblings, 1 reply; 11+ messages in thread
From: G.W. Haywood @ 2020-02-11 14:15 UTC (permalink / raw)
To: 양유석; +Cc: netfilter
[-- Attachment #1: Type: text/plain, Size: 264 bytes --]
Hi there,
On Tue, 11 Feb 2020, 양유석 wrote:
> 2020년 2월 11일 (화) 오후 9:47, G.W. Haywood ...
>> ... Mr. Millar's suggestion is good too.
>
> ... where is Mr.Millar's suggestion? ...
https://marc.info/?l=netfilter&m=158142378531784&w=2
--
73,
Ged.
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Does anybody experience kernel crush when 'ebtable -t nat -L'
2020-02-11 14:15 ` G.W. Haywood
@ 2020-02-12 2:37 ` 양유석
2020-02-13 8:25 ` Youseok Yang
0 siblings, 1 reply; 11+ messages in thread
From: 양유석 @ 2020-02-12 2:37 UTC (permalink / raw)
To: G.W. Haywood; +Cc: netfilter
I wish I can check rsyslog first. It do have the last command (etables
--concurrent -t nat -L). But unfortunately, there is no more command.
I will try to use netconsole which seems to log for kernel.
2020년 2월 11일 (화) 오후 11:15, G.W. Haywood <ged@jubileegroup.co.uk>님이 작성:
>
> Hi there,
>
> On Tue, 11 Feb 2020, 양유석 wrote:
>
> > 2020년 2월 11일 (화) 오후 9:47, G.W. Haywood ...
> >> ... Mr. Millar's suggestion is good too.
> >
> > ... where is Mr.Millar's suggestion? ...
>
> https://marc.info/?l=netfilter&m=158142378531784&w=2
>
> --
>
> 73,
> Ged.
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Does anybody experience kernel crush when 'ebtable -t nat -L'
2020-02-12 2:37 ` 양유석
@ 2020-02-13 8:25 ` Youseok Yang
2020-02-13 9:08 ` Youseok Yang
0 siblings, 1 reply; 11+ messages in thread
From: Youseok Yang @ 2020-02-13 8:25 UTC (permalink / raw)
To: G.W. Haywood; +Cc: netfilter
We got strace log
root@krane-ay1prod-com0280:~# strace -fv ebtables -t nat --concurrent -L
execve("/sbin/ebtables", ["ebtables", "-t", "nat", "--concurrent",
"-L"], ["PYENV_ROOT=/opt/pyenv", "SHELL=/bin/bash",
"TERM=xterm-256color", "HISTSIZE=1000", "SSH_TTY=/dev/pts/11",
"LC_ALL=en_US.UTF-8", "USER=root",
"LS_COLORS=rs=0:di=01;34:ln=01;36"..., "SUDO_USER=deploy",
"SUDO_UID=1000", "TMOUT=7200", "USERNAME=root",
"LIBVIRT_DEFAULT_URI=qemu:///syst"..., "SSH_CLI=10.60.39.222",
"MAIL=/var/mail/root", "PATH=/opt/pyenv/shims:/opt/pyenv"...,
"PWD=/root", "LANG=en_US.UTF-8", "PYENV_SHELL=bash", "SHLVL=1",
"SUDO_COMMAND=/bin/bash", "HOME=/root",
"ADDITIONAL_NO_PROXY=.krane.iwila"..., "LANGUAGE=en_US:",
"LC_TERMINAL_VERSION=3.3.7", "LOGNAME=root", "LC_CTYPE=UTF-8",
"LESSOPEN=| /usr/bin/lesspipe %s", "PROMPT_COMMAND=EXIT_CODE=$? &&
d"..., "SUDO_GID=1000", "LC_TERMINAL=iTerm2",
"LESSCLOSE=/usr/bin/lesspipe %s %"..., "HISTTIMEFORMAT=%F %T ",
"SSH_ID=RSA-SHA256:Ap2X89VM+tClGr"..., "_=/usr/bin/strace"]) = 0
brk(NULL) = 0x1f98000
access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or directory)
open("/lib/ebtables/tls/x86_64/libebt_802_3.so", O_RDONLY|O_CLOEXEC) =
-1 ENOENT (No such file or directory)
stat("/lib/ebtables/tls/x86_64", 0x7ffe0d016c20) = -1 ENOENT (No such
file or directory)
open("/lib/ebtables/tls/libebt_802_3.so", O_RDONLY|O_CLOEXEC) = -1
ENOENT (No such file or directory)
stat("/lib/ebtables/tls", 0x7ffe0d016c20) = -1 ENOENT (No such file or
directory)
open("/lib/ebtables/x86_64/libebt_802_3.so", O_RDONLY|O_CLOEXEC) = -1
ENOENT (No such file or directory)
stat("/lib/ebtables/x86_64", 0x7ffe0d016c20) = -1 ENOENT (No such file
or directory)
open("/lib/ebtables/libebt_802_3.so", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0@\7\0\0\0\0\0\0"...,
832) = 832
fstat(3, {st_dev=makedev(259, 4), st_ino=87819095,
st_mode=S_IFREG|0644, st_nlink=1, st_uid=0, st_gid=0, st_blksize=4096,
st_blocks=24, st_size=10032, st_atime=2020/02/13-14:54:35.335391990,
st_mtime=2018/06/29-03:17:40, st_ctime=2019/10/11-17:14:44.944338761})
= 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
0) = 0x7ff859fcc000
mmap(NULL, 2105664, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3,
0) = 0x7ff859ba5000
mprotect(0x7ff859ba6000, 2097152, PROT_NONE) = 0
mmap(0x7ff859da6000, 8192, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1000) = 0x7ff859da6000
close(3) = 0
open("/lib/ebtables/libebt_nat.so", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0000\t\0\0\0\0\0\0"...,
832) = 832
fstat(3, {st_dev=makedev(259, 4), st_ino=87819085,
st_mode=S_IFREG|0644, st_nlink=1, st_uid=0, st_gid=0, st_blksize=4096,
st_blocks=24, st_size=10456, st_atime=2020/02/13-14:54:35.335391990,
st_mtime=2018/06/29-03:17:40, st_ctime=2019/10/11-17:14:44.944338761})
= 0
mmap(NULL, 2106024, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3,
0) = 0x7ff8599a2000
mprotect(0x7ff8599a4000, 2093056, PROT_NONE) = 0
mmap(0x7ff859ba3000, 8192, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1000) = 0x7ff859ba3000
close(3) = 0
open("/lib/ebtables/libebt_arp.so", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\20\f\0\0\0\0\0\0"...,
832) = 832
fstat(3, {st_dev=makedev(259, 4), st_ino=87819109,
st_mode=S_IFREG|0644, st_nlink=1, st_uid=0, st_gid=0, st_blksize=4096,
st_blocks=32, st_size=14496, st_atime=2020/02/13-14:54:35.335391990,
st_mtime=2018/06/29-03:17:40, st_ctime=2019/10/11-17:14:44.944338761})
= 0
mmap(NULL, 2110048, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3,
0) = 0x7ff85979e000
mprotect(0x7ff8597a0000, 2097152, PROT_NONE) = 0
mmap(0x7ff8599a0000, 8192, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x2000) = 0x7ff8599a0000
close(3) = 0
open("/lib/ebtables/libebt_arpreply.so", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\20\10\0\0\0\0\0\0"...,
832) = 832
fstat(3, {st_dev=makedev(259, 4), st_ino=87819097,
st_mode=S_IFREG|0644, st_nlink=1, st_uid=0, st_gid=0, st_blksize=4096,
st_blocks=24, st_size=10136, st_atime=2020/02/13-14:54:35.335391990,
st_mtime=2018/06/29-03:17:40, st_ctime=2019/10/11-17:14:44.944338761})
= 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
0) = 0x7ff859fcb000
mmap(NULL, 2105704, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3,
0) = 0x7ff85959b000
mprotect(0x7ff85959d000, 2093056, PROT_NONE) = 0
mmap(0x7ff85979c000, 8192, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1000) = 0x7ff85979c000
close(3) = 0
open("/lib/ebtables/libebt_ip.so", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\300\n\0\0\0\0\0\0"...,
832) = 832
fstat(3, {st_dev=makedev(259, 4), st_ino=87819079,
st_mode=S_IFREG|0644, st_nlink=1, st_uid=0, st_gid=0, st_blksize=4096,
st_blocks=24, st_size=10384, st_atime=2020/02/13-14:54:35.335391990,
st_mtime=2018/06/29-03:17:40, st_ctime=2019/10/11-17:14:44.944338761})
= 0
mmap(NULL, 2106016, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3,
0) = 0x7ff859398000
mprotect(0x7ff85939a000, 2093056, PROT_NONE) = 0
mmap(0x7ff859599000, 8192, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1000) = 0x7ff859599000
close(3) = 0
open("/lib/ebtables/libebt_ip6.so", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\300\16\0\0\0\0\0\0"...,
832) = 832
fstat(3, {st_dev=makedev(259, 4), st_ino=87819103,
st_mode=S_IFREG|0644, st_nlink=1, st_uid=0, st_gid=0, st_blksize=4096,
st_blocks=40, st_size=18304, st_atime=2020/02/13-14:54:35.335391990,
st_mtime=2018/06/29-03:17:40, st_ctime=2019/10/11-17:14:44.944338761})
= 0
mmap(NULL, 2113856, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3,
0) = 0x7ff859193000
mprotect(0x7ff859196000, 2097152, PROT_NONE) = 0
mmap(0x7ff859396000, 8192, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x3000) = 0x7ff859396000
close(3) = 0
open("/lib/ebtables/libebt_standard.so", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\240\5\0\0\0\0\0\0"...,
832) = 832
fstat(3, {st_dev=makedev(259, 4), st_ino=87819073,
st_mode=S_IFREG|0644, st_nlink=1, st_uid=0, st_gid=0, st_blksize=4096,
st_blocks=16, st_size=5808, st_atime=2020/02/13-14:54:35.339392028,
st_mtime=2018/06/29-03:17:40, st_ctime=2019/10/11-17:14:44.944338761})
= 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
0) = 0x7ff859fca000
mmap(NULL, 2101472, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3,
0) = 0x7ff858f91000
mprotect(0x7ff858f92000, 2093056, PROT_NONE) = 0
mmap(0x7ff859191000, 8192, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0) = 0x7ff859191000
close(3) = 0
open("/lib/ebtables/libebt_log.so", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\320\t\0\0\0\0\0\0"...,
832) = 832
fstat(3, {st_dev=makedev(259, 4), st_ino=87819087,
st_mode=S_IFREG|0644, st_nlink=1, st_uid=0, st_gid=0, st_blksize=4096,
st_blocks=24, st_size=10272, st_atime=2020/02/13-14:54:35.339392028,
st_mtime=2018/06/29-03:17:40, st_ctime=2019/10/11-17:14:44.944338761})
= 0
mmap(NULL, 2105824, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3,
0) = 0x7ff858d8e000
mprotect(0x7ff858d90000, 2093056, PROT_NONE) = 0
mmap(0x7ff858f8f000, 8192, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1000) = 0x7ff858f8f000
close(3) = 0
open("/lib/ebtables/libebt_redirect.so", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\20\7\0\0\0\0\0\0"...,
832) = 832
fstat(3, {st_dev=makedev(259, 4), st_ino=87819071,
st_mode=S_IFREG|0644, st_nlink=1, st_uid=0, st_gid=0, st_blksize=4096,
st_blocks=16, st_size=5904, st_atime=2020/02/13-14:54:35.339392028,
st_mtime=2018/06/29-03:17:40, st_ctime=2019/10/11-17:14:44.944338761})
= 0
mmap(NULL, 2101536, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3,
0) = 0x7ff858b8c000
mprotect(0x7ff858b8d000, 2093056, PROT_NONE) = 0
mmap(0x7ff858d8c000, 8192, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0) = 0x7ff858d8c000
close(3) = 0
open("/lib/ebtables/libebt_vlan.so", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0000\10\0\0\0\0\0\0"...,
832) = 832
fstat(3, {st_dev=makedev(259, 4), st_ino=87819107,
st_mode=S_IFREG|0644, st_nlink=1, st_uid=0, st_gid=0, st_blksize=4096,
st_blocks=24, st_size=10168, st_atime=2020/02/13-14:54:35.339392028,
st_mtime=2018/06/29-03:17:40, st_ctime=2019/10/11-17:14:44.944338761})
= 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
0) = 0x7ff859fc9000
mmap(NULL, 2105736, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3,
0) = 0x7ff858989000
mprotect(0x7ff85898b000, 2093056, PROT_NONE) = 0
mmap(0x7ff858b8a000, 8192, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1000) = 0x7ff858b8a000
close(3) = 0
open("/lib/ebtables/libebt_mark_m.so", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0
\7\0\0\0\0\0\0"..., 832) = 832
fstat(3, {st_dev=makedev(259, 4), st_ino=87819099,
st_mode=S_IFREG|0644, st_nlink=1, st_uid=0, st_gid=0, st_blksize=4096,
st_blocks=16, st_size=5904, st_atime=2020/02/13-14:54:35.339392028,
st_mtime=2018/06/29-03:17:40, st_ctime=2019/10/11-17:14:44.944338761})
= 0
mmap(NULL, 2101536, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3,
0) = 0x7ff858787000
mprotect(0x7ff858788000, 2093056, PROT_NONE) = 0
mmap(0x7ff858987000, 8192, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0) = 0x7ff858987000
close(3) = 0
open("/lib/ebtables/libebt_mark.so", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\320\7\0\0\0\0\0\0"...,
832) = 832
fstat(3, {st_dev=makedev(259, 4), st_ino=87819111,
st_mode=S_IFREG|0644, st_nlink=1, st_uid=0, st_gid=0, st_blksize=4096,
st_blocks=24, st_size=10232, st_atime=2020/02/13-14:54:35.339392028,
st_mtime=2018/06/29-03:17:40, st_ctime=2019/10/11-17:14:44.944338761})
= 0
mmap(NULL, 2105800, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3,
0) = 0x7ff858584000
mprotect(0x7ff858586000, 2093056, PROT_NONE) = 0
mmap(0x7ff858785000, 8192, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1000) = 0x7ff858785000
close(3) = 0
open("/lib/ebtables/libebt_pkttype.so", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0p\10\0\0\0\0\0\0"...,
832) = 832
fstat(3, {st_dev=makedev(259, 4), st_ino=87819093,
st_mode=S_IFREG|0644, st_nlink=1, st_uid=0, st_gid=0, st_blksize=4096,
st_blocks=24, st_size=10064, st_atime=2020/02/13-14:54:35.339392028,
st_mtime=2018/06/29-03:17:40, st_ctime=2019/10/11-17:14:44.944338761})
= 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
0) = 0x7ff859fc8000
mmap(NULL, 2105696, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3,
0) = 0x7ff858381000
mprotect(0x7ff858382000, 2097152, PROT_NONE) = 0
mmap(0x7ff858582000, 8192, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1000) = 0x7ff858582000
close(3) = 0
open("/lib/ebtables/libebt_stp.so", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\320\n\0\0\0\0\0\0"...,
832) = 832
fstat(3, {st_dev=makedev(259, 4), st_ino=87819113,
st_mode=S_IFREG|0644, st_nlink=1, st_uid=0, st_gid=0, st_blksize=4096,
st_blocks=32, st_size=14512, st_atime=2020/02/13-14:54:35.339392028,
st_mtime=2018/06/29-03:17:40, st_ctime=2019/10/11-17:14:44.944338761})
= 0
mmap(NULL, 2110144, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3,
0) = 0x7ff85817d000
mprotect(0x7ff858180000, 2093056, PROT_NONE) = 0
mmap(0x7ff85837f000, 8192, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x2000) = 0x7ff85837f000
close(3) = 0
open("/lib/ebtables/libebt_among.so", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\320\v\0\0\0\0\0\0"...,
832) = 832
fstat(3, {st_dev=makedev(259, 4), st_ino=87819075,
st_mode=S_IFREG|0644, st_nlink=1, st_uid=0, st_gid=0, st_blksize=4096,
st_blocks=32, st_size=14392, st_atime=2020/02/13-14:54:35.339392028,
st_mtime=2018/06/29-03:17:40, st_ctime=2019/10/11-17:14:44.944338761})
= 0
mmap(NULL, 2109960, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3,
0) = 0x7ff857f79000
mprotect(0x7ff857f7c000, 2093056, PROT_NONE) = 0
mmap(0x7ff85817b000, 8192, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x2000) = 0x7ff85817b000
close(3) = 0
open("/lib/ebtables/libebt_limit.so", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\200\10\0\0\0\0\0\0"...,
832) = 832
fstat(3, {st_dev=makedev(259, 4), st_ino=87819105,
st_mode=S_IFREG|0644, st_nlink=1, st_uid=0, st_gid=0, st_blksize=4096,
st_blocks=24, st_size=10144, st_atime=2020/02/13-14:54:35.339392028,
st_mtime=2018/06/29-03:17:40, st_ctime=2019/10/11-17:14:44.944338761})
= 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
0) = 0x7ff859fc7000
mmap(NULL, 2105696, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3,
0) = 0x7ff857d76000
mprotect(0x7ff857d78000, 2093056, PROT_NONE) = 0
mmap(0x7ff857f77000, 8192, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1000) = 0x7ff857f77000
close(3) = 0
open("/lib/ebtables/libebt_ulog.so", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0`\10\0\0\0\0\0\0"...,
832) = 832
fstat(3, {st_dev=makedev(259, 4), st_ino=87819083,
st_mode=S_IFREG|0644, st_nlink=1, st_uid=0, st_gid=0, st_blksize=4096,
st_blocks=24, st_size=10160, st_atime=2020/02/13-14:54:35.339392028,
st_mtime=2018/06/29-03:17:40, st_ctime=2019/10/11-17:14:44.944338761})
= 0
mmap(NULL, 2105792, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3,
0) = 0x7ff857b73000
mprotect(0x7ff857b75000, 2093056, PROT_NONE) = 0
mmap(0x7ff857d74000, 8192, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1000) = 0x7ff857d74000
close(3) = 0
open("/lib/ebtables/libebt_nflog.so", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0`\10\0\0\0\0\0\0"...,
832) = 832
fstat(3, {st_dev=makedev(259, 4), st_ino=87819081,
st_mode=S_IFREG|0644, st_nlink=1, st_uid=0, st_gid=0, st_blksize=4096,
st_blocks=24, st_size=10160, st_atime=2020/02/13-14:54:35.343392065,
st_mtime=2018/06/29-03:17:40, st_ctime=2019/10/11-17:14:44.944338761})
= 0
mmap(NULL, 2105792, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3,
0) = 0x7ff857970000
mprotect(0x7ff857972000, 2093056, PROT_NONE) = 0
mmap(0x7ff857b71000, 8192, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1000) = 0x7ff857b71000
close(3) = 0
open("/lib/ebtables/libebtable_filter.so", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\0\5\0\0\0\0\0\0"...,
832) = 832
fstat(3, {st_dev=makedev(259, 4), st_ino=87819091,
st_mode=S_IFREG|0644, st_nlink=1, st_uid=0, st_gid=0, st_blksize=4096,
st_blocks=16, st_size=5672, st_atime=2020/02/13-14:54:35.343392065,
st_mtime=2018/06/29-03:17:40, st_ctime=2019/10/11-17:14:44.944338761})
= 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
0) = 0x7ff859fc6000
mmap(NULL, 2101368, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3,
0) = 0x7ff85776e000
mprotect(0x7ff85776f000, 2093056, PROT_NONE) = 0
mmap(0x7ff85796e000, 8192, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0) = 0x7ff85796e000
close(3) = 0
open("/lib/ebtables/libebtable_nat.so", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\0\5\0\0\0\0\0\0"...,
832) = 832
fstat(3, {st_dev=makedev(259, 4), st_ino=87819101,
st_mode=S_IFREG|0644, st_nlink=1, st_uid=0, st_gid=0, st_blksize=4096,
st_blocks=16, st_size=5672, st_atime=2020/02/13-14:54:35.343392065,
st_mtime=2018/06/29-03:17:40, st_ctime=2019/10/11-17:14:44.944338761})
= 0
mmap(NULL, 2101368, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3,
0) = 0x7ff85756c000
mprotect(0x7ff85756d000, 2093056, PROT_NONE) = 0
mmap(0x7ff85776c000, 8192, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0) = 0x7ff85776c000
close(3) = 0
open("/lib/ebtables/libebtable_broute.so", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0P\4\0\0\0\0\0\0"...,
832) = 832
fstat(3, {st_dev=makedev(259, 4), st_ino=87819077,
st_mode=S_IFREG|0644, st_nlink=1, st_uid=0, st_gid=0, st_blksize=4096,
st_blocks=16, st_size=5672, st_atime=2020/02/13-14:54:35.343392065,
st_mtime=2018/06/29-03:17:40, st_ctime=2019/10/11-17:14:44.944338761})
= 0
mmap(NULL, 2101368, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3,
0) = 0x7ff85736a000
mprotect(0x7ff85736b000, 2093056, PROT_NONE) = 0
mmap(0x7ff85756a000, 8192, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0) = 0x7ff85756a000
close(3) = 0
open("/lib/ebtables/libebtc.so", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0
0\0\0\0\0\0\0"..., 832) = 832
fstat(3, {st_dev=makedev(259, 4), st_ino=87819089,
st_mode=S_IFREG|0644, st_nlink=1, st_uid=0, st_gid=0, st_blksize=4096,
st_blocks=136, st_size=69536, st_atime=2020/02/13-14:54:35.343392065,
st_mtime=2018/06/29-03:17:40, st_ctime=2019/10/11-17:14:44.944338761})
= 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
0) = 0x7ff859fc5000
mmap(NULL, 2173680, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3,
0) = 0x7ff857157000
mprotect(0x7ff857167000, 2093056, PROT_NONE) = 0
mmap(0x7ff857366000, 8192, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0xf000) = 0x7ff857366000
mmap(0x7ff857368000, 6896, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7ff857368000
close(3) = 0
open("/lib/ebtables/libc.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No
such file or directory)
open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_dev=makedev(259, 4), st_ino=18352138,
st_mode=S_IFREG|0644, st_nlink=1, st_uid=0, st_gid=0, st_blksize=4096,
st_blocks=88, st_size=42101, st_atime=2020/02/13-14:39:47.058940019,
st_mtime=2020/02/13-14:39:47.054939981,
st_ctime=2020/02/13-14:39:47.054939981}) = 0
mmap(NULL, 42101, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7ff859fba000
close(3) = 0
access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/libc.so.6", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\3\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0P\t\2\0\0\0\0\0"...,
832) = 832
fstat(3, {st_dev=makedev(259, 4), st_ino=87818433,
st_mode=S_IFREG|0755, st_nlink=1, st_uid=0, st_gid=0, st_blksize=4096,
st_blocks=3656, st_size=1868984,
st_atime=2020/02/12-15:53:36.963494808, st_mtime=2019/02/06-05:11:22,
st_ctime=2019/10/10-15:50:21.797152380}) = 0
mmap(NULL, 3971488, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3,
0) = 0x7ff856d8d000
mprotect(0x7ff856f4d000, 2097152, PROT_NONE) = 0
mmap(0x7ff85714d000, 24576, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1c0000) = 0x7ff85714d000
mmap(0x7ff857153000, 14752, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7ff857153000
close(3) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
0) = 0x7ff859fb9000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
0) = 0x7ff859fb8000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
0) = 0x7ff859fb7000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
0) = 0x7ff859fb6000
arch_prctl(ARCH_SET_FS, 0x7ff859fb7700) = 0
mprotect(0x7ff85714d000, 16384, PROT_READ) = 0
mprotect(0x7ff857366000, 4096, PROT_READ) = 0
mprotect(0x7ff85756a000, 4096, PROT_READ) = 0
mprotect(0x7ff85776c000, 4096, PROT_READ) = 0
mprotect(0x7ff85796e000, 4096, PROT_READ) = 0
mprotect(0x7ff857b71000, 4096, PROT_READ) = 0
mprotect(0x7ff857d74000, 4096, PROT_READ) = 0
mprotect(0x7ff857f77000, 4096, PROT_READ) = 0
mprotect(0x7ff85817b000, 4096, PROT_READ) = 0
mprotect(0x7ff85837f000, 4096, PROT_READ) = 0
mprotect(0x7ff858582000, 4096, PROT_READ) = 0
mprotect(0x7ff858785000, 4096, PROT_READ) = 0
mprotect(0x7ff858987000, 4096, PROT_READ) = 0
mprotect(0x7ff858b8a000, 4096, PROT_READ) = 0
mprotect(0x7ff858d8c000, 4096, PROT_READ) = 0
mprotect(0x7ff858f8f000, 4096, PROT_READ) = 0
mprotect(0x7ff859191000, 4096, PROT_READ) = 0
mprotect(0x7ff859396000, 4096, PROT_READ) = 0
mprotect(0x7ff859599000, 4096, PROT_READ) = 0
mprotect(0x7ff85979c000, 4096, PROT_READ) = 0
mprotect(0x7ff8599a0000, 4096, PROT_READ) = 0
mprotect(0x7ff859ba3000, 4096, PROT_READ) = 0
mprotect(0x7ff859da6000, 4096, PROT_READ) = 0
mprotect(0x600000, 4096, PROT_READ) = 0
mprotect(0x7ff859fcd000, 4096, PROT_READ) = 0
munmap(0x7ff859fba000, 42101) = 0
brk(NULL) = 0x1f98000
brk(0x1fb9000) = 0x1fb9000
rt_sigaction(SIGINT, {0x7ff85715f8d0, [INT], SA_RESTORER|SA_RESTART,
0x7ff856dc24b0}, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGTERM, {0x7ff85715f8d0, [TERM], SA_RESTORER|SA_RESTART,
0x7ff856dc24b0}, {SIG_DFL, [], 0}, 8) = 0
open("/run/ebtables.lock", O_WRONLY|O_CREAT, 0600) = 3
fcntl(3, F_SETLK, {l_type=F_WRLCK, l_whence=SEEK_SET, l_start=0, l_len=0}) = 0
socket(PF_INET, SOCK_RAW, IPPROTO_IGMP) = 4
getsockopt(4, SOL_IP, EBT_SO_GET_INFO, packet_write_wait: Connection
to 10.91.13.28 port 22: Broken pipe
And bug found which likely to affect this case. (ebtables: fix race
condition in frame_filter_net_init():
https://lore.kernel.org/patchwork/patch/834743/). I checked our kernel
(4.13.0-41-generic) is running without the patch and I suspect the bug
makes kernel crush.
But I cannot replay the bug with some script (while test 1; do rmmod
ebtable_nat; rmmod ebtable_filter; ebtables -t nat -L; done)
Does anybody let me know how to reproduce the bug?
2020년 2월 12일 (수) 오전 11:37, 양유석 <ileixe@gmail.com>님이 작성:
>
> I wish I can check rsyslog first. It do have the last command (etables
> --concurrent -t nat -L). But unfortunately, there is no more command.
> I will try to use netconsole which seems to log for kernel.
>
> 2020년 2월 11일 (화) 오후 11:15, G.W. Haywood <ged@jubileegroup.co.uk>님이 작성:
> >
> > Hi there,
> >
> > On Tue, 11 Feb 2020, 양유석 wrote:
> >
> > > 2020년 2월 11일 (화) 오후 9:47, G.W. Haywood ...
> > >> ... Mr. Millar's suggestion is good too.
> > >
> > > ... where is Mr.Millar's suggestion? ...
> >
> > https://marc.info/?l=netfilter&m=158142378531784&w=2
> >
> > --
> >
> > 73,
> > Ged.
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Does anybody experience kernel crush when 'ebtable -t nat -L'
2020-02-13 8:25 ` Youseok Yang
@ 2020-02-13 9:08 ` Youseok Yang
0 siblings, 0 replies; 11+ messages in thread
From: Youseok Yang @ 2020-02-13 9:08 UTC (permalink / raw)
To: G.W. Haywood; +Cc: netfilter
Finally I got an answer.
The bug do affect our crush. I reproduce the crush with massive bridge
traffic + module load/unload repeat which is
iperf / while test 1; do rmmod ebtable_nat; rmmod ebtable_filter;
ebtables -t nat -L; done
Unfortunately, netconsole does not work in that case though, It really
helps to track! Thanks
2020년 2월 13일 (목) 오후 5:25, Youseok Yang <ileixe@gmail.com>님이 작성:
>
> We got strace log
>
> root@krane-ay1prod-com0280:~# strace -fv ebtables -t nat --concurrent -L
> execve("/sbin/ebtables", ["ebtables", "-t", "nat", "--concurrent",
> "-L"], ["PYENV_ROOT=/opt/pyenv", "SHELL=/bin/bash",
> "TERM=xterm-256color", "HISTSIZE=1000", "SSH_TTY=/dev/pts/11",
> "LC_ALL=en_US.UTF-8", "USER=root",
> "LS_COLORS=rs=0:di=01;34:ln=01;36"..., "SUDO_USER=deploy",
> "SUDO_UID=1000", "TMOUT=7200", "USERNAME=root",
> "LIBVIRT_DEFAULT_URI=qemu:///syst"..., "SSH_CLI=10.60.39.222",
> "MAIL=/var/mail/root", "PATH=/opt/pyenv/shims:/opt/pyenv"...,
> "PWD=/root", "LANG=en_US.UTF-8", "PYENV_SHELL=bash", "SHLVL=1",
> "SUDO_COMMAND=/bin/bash", "HOME=/root",
> "ADDITIONAL_NO_PROXY=.krane.iwila"..., "LANGUAGE=en_US:",
> "LC_TERMINAL_VERSION=3.3.7", "LOGNAME=root", "LC_CTYPE=UTF-8",
> "LESSOPEN=| /usr/bin/lesspipe %s", "PROMPT_COMMAND=EXIT_CODE=$? &&
> d"..., "SUDO_GID=1000", "LC_TERMINAL=iTerm2",
> "LESSCLOSE=/usr/bin/lesspipe %s %"..., "HISTTIMEFORMAT=%F %T ",
> "SSH_ID=RSA-SHA256:Ap2X89VM+tClGr"..., "_=/usr/bin/strace"]) = 0
> brk(NULL) = 0x1f98000
> access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory)
> access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or directory)
> open("/lib/ebtables/tls/x86_64/libebt_802_3.so", O_RDONLY|O_CLOEXEC) =
> -1 ENOENT (No such file or directory)
> stat("/lib/ebtables/tls/x86_64", 0x7ffe0d016c20) = -1 ENOENT (No such
> file or directory)
> open("/lib/ebtables/tls/libebt_802_3.so", O_RDONLY|O_CLOEXEC) = -1
> ENOENT (No such file or directory)
> stat("/lib/ebtables/tls", 0x7ffe0d016c20) = -1 ENOENT (No such file or
> directory)
> open("/lib/ebtables/x86_64/libebt_802_3.so", O_RDONLY|O_CLOEXEC) = -1
> ENOENT (No such file or directory)
> stat("/lib/ebtables/x86_64", 0x7ffe0d016c20) = -1 ENOENT (No such file
> or directory)
> open("/lib/ebtables/libebt_802_3.so", O_RDONLY|O_CLOEXEC) = 3
> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0@\7\0\0\0\0\0\0"...,
> 832) = 832
> fstat(3, {st_dev=makedev(259, 4), st_ino=87819095,
> st_mode=S_IFREG|0644, st_nlink=1, st_uid=0, st_gid=0, st_blksize=4096,
> st_blocks=24, st_size=10032, st_atime=2020/02/13-14:54:35.335391990,
> st_mtime=2018/06/29-03:17:40, st_ctime=2019/10/11-17:14:44.944338761})
> = 0
> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
> 0) = 0x7ff859fcc000
> mmap(NULL, 2105664, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3,
> 0) = 0x7ff859ba5000
> mprotect(0x7ff859ba6000, 2097152, PROT_NONE) = 0
> mmap(0x7ff859da6000, 8192, PROT_READ|PROT_WRITE,
> MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1000) = 0x7ff859da6000
> close(3) = 0
> open("/lib/ebtables/libebt_nat.so", O_RDONLY|O_CLOEXEC) = 3
> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0000\t\0\0\0\0\0\0"...,
> 832) = 832
> fstat(3, {st_dev=makedev(259, 4), st_ino=87819085,
> st_mode=S_IFREG|0644, st_nlink=1, st_uid=0, st_gid=0, st_blksize=4096,
> st_blocks=24, st_size=10456, st_atime=2020/02/13-14:54:35.335391990,
> st_mtime=2018/06/29-03:17:40, st_ctime=2019/10/11-17:14:44.944338761})
> = 0
> mmap(NULL, 2106024, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3,
> 0) = 0x7ff8599a2000
> mprotect(0x7ff8599a4000, 2093056, PROT_NONE) = 0
> mmap(0x7ff859ba3000, 8192, PROT_READ|PROT_WRITE,
> MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1000) = 0x7ff859ba3000
> close(3) = 0
> open("/lib/ebtables/libebt_arp.so", O_RDONLY|O_CLOEXEC) = 3
> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\20\f\0\0\0\0\0\0"...,
> 832) = 832
> fstat(3, {st_dev=makedev(259, 4), st_ino=87819109,
> st_mode=S_IFREG|0644, st_nlink=1, st_uid=0, st_gid=0, st_blksize=4096,
> st_blocks=32, st_size=14496, st_atime=2020/02/13-14:54:35.335391990,
> st_mtime=2018/06/29-03:17:40, st_ctime=2019/10/11-17:14:44.944338761})
> = 0
> mmap(NULL, 2110048, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3,
> 0) = 0x7ff85979e000
> mprotect(0x7ff8597a0000, 2097152, PROT_NONE) = 0
> mmap(0x7ff8599a0000, 8192, PROT_READ|PROT_WRITE,
> MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x2000) = 0x7ff8599a0000
> close(3) = 0
> open("/lib/ebtables/libebt_arpreply.so", O_RDONLY|O_CLOEXEC) = 3
> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\20\10\0\0\0\0\0\0"...,
> 832) = 832
> fstat(3, {st_dev=makedev(259, 4), st_ino=87819097,
> st_mode=S_IFREG|0644, st_nlink=1, st_uid=0, st_gid=0, st_blksize=4096,
> st_blocks=24, st_size=10136, st_atime=2020/02/13-14:54:35.335391990,
> st_mtime=2018/06/29-03:17:40, st_ctime=2019/10/11-17:14:44.944338761})
> = 0
> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
> 0) = 0x7ff859fcb000
> mmap(NULL, 2105704, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3,
> 0) = 0x7ff85959b000
> mprotect(0x7ff85959d000, 2093056, PROT_NONE) = 0
> mmap(0x7ff85979c000, 8192, PROT_READ|PROT_WRITE,
> MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1000) = 0x7ff85979c000
> close(3) = 0
> open("/lib/ebtables/libebt_ip.so", O_RDONLY|O_CLOEXEC) = 3
> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\300\n\0\0\0\0\0\0"...,
> 832) = 832
> fstat(3, {st_dev=makedev(259, 4), st_ino=87819079,
> st_mode=S_IFREG|0644, st_nlink=1, st_uid=0, st_gid=0, st_blksize=4096,
> st_blocks=24, st_size=10384, st_atime=2020/02/13-14:54:35.335391990,
> st_mtime=2018/06/29-03:17:40, st_ctime=2019/10/11-17:14:44.944338761})
> = 0
> mmap(NULL, 2106016, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3,
> 0) = 0x7ff859398000
> mprotect(0x7ff85939a000, 2093056, PROT_NONE) = 0
> mmap(0x7ff859599000, 8192, PROT_READ|PROT_WRITE,
> MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1000) = 0x7ff859599000
> close(3) = 0
> open("/lib/ebtables/libebt_ip6.so", O_RDONLY|O_CLOEXEC) = 3
> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\300\16\0\0\0\0\0\0"...,
> 832) = 832
> fstat(3, {st_dev=makedev(259, 4), st_ino=87819103,
> st_mode=S_IFREG|0644, st_nlink=1, st_uid=0, st_gid=0, st_blksize=4096,
> st_blocks=40, st_size=18304, st_atime=2020/02/13-14:54:35.335391990,
> st_mtime=2018/06/29-03:17:40, st_ctime=2019/10/11-17:14:44.944338761})
> = 0
> mmap(NULL, 2113856, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3,
> 0) = 0x7ff859193000
> mprotect(0x7ff859196000, 2097152, PROT_NONE) = 0
> mmap(0x7ff859396000, 8192, PROT_READ|PROT_WRITE,
> MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x3000) = 0x7ff859396000
> close(3) = 0
> open("/lib/ebtables/libebt_standard.so", O_RDONLY|O_CLOEXEC) = 3
> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\240\5\0\0\0\0\0\0"...,
> 832) = 832
> fstat(3, {st_dev=makedev(259, 4), st_ino=87819073,
> st_mode=S_IFREG|0644, st_nlink=1, st_uid=0, st_gid=0, st_blksize=4096,
> st_blocks=16, st_size=5808, st_atime=2020/02/13-14:54:35.339392028,
> st_mtime=2018/06/29-03:17:40, st_ctime=2019/10/11-17:14:44.944338761})
> = 0
> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
> 0) = 0x7ff859fca000
> mmap(NULL, 2101472, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3,
> 0) = 0x7ff858f91000
> mprotect(0x7ff858f92000, 2093056, PROT_NONE) = 0
> mmap(0x7ff859191000, 8192, PROT_READ|PROT_WRITE,
> MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0) = 0x7ff859191000
> close(3) = 0
> open("/lib/ebtables/libebt_log.so", O_RDONLY|O_CLOEXEC) = 3
> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\320\t\0\0\0\0\0\0"...,
> 832) = 832
> fstat(3, {st_dev=makedev(259, 4), st_ino=87819087,
> st_mode=S_IFREG|0644, st_nlink=1, st_uid=0, st_gid=0, st_blksize=4096,
> st_blocks=24, st_size=10272, st_atime=2020/02/13-14:54:35.339392028,
> st_mtime=2018/06/29-03:17:40, st_ctime=2019/10/11-17:14:44.944338761})
> = 0
> mmap(NULL, 2105824, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3,
> 0) = 0x7ff858d8e000
> mprotect(0x7ff858d90000, 2093056, PROT_NONE) = 0
> mmap(0x7ff858f8f000, 8192, PROT_READ|PROT_WRITE,
> MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1000) = 0x7ff858f8f000
> close(3) = 0
> open("/lib/ebtables/libebt_redirect.so", O_RDONLY|O_CLOEXEC) = 3
> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\20\7\0\0\0\0\0\0"...,
> 832) = 832
> fstat(3, {st_dev=makedev(259, 4), st_ino=87819071,
> st_mode=S_IFREG|0644, st_nlink=1, st_uid=0, st_gid=0, st_blksize=4096,
> st_blocks=16, st_size=5904, st_atime=2020/02/13-14:54:35.339392028,
> st_mtime=2018/06/29-03:17:40, st_ctime=2019/10/11-17:14:44.944338761})
> = 0
> mmap(NULL, 2101536, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3,
> 0) = 0x7ff858b8c000
> mprotect(0x7ff858b8d000, 2093056, PROT_NONE) = 0
> mmap(0x7ff858d8c000, 8192, PROT_READ|PROT_WRITE,
> MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0) = 0x7ff858d8c000
> close(3) = 0
> open("/lib/ebtables/libebt_vlan.so", O_RDONLY|O_CLOEXEC) = 3
> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0000\10\0\0\0\0\0\0"...,
> 832) = 832
> fstat(3, {st_dev=makedev(259, 4), st_ino=87819107,
> st_mode=S_IFREG|0644, st_nlink=1, st_uid=0, st_gid=0, st_blksize=4096,
> st_blocks=24, st_size=10168, st_atime=2020/02/13-14:54:35.339392028,
> st_mtime=2018/06/29-03:17:40, st_ctime=2019/10/11-17:14:44.944338761})
> = 0
> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
> 0) = 0x7ff859fc9000
> mmap(NULL, 2105736, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3,
> 0) = 0x7ff858989000
> mprotect(0x7ff85898b000, 2093056, PROT_NONE) = 0
> mmap(0x7ff858b8a000, 8192, PROT_READ|PROT_WRITE,
> MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1000) = 0x7ff858b8a000
> close(3) = 0
> open("/lib/ebtables/libebt_mark_m.so", O_RDONLY|O_CLOEXEC) = 3
> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0
> \7\0\0\0\0\0\0"..., 832) = 832
> fstat(3, {st_dev=makedev(259, 4), st_ino=87819099,
> st_mode=S_IFREG|0644, st_nlink=1, st_uid=0, st_gid=0, st_blksize=4096,
> st_blocks=16, st_size=5904, st_atime=2020/02/13-14:54:35.339392028,
> st_mtime=2018/06/29-03:17:40, st_ctime=2019/10/11-17:14:44.944338761})
> = 0
> mmap(NULL, 2101536, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3,
> 0) = 0x7ff858787000
> mprotect(0x7ff858788000, 2093056, PROT_NONE) = 0
> mmap(0x7ff858987000, 8192, PROT_READ|PROT_WRITE,
> MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0) = 0x7ff858987000
> close(3) = 0
> open("/lib/ebtables/libebt_mark.so", O_RDONLY|O_CLOEXEC) = 3
> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\320\7\0\0\0\0\0\0"...,
> 832) = 832
> fstat(3, {st_dev=makedev(259, 4), st_ino=87819111,
> st_mode=S_IFREG|0644, st_nlink=1, st_uid=0, st_gid=0, st_blksize=4096,
> st_blocks=24, st_size=10232, st_atime=2020/02/13-14:54:35.339392028,
> st_mtime=2018/06/29-03:17:40, st_ctime=2019/10/11-17:14:44.944338761})
> = 0
> mmap(NULL, 2105800, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3,
> 0) = 0x7ff858584000
> mprotect(0x7ff858586000, 2093056, PROT_NONE) = 0
> mmap(0x7ff858785000, 8192, PROT_READ|PROT_WRITE,
> MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1000) = 0x7ff858785000
> close(3) = 0
> open("/lib/ebtables/libebt_pkttype.so", O_RDONLY|O_CLOEXEC) = 3
> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0p\10\0\0\0\0\0\0"...,
> 832) = 832
> fstat(3, {st_dev=makedev(259, 4), st_ino=87819093,
> st_mode=S_IFREG|0644, st_nlink=1, st_uid=0, st_gid=0, st_blksize=4096,
> st_blocks=24, st_size=10064, st_atime=2020/02/13-14:54:35.339392028,
> st_mtime=2018/06/29-03:17:40, st_ctime=2019/10/11-17:14:44.944338761})
> = 0
> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
> 0) = 0x7ff859fc8000
> mmap(NULL, 2105696, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3,
> 0) = 0x7ff858381000
> mprotect(0x7ff858382000, 2097152, PROT_NONE) = 0
> mmap(0x7ff858582000, 8192, PROT_READ|PROT_WRITE,
> MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1000) = 0x7ff858582000
> close(3) = 0
> open("/lib/ebtables/libebt_stp.so", O_RDONLY|O_CLOEXEC) = 3
> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\320\n\0\0\0\0\0\0"...,
> 832) = 832
> fstat(3, {st_dev=makedev(259, 4), st_ino=87819113,
> st_mode=S_IFREG|0644, st_nlink=1, st_uid=0, st_gid=0, st_blksize=4096,
> st_blocks=32, st_size=14512, st_atime=2020/02/13-14:54:35.339392028,
> st_mtime=2018/06/29-03:17:40, st_ctime=2019/10/11-17:14:44.944338761})
> = 0
> mmap(NULL, 2110144, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3,
> 0) = 0x7ff85817d000
> mprotect(0x7ff858180000, 2093056, PROT_NONE) = 0
> mmap(0x7ff85837f000, 8192, PROT_READ|PROT_WRITE,
> MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x2000) = 0x7ff85837f000
> close(3) = 0
> open("/lib/ebtables/libebt_among.so", O_RDONLY|O_CLOEXEC) = 3
> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\320\v\0\0\0\0\0\0"...,
> 832) = 832
> fstat(3, {st_dev=makedev(259, 4), st_ino=87819075,
> st_mode=S_IFREG|0644, st_nlink=1, st_uid=0, st_gid=0, st_blksize=4096,
> st_blocks=32, st_size=14392, st_atime=2020/02/13-14:54:35.339392028,
> st_mtime=2018/06/29-03:17:40, st_ctime=2019/10/11-17:14:44.944338761})
> = 0
> mmap(NULL, 2109960, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3,
> 0) = 0x7ff857f79000
> mprotect(0x7ff857f7c000, 2093056, PROT_NONE) = 0
> mmap(0x7ff85817b000, 8192, PROT_READ|PROT_WRITE,
> MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x2000) = 0x7ff85817b000
> close(3) = 0
> open("/lib/ebtables/libebt_limit.so", O_RDONLY|O_CLOEXEC) = 3
> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\200\10\0\0\0\0\0\0"...,
> 832) = 832
> fstat(3, {st_dev=makedev(259, 4), st_ino=87819105,
> st_mode=S_IFREG|0644, st_nlink=1, st_uid=0, st_gid=0, st_blksize=4096,
> st_blocks=24, st_size=10144, st_atime=2020/02/13-14:54:35.339392028,
> st_mtime=2018/06/29-03:17:40, st_ctime=2019/10/11-17:14:44.944338761})
> = 0
> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
> 0) = 0x7ff859fc7000
> mmap(NULL, 2105696, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3,
> 0) = 0x7ff857d76000
> mprotect(0x7ff857d78000, 2093056, PROT_NONE) = 0
> mmap(0x7ff857f77000, 8192, PROT_READ|PROT_WRITE,
> MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1000) = 0x7ff857f77000
> close(3) = 0
> open("/lib/ebtables/libebt_ulog.so", O_RDONLY|O_CLOEXEC) = 3
> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0`\10\0\0\0\0\0\0"...,
> 832) = 832
> fstat(3, {st_dev=makedev(259, 4), st_ino=87819083,
> st_mode=S_IFREG|0644, st_nlink=1, st_uid=0, st_gid=0, st_blksize=4096,
> st_blocks=24, st_size=10160, st_atime=2020/02/13-14:54:35.339392028,
> st_mtime=2018/06/29-03:17:40, st_ctime=2019/10/11-17:14:44.944338761})
> = 0
> mmap(NULL, 2105792, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3,
> 0) = 0x7ff857b73000
> mprotect(0x7ff857b75000, 2093056, PROT_NONE) = 0
> mmap(0x7ff857d74000, 8192, PROT_READ|PROT_WRITE,
> MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1000) = 0x7ff857d74000
> close(3) = 0
> open("/lib/ebtables/libebt_nflog.so", O_RDONLY|O_CLOEXEC) = 3
> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0`\10\0\0\0\0\0\0"...,
> 832) = 832
> fstat(3, {st_dev=makedev(259, 4), st_ino=87819081,
> st_mode=S_IFREG|0644, st_nlink=1, st_uid=0, st_gid=0, st_blksize=4096,
> st_blocks=24, st_size=10160, st_atime=2020/02/13-14:54:35.343392065,
> st_mtime=2018/06/29-03:17:40, st_ctime=2019/10/11-17:14:44.944338761})
> = 0
> mmap(NULL, 2105792, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3,
> 0) = 0x7ff857970000
> mprotect(0x7ff857972000, 2093056, PROT_NONE) = 0
> mmap(0x7ff857b71000, 8192, PROT_READ|PROT_WRITE,
> MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1000) = 0x7ff857b71000
> close(3) = 0
> open("/lib/ebtables/libebtable_filter.so", O_RDONLY|O_CLOEXEC) = 3
> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\0\5\0\0\0\0\0\0"...,
> 832) = 832
> fstat(3, {st_dev=makedev(259, 4), st_ino=87819091,
> st_mode=S_IFREG|0644, st_nlink=1, st_uid=0, st_gid=0, st_blksize=4096,
> st_blocks=16, st_size=5672, st_atime=2020/02/13-14:54:35.343392065,
> st_mtime=2018/06/29-03:17:40, st_ctime=2019/10/11-17:14:44.944338761})
> = 0
> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
> 0) = 0x7ff859fc6000
> mmap(NULL, 2101368, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3,
> 0) = 0x7ff85776e000
> mprotect(0x7ff85776f000, 2093056, PROT_NONE) = 0
> mmap(0x7ff85796e000, 8192, PROT_READ|PROT_WRITE,
> MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0) = 0x7ff85796e000
> close(3) = 0
> open("/lib/ebtables/libebtable_nat.so", O_RDONLY|O_CLOEXEC) = 3
> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\0\5\0\0\0\0\0\0"...,
> 832) = 832
> fstat(3, {st_dev=makedev(259, 4), st_ino=87819101,
> st_mode=S_IFREG|0644, st_nlink=1, st_uid=0, st_gid=0, st_blksize=4096,
> st_blocks=16, st_size=5672, st_atime=2020/02/13-14:54:35.343392065,
> st_mtime=2018/06/29-03:17:40, st_ctime=2019/10/11-17:14:44.944338761})
> = 0
> mmap(NULL, 2101368, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3,
> 0) = 0x7ff85756c000
> mprotect(0x7ff85756d000, 2093056, PROT_NONE) = 0
> mmap(0x7ff85776c000, 8192, PROT_READ|PROT_WRITE,
> MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0) = 0x7ff85776c000
> close(3) = 0
> open("/lib/ebtables/libebtable_broute.so", O_RDONLY|O_CLOEXEC) = 3
> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0P\4\0\0\0\0\0\0"...,
> 832) = 832
> fstat(3, {st_dev=makedev(259, 4), st_ino=87819077,
> st_mode=S_IFREG|0644, st_nlink=1, st_uid=0, st_gid=0, st_blksize=4096,
> st_blocks=16, st_size=5672, st_atime=2020/02/13-14:54:35.343392065,
> st_mtime=2018/06/29-03:17:40, st_ctime=2019/10/11-17:14:44.944338761})
> = 0
> mmap(NULL, 2101368, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3,
> 0) = 0x7ff85736a000
> mprotect(0x7ff85736b000, 2093056, PROT_NONE) = 0
> mmap(0x7ff85756a000, 8192, PROT_READ|PROT_WRITE,
> MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0) = 0x7ff85756a000
> close(3) = 0
> open("/lib/ebtables/libebtc.so", O_RDONLY|O_CLOEXEC) = 3
> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0
> 0\0\0\0\0\0\0"..., 832) = 832
> fstat(3, {st_dev=makedev(259, 4), st_ino=87819089,
> st_mode=S_IFREG|0644, st_nlink=1, st_uid=0, st_gid=0, st_blksize=4096,
> st_blocks=136, st_size=69536, st_atime=2020/02/13-14:54:35.343392065,
> st_mtime=2018/06/29-03:17:40, st_ctime=2019/10/11-17:14:44.944338761})
> = 0
> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
> 0) = 0x7ff859fc5000
> mmap(NULL, 2173680, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3,
> 0) = 0x7ff857157000
> mprotect(0x7ff857167000, 2093056, PROT_NONE) = 0
> mmap(0x7ff857366000, 8192, PROT_READ|PROT_WRITE,
> MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0xf000) = 0x7ff857366000
> mmap(0x7ff857368000, 6896, PROT_READ|PROT_WRITE,
> MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7ff857368000
> close(3) = 0
> open("/lib/ebtables/libc.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No
> such file or directory)
> open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
> fstat(3, {st_dev=makedev(259, 4), st_ino=18352138,
> st_mode=S_IFREG|0644, st_nlink=1, st_uid=0, st_gid=0, st_blksize=4096,
> st_blocks=88, st_size=42101, st_atime=2020/02/13-14:39:47.058940019,
> st_mtime=2020/02/13-14:39:47.054939981,
> st_ctime=2020/02/13-14:39:47.054939981}) = 0
> mmap(NULL, 42101, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7ff859fba000
> close(3) = 0
> access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory)
> open("/lib/x86_64-linux-gnu/libc.so.6", O_RDONLY|O_CLOEXEC) = 3
> read(3, "\177ELF\2\1\1\3\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0P\t\2\0\0\0\0\0"...,
> 832) = 832
> fstat(3, {st_dev=makedev(259, 4), st_ino=87818433,
> st_mode=S_IFREG|0755, st_nlink=1, st_uid=0, st_gid=0, st_blksize=4096,
> st_blocks=3656, st_size=1868984,
> st_atime=2020/02/12-15:53:36.963494808, st_mtime=2019/02/06-05:11:22,
> st_ctime=2019/10/10-15:50:21.797152380}) = 0
> mmap(NULL, 3971488, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3,
> 0) = 0x7ff856d8d000
> mprotect(0x7ff856f4d000, 2097152, PROT_NONE) = 0
> mmap(0x7ff85714d000, 24576, PROT_READ|PROT_WRITE,
> MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1c0000) = 0x7ff85714d000
> mmap(0x7ff857153000, 14752, PROT_READ|PROT_WRITE,
> MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7ff857153000
> close(3) = 0
> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
> 0) = 0x7ff859fb9000
> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
> 0) = 0x7ff859fb8000
> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
> 0) = 0x7ff859fb7000
> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
> 0) = 0x7ff859fb6000
> arch_prctl(ARCH_SET_FS, 0x7ff859fb7700) = 0
> mprotect(0x7ff85714d000, 16384, PROT_READ) = 0
> mprotect(0x7ff857366000, 4096, PROT_READ) = 0
> mprotect(0x7ff85756a000, 4096, PROT_READ) = 0
> mprotect(0x7ff85776c000, 4096, PROT_READ) = 0
> mprotect(0x7ff85796e000, 4096, PROT_READ) = 0
> mprotect(0x7ff857b71000, 4096, PROT_READ) = 0
> mprotect(0x7ff857d74000, 4096, PROT_READ) = 0
> mprotect(0x7ff857f77000, 4096, PROT_READ) = 0
> mprotect(0x7ff85817b000, 4096, PROT_READ) = 0
> mprotect(0x7ff85837f000, 4096, PROT_READ) = 0
> mprotect(0x7ff858582000, 4096, PROT_READ) = 0
> mprotect(0x7ff858785000, 4096, PROT_READ) = 0
> mprotect(0x7ff858987000, 4096, PROT_READ) = 0
> mprotect(0x7ff858b8a000, 4096, PROT_READ) = 0
> mprotect(0x7ff858d8c000, 4096, PROT_READ) = 0
> mprotect(0x7ff858f8f000, 4096, PROT_READ) = 0
> mprotect(0x7ff859191000, 4096, PROT_READ) = 0
> mprotect(0x7ff859396000, 4096, PROT_READ) = 0
> mprotect(0x7ff859599000, 4096, PROT_READ) = 0
> mprotect(0x7ff85979c000, 4096, PROT_READ) = 0
> mprotect(0x7ff8599a0000, 4096, PROT_READ) = 0
> mprotect(0x7ff859ba3000, 4096, PROT_READ) = 0
> mprotect(0x7ff859da6000, 4096, PROT_READ) = 0
> mprotect(0x600000, 4096, PROT_READ) = 0
> mprotect(0x7ff859fcd000, 4096, PROT_READ) = 0
> munmap(0x7ff859fba000, 42101) = 0
> brk(NULL) = 0x1f98000
> brk(0x1fb9000) = 0x1fb9000
> rt_sigaction(SIGINT, {0x7ff85715f8d0, [INT], SA_RESTORER|SA_RESTART,
> 0x7ff856dc24b0}, {SIG_DFL, [], 0}, 8) = 0
> rt_sigaction(SIGTERM, {0x7ff85715f8d0, [TERM], SA_RESTORER|SA_RESTART,
> 0x7ff856dc24b0}, {SIG_DFL, [], 0}, 8) = 0
> open("/run/ebtables.lock", O_WRONLY|O_CREAT, 0600) = 3
> fcntl(3, F_SETLK, {l_type=F_WRLCK, l_whence=SEEK_SET, l_start=0, l_len=0}) = 0
> socket(PF_INET, SOCK_RAW, IPPROTO_IGMP) = 4
> getsockopt(4, SOL_IP, EBT_SO_GET_INFO, packet_write_wait: Connection
> to 10.91.13.28 port 22: Broken pipe
>
>
>
> And bug found which likely to affect this case. (ebtables: fix race
> condition in frame_filter_net_init():
> https://lore.kernel.org/patchwork/patch/834743/). I checked our kernel
> (4.13.0-41-generic) is running without the patch and I suspect the bug
> makes kernel crush.
>
> But I cannot replay the bug with some script (while test 1; do rmmod
> ebtable_nat; rmmod ebtable_filter; ebtables -t nat -L; done)
>
> Does anybody let me know how to reproduce the bug?
>
> 2020년 2월 12일 (수) 오전 11:37, 양유석 <ileixe@gmail.com>님이 작성:
> >
> > I wish I can check rsyslog first. It do have the last command (etables
> > --concurrent -t nat -L). But unfortunately, there is no more command.
> > I will try to use netconsole which seems to log for kernel.
> >
> > 2020년 2월 11일 (화) 오후 11:15, G.W. Haywood <ged@jubileegroup.co.uk>님이 작성:
> > >
> > > Hi there,
> > >
> > > On Tue, 11 Feb 2020, 양유석 wrote:
> > >
> > > > 2020년 2월 11일 (화) 오후 9:47, G.W. Haywood ...
> > > >> ... Mr. Millar's suggestion is good too.
> > > >
> > > > ... where is Mr.Millar's suggestion? ...
> > >
> > > https://marc.info/?l=netfilter&m=158142378531784&w=2
> > >
> > > --
> > >
> > > 73,
> > > Ged.
^ permalink raw reply [flat|nested] 11+ messages in thread
end of thread, other threads:[~2020-02-13 9:08 UTC | newest]
Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-02-11 11:57 Does anybody experience kernel crush when 'ebtable -t nat -L' 양유석
2020-02-11 12:03 ` G.W. Haywood
2020-02-11 12:07 ` 양유석
2020-02-11 12:23 ` kfm
2020-02-11 12:47 ` G.W. Haywood
2020-02-11 13:29 ` 양유석
2020-02-11 14:15 ` G.W. Haywood
2020-02-12 2:37 ` 양유석
2020-02-13 8:25 ` Youseok Yang
2020-02-13 9:08 ` Youseok Yang
2020-02-11 14:00 ` 양유석
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.