Backlog not working with kernel 3.10

Backlog not working with kernel 3.10

[Alan Evangelista]

[-- Attachment #1.1: Type: text/plain, Size: 1231 bytes --]

AFAIK, the purpose of the backlog (a queue of audit events in the kernel)
is to assure no events are lost when events are generated at a faster speed
than they are consumed.

I'm using CentOS7 with kernel 3.10.0-1160.15.2.el7.x86_64 and trying to
test the backlog, but it seems it's not working at all.

Audit rule:
-a always,exit -F dir=/sasdata -F arch=b64 -S creat -S open -S openat -S
unlink -S unlinkat -S symlink -S symlinkat -S link -S linkat -S rename -S
renameat -S chmod -S fchmod -S fchmodat -S chown -S fchown -S fchownat -S
mkdir -S mkdirat -S rmdir -S setxattr -S lsetxattr -S fsetxattr -S
removexattr -S lremovexattr -S fremovexattr -k filesystem_op

First I turned auditd off so that events are not consumed:

# service stop auditd

Then I make sure that the backlog size is greater than 0:

# auditctl -s
enabled 1
failure 1
pid 0
rate_limit 5000
backlog_limit 8192
lost 0
backlog 0
loginuid_immutable 0 unlocked

I have run some simple commands in /data that  should be logged , e.g.
touch file, mkdir dir. Finally, I have run auditctl-s and expected to see
the backlog events counter go up, but it's still 0. If I start auditd
again, the events are never logged. Am I missing something here?

Thanks in advance.

[-- Attachment #1.2: Type: text/html, Size: 1507 bytes --]

[-- Attachment #2: Type: text/plain, Size: 106 bytes --]

--
Linux-audit mailing list
Linux-audit@redhat.com
https://listman.redhat.com/mailman/listinfo/linux-audit

Re: Backlog not working with kernel 3.10

[Paul Moore]

On Tue, Mar 16, 2021 at 5:26 PM Alan Evangelista <alan.vitor@gmail.com> wrote:
>
> AFAIK, the purpose of the backlog (a queue of audit events in the kernel) is to assure no events are lost when events are generated at a faster speed than they are consumed.
>
> I'm using CentOS7 with kernel 3.10.0-1160.15.2.el7.x86_64 and trying to test the backlog, but it seems it's not working at all.
>
> Audit rule:
> -a always,exit -F dir=/sasdata -F arch=b64 -S creat -S open -S openat -S unlink -S unlinkat -S symlink -S symlinkat -S link -S linkat -S rename -S renameat -S chmod -S fchmod -S fchmodat -S chown -S fchown -S fchownat -S mkdir -S mkdirat -S rmdir -S setxattr -S lsetxattr -S fsetxattr -S removexattr -S lremovexattr -S fremovexattr -k filesystem_op
>
> First I turned auditd off so that events are not consumed:
>
> # service stop auditd
>
> Then I make sure that the backlog size is greater than 0:
>
> # auditctl -s
> enabled 1
> failure 1
> pid 0
> rate_limit 5000
> backlog_limit 8192
> lost 0
> backlog 0
> loginuid_immutable 0 unlocked
>
> I have run some simple commands in /data that  should be logged , e.g. touch file, mkdir dir. Finally, I have run auditctl-s and expected to see the backlog events counter go up, but it's still 0. If I start auditd again, the events are never logged. Am I missing something here?
>
> Thanks in advance.

The audit queue mechanism (backlog) was pretty messed up in older
kernels, and while we've fixed it in modern kernels, I believe that
not all of the changes have been backported to the older distribution
kernels.  If you are a RHEL customer you *may* have some luck pursuing
this via your RH/IBM support person, but I can't say for certain (I
don't work for RH/IBM).

>From an upstream perspective - which is what this mailing list focuses
on - there isn't much for us to do here unless you are seeing problems
with a more current kernel.

-- 
paul moore
www.paul-moore.com


--
Linux-audit mailing list
Linux-audit@redhat.com
https://listman.redhat.com/mailman/listinfo/linux-audit

Re: Backlog not working with kernel 3.10

[Richard Guy Briggs]

On 2021-03-16 18:25, Alan Evangelista wrote:
> AFAIK, the purpose of the backlog (a queue of audit events in the kernel)
> is to assure no events are lost when events are generated at a faster speed
> than they are consumed.
> 
> I'm using CentOS7 with kernel 3.10.0-1160.15.2.el7.x86_64 and trying to
> test the backlog, but it seems it's not working at all.
> 
> Audit rule:
> -a always,exit -F dir=/sasdata -F arch=b64 -S creat -S open -S openat -S
> unlink -S unlinkat -S symlink -S symlinkat -S link -S linkat -S rename -S
> renameat -S chmod -S fchmod -S fchmodat -S chown -S fchown -S fchownat -S
> mkdir -S mkdirat -S rmdir -S setxattr -S lsetxattr -S fsetxattr -S
> removexattr -S lremovexattr -S fremovexattr -k filesystem_op
> 
> First I turned auditd off so that events are not consumed:
> 
> # service stop auditd
> 
> Then I make sure that the backlog size is greater than 0:
> 
> # auditctl -s
> enabled 1
> failure 1
> pid 0
> rate_limit 5000
> backlog_limit 8192
> lost 0
> backlog 0
> loginuid_immutable 0 unlocked
> 
> I have run some simple commands in /data that  should be logged , e.g.
> touch file, mkdir dir. Finally, I have run auditctl-s and expected to see
> the backlog events counter go up, but it's still 0. If I start auditd
> again, the events are never logged. Am I missing something here?

So, since you haven't indicated if you have tried and tested this
already, please start by running those simple commands while the auditd
service is running and verifying that those commands do get logged as
expected.  If they don't, fix that first.

- RGB

--
Richard Guy Briggs <rgb@redhat.com>
Sr. S/W Engineer, Kernel Security, Base Operating Systems
Remote, Ottawa, Red Hat Canada
IRC: rgb, SunRaycer
Voice: +1.647.777.2635, Internal: (81) 32635

--
Linux-audit mailing list
Linux-audit@redhat.com
https://listman.redhat.com/mailman/listinfo/linux-audit

Fwd: Backlog not working with kernel 3.10

[Alan Evangelista]

[-- Attachment #1.1: Type: text/plain, Size: 336 bytes --]

RB> since you haven't indicated if you have tried and
    >  tested this already, please start by running those
    > simple commands while the auditd service is running
    > and verifying that those commands do get logged as
    > expected.  If they don't, fix that first.

The commands are logged when the auditd service is running.

[-- Attachment #1.2: Type: text/html, Size: 902 bytes --]

[-- Attachment #2: Type: text/plain, Size: 106 bytes --]

--
Linux-audit mailing list
Linux-audit@redhat.com
https://listman.redhat.com/mailman/listinfo/linux-audit

Re: Backlog not working with kernel 3.10

[Alan Evangelista]

[-- Attachment #1.1: Type: text/plain, Size: 462 bytes --]

PM> The audit queue mechanism (backlog) was pretty messed up in older
kernels, and while we've fixed it in modern kernels, I believe that
not all of the changes have been backported to the older distribution
kernels.

Do you know in which kernel version this mechanism was fixed?
This information would help me decide which distro I should move to.

PM> If you are a RHEL customer you *may* have some luck pursuing
this via your RH/IBM support person

I am not.

[-- Attachment #1.2: Type: text/html, Size: 612 bytes --]

[-- Attachment #2: Type: text/plain, Size: 106 bytes --]

--
Linux-audit mailing list
Linux-audit@redhat.com
https://listman.redhat.com/mailman/listinfo/linux-audit

Re: Backlog not working with kernel 3.10

[Lenny Bruzenak]

[-- Attachment #1.1: Type: text/plain, Size: 976 bytes --]

On 3/16/21 8:46 PM, Richard Guy Briggs wrote:

>> I have run some simple commands in /data that  should be logged , e.g.
>> touch file, mkdir dir. Finally, I have run auditctl-s and expected to see
>> the backlog events counter go up, but it's still 0. If I start auditd
>> again, the events are never logged. Am I missing something here?
> So, since you haven't indicated if you have tried and tested this
> already, please start by running those simple commands while the auditd
> service is running and verifying that those commands do get logged as
> expected.  If they don't, fix that first.

I was wondering if the events are delivered to syslog
(/var/log/messages) instead while the auditd is down?

Mine are, same kernel version 3.10.0. From the kernel perspective, no
backlog?. However, if I stop both audit and rsyslog, add some events the
backlog count doesn't increase and I can't see where the events may have
been delivered.

LCB

-- 
Lenny Bruzenak
MagitekLTD


[-- Attachment #1.2: Type: text/html, Size: 1510 bytes --]

[-- Attachment #2: Type: text/plain, Size: 106 bytes --]

--
Linux-audit mailing list
Linux-audit@redhat.com
https://listman.redhat.com/mailman/listinfo/linux-audit

Re: Backlog not working with kernel 3.10

[Richard Guy Briggs]

On 2021-03-16 18:25, Alan Evangelista wrote:
> I'm using CentOS7 with kernel 3.10.0-1160.15.2.el7.x86_64 and trying to
> test the backlog, but it seems it's not working at all.

Which minor version of CentOS7 is this?


- RGB

--
Richard Guy Briggs <rgb@redhat.com>
Sr. S/W Engineer, Kernel Security, Base Operating Systems
Remote, Ottawa, Red Hat Canada
IRC: rgb, SunRaycer
Voice: +1.647.777.2635, Internal: (81) 32635

--
Linux-audit mailing list
Linux-audit@redhat.com
https://listman.redhat.com/mailman/listinfo/linux-audit

Re: Backlog not working with kernel 3.10

[Richard Guy Briggs]

On 2021-03-17 09:32, Lenny Bruzenak wrote:
> On 3/16/21 8:46 PM, Richard Guy Briggs wrote:
> 
> >> I have run some simple commands in /data that  should be logged , e.g.
> >> touch file, mkdir dir. Finally, I have run auditctl-s and expected to see
> >> the backlog events counter go up, but it's still 0. If I start auditd
> >> again, the events are never logged. Am I missing something here?
> > So, since you haven't indicated if you have tried and tested this
> > already, please start by running those simple commands while the auditd
> > service is running and verifying that those commands do get logged as
> > expected.  If they don't, fix that first.
> 
> I was wondering if the events are delivered to syslog
> (/var/log/messages) instead while the auditd is down?
> 
> Mine are, same kernel version 3.10.0. From the kernel perspective, no
> backlog?. However, if I stop both audit and rsyslog, add some events the
> backlog count doesn't increase and I can't see where the events may have
> been delivered.

If audit is enabled, but auditd isn't registered, it should fill the
backlog since rsyslog and journald aren't considered reliable delivery
even if those messages appear in the latter two.

> LCB

- RGB

--
Richard Guy Briggs <rgb@redhat.com>
Sr. S/W Engineer, Kernel Security, Base Operating Systems
Remote, Ottawa, Red Hat Canada
IRC: rgb, SunRaycer
Voice: +1.647.777.2635, Internal: (81) 32635

--
Linux-audit mailing list
Linux-audit@redhat.com
https://listman.redhat.com/mailman/listinfo/linux-audit

Re: Backlog not working with kernel 3.10

[Paul Moore]

On Wed, Mar 17, 2021 at 4:41 AM Alan Evangelista <alan.vitor@gmail.com> wrote:
> PM> The audit queue mechanism (backlog) was pretty messed up in older
> kernels, and while we've fixed it in modern kernels, I believe that
> not all of the changes have been backported to the older distribution
> kernels.
>
> Do you know in which kernel version this mechanism was fixed?
> This information would help me decide which distro I should move to.

Ignoring any backporting that may have taken place, I believe all of
the critical patches were present in Linux v4.13 with a few
nice-to-have patches in v4.15.

-- 
paul moore
www.paul-moore.com

--
Linux-audit mailing list
Linux-audit@redhat.com
https://listman.redhat.com/mailman/listinfo/linux-audit

Re: Backlog not working with kernel 3.10

[Alan Evangelista]

[-- Attachment #1.1: Type: text/plain, Size: 219 bytes --]

AV> I'm using CentOS7 with kernel 3.10.0-1160.15.2.el7.x86_64 and trying to
    > test the backlog, but it seems it's not working at all.

RB> Which minor version of CentOS7 is this?

7.9.2009 (latest CentOS 7 version)

[-- Attachment #1.2: Type: text/html, Size: 295 bytes --]

[-- Attachment #2: Type: text/plain, Size: 106 bytes --]

--
Linux-audit mailing list
Linux-audit@redhat.com
https://listman.redhat.com/mailman/listinfo/linux-audit

Re: Backlog not working with kernel 3.10

[Richard Guy Briggs]

On 2021-03-16 18:25, Alan Evangelista wrote:
> I'm using CentOS7 with kernel 3.10.0-1160.15.2.el7.x86_64 and trying to
> test the backlog, but it seems it's not working at all.

> First I turned auditd off so that events are not consumed:
> # service stop auditd
> 
> Then I make sure that the backlog size is greater than 0:
> # auditctl -s
> enabled 1
> failure 1
> pid 0
> backlog_limit 8192
> lost 0
> backlog 0

This is a bit of a long shot, and I note the "enabled 1" while "pid 0"
above, but have you got "audit=1" in the kernel boot parameters?  If
not, what happens if you add it?

- RGB

--
Richard Guy Briggs <rgb@redhat.com>
Sr. S/W Engineer, Kernel Security, Base Operating Systems
Remote, Ottawa, Red Hat Canada
IRC: rgb, SunRaycer
Voice: +1.647.777.2635, Internal: (81) 32635

--
Linux-audit mailing list
Linux-audit@redhat.com
https://listman.redhat.com/mailman/listinfo/linux-audit