linux-audit.redhat.com archive mirror
 help / color / mirror / Atom feed
* Fw:Re:[RFC,v1,1/1] audit: speed up syscall rule match while exiting syscall
@ 2021-01-14 13:25 yang.yang29
  2021-01-14 15:22 ` Paul Moore
  0 siblings, 1 reply; 2+ messages in thread
From: yang.yang29 @ 2021-01-14 13:25 UTC (permalink / raw)
  To: paul; +Cc: linux-audit, linux-kernel


[-- Attachment #1.1: Type: text/plain, Size: 3979 bytes --]

Performance measurements:
1.Environment
CPU: Intel(R) Core(TM) i7-6700 CPU @ 3.40GHz
Linux kernel version: 5.11-rc3
Audit version: 2.8.4

2.Result
2.1 Syscall invocations
Test method: 
    Run command "top" with no-load.
    Add rule likes "auditctl -a always,exit -F arch=b64 -S chmod -F auid=[number]" which doesn't hit audit.
    User command "perf record -Rg -t [top's pid] sleep 900" to get audit_filter_syscall()'s execute time ratio.

audit_filter_syscall() ratio with 100 rules:
    before this patch: 15.29%.
    after this patch: 0.88%, reduce 14.41%.
audit_filter_syscall() ratio with CIS[1] rules:
    before this patch: 2.25%.
    after this patch: 1.93%%, reduce 0.32%.
audit_filter_syscall() ratio with 10 rules:
    before this patch: 0.94%.
    after this patch: 1.02%, increase 0.08%.
audit_filter_syscall() ratio with 1 rule:
    before this patch: 0.20%.
    after this patch: 0.88%, increase 0.68%.

Analyse:
    With 1 rule, after this patch performance is worse, because mutex_lock()/mutex_unlock(). But user just add one rule seems unusual.
    With more rule, after this patch performance is improved.Typical likes CIS benchmark.

2.2 Rule change
Test method: 
    Use ktime_get_real_ts64() before and after audit_add_rule()/audit_del_rule() to calculate time.
     Add/delete rule by command "auditctl". Each test 10times and get average.

audit_add_rule() time:
    before this patch: 3120ns.
    after this patch: 7783ns, increase 149%.
audit_del_rule() time:
    before this patch: 3510ns.
    after this patch: 8519ns, increase 143%.

Analyse:
    After this patch, rule change time obviously increase. But rule change may not happen very often.

[1] CIS is a Linux Benchmarks for security purpose.
https://www.cisecurity.org/benchmark/distribution_independent_linux/

------------------Original Mail------------------
Sender: PaulMoore
To: yang yang10192021;
CC: Eric Paris;linux-audit@redhat.com;linux-kernel@vger.kernel.org;
Date: 2021/01/13 21:48
Subject: Re:[RFC,v1,1/1] audit: speed up syscall rule match while exiting syscall
On Wed, Jan 13, 2021 at 7:39 AM <yang.yang29@zte.com.cn> wrote:
> From 82ebcf43481be21ee3e32ec1749b42f651737880 Mon Sep 17 00:00:00 2001
> From: Yang Yang <yang.yang29@zte.com.cn>
> Date: Wed, 13 Jan 2021 20:18:04 +0800
> Subject: [PATCH] [RFC,v1,1/1] speed up syscall rule match while exiting syscall
>  If user add any syscall rule, in all syscalls, audit_filter_syscall()
>  traverses struct list_head audit_filter_list to find out whether current
>  syscall match one rule. This takes o(n), which is not necessary, specially
>  for user who add a very few syscall rules. On the other hand, user may not
>  much care about rule add/delete speed. So do o(n) calculate at rule changing,
>  and ease the burden of audit_filter_syscall().
>
>  Define audit_syscall[NR_syscalls], every element stands for one syscall.
>  audit_filter_syscall() checks audit_syscall[NR_syscalls].
>  audit_syscall[n] == 0 indicates no rule audit syscall n, do a quick exit.
>  audit_syscall[n] > 0 indicates at least one rule audit syscall n.
>  audit_syscall[n] update when syscall rule changes.
>
> Signed-off-by: Yang Yang <yang.yang29@zte.com.cn>
> ---
>  include/linux/audit.h |  2 ++
>  kernel/audit.c        |  2 ++
>  kernel/auditfilter.c  | 16 ++++++++++++++++
>  kernel/auditsc.c      |  9 ++++++++-
>  4 files changed, 28 insertions(+), 1 deletion(-)

Before we go too far into a review of this patch, please provide some
performance measurements using a variety of rule counts, both common
and extreme, so that we can better judge the benefits of this patch.
The measurements should include both the rule add/delete time deltas
as well as the impact on the syscall invocations.  If non-obvious,
please also include how you performed the measurements and captured
the data.

These are good things to include in the commit description when
submitting patches focused on improving performance.

--
paul moore
www.paul-moore.com

[-- Attachment #2: Type: text/plain, Size: 102 bytes --]

--
Linux-audit mailing list
Linux-audit@redhat.com
https://www.redhat.com/mailman/listinfo/linux-audit

^ permalink raw reply	[flat|nested] 2+ messages in thread

* Re: Fw:Re:[RFC,v1,1/1] audit: speed up syscall rule match while exiting syscall
  2021-01-14 13:25 Fw:Re:[RFC,v1,1/1] audit: speed up syscall rule match while exiting syscall yang.yang29
@ 2021-01-14 15:22 ` Paul Moore
  0 siblings, 0 replies; 2+ messages in thread
From: Paul Moore @ 2021-01-14 15:22 UTC (permalink / raw)
  To: yang.yang29; +Cc: linux-audit, linux-kernel

On Thu, Jan 14, 2021 at 8:25 AM <yang.yang29@zte.com.cn> wrote:
>
> Performance measurements:
> 1.Environment
> CPU: Intel(R) Core(TM) i7-6700 CPU @ 3.40GHz
> Linux kernel version: 5.11-rc3
> Audit version: 2.8.4
>
> 2.Result
> 2.1 Syscall invocations
> Test method:
>     Run command "top" with no-load.
>     Add rule likes "auditctl -a always,exit -F arch=b64 -S chmod -F auid=[number]" which doesn't hit audit.
>     User command "perf record -Rg -t [top's pid] sleep 900" to get audit_filter_syscall()'s execute time ratio.

Thanks for providing some performance numbers so quickly, a few
comments and thoughts below ...

> audit_filter_syscall() ratio with 100 rules:
>     before this patch: 15.29%.
>     after this patch: 0.88%, reduce 14.41%.
> audit_filter_syscall() ratio with CIS[1] rules:
>     before this patch: 2.25%.
>     after this patch: 1.93%%, reduce 0.32%.
> audit_filter_syscall() ratio with 10 rules:
>     before this patch: 0.94%.
>     after this patch: 1.02%, increase 0.08%.
> audit_filter_syscall() ratio with 1 rule:
>     before this patch: 0.20%.
>     after this patch: 0.88%, increase 0.68%.

If we assume the CIS rules to be a reasonable common case (I'm not
sure if that is correct or not, but we'll skip that discussion for
now), we see an performance improvement of 0.32% correct, yes?  We
also see a performance regression with small number of syscall rules
that equalizes above ten rules, yes?

On your system can you provide some absolute numbers?  For example,
what does 0.32% equate to in terms of wall clock time for a given
syscall invocation?

> Analyse:
>     With 1 rule, after this patch performance is worse, because mutex_lock()/mutex_unlock(). But user just add one rule seems unusual.
>     With more rule, after this patch performance is improved.Typical likes CIS benchmark.
>
> 2.2 Rule change
> Test method:
>     Use ktime_get_real_ts64() before and after audit_add_rule()/audit_del_rule() to calculate time.
>      Add/delete rule by command "auditctl". Each test 10times and get average.

In this case I'm less concerned about micro benchmarks, and more
interested in the wall clock time difference when running auditctl to
add/remove rules.  The difference here in the micro benchmark is not
trivial, but with a delta of 4~5us it is possible that it is a
small(er) percentage when compared to the total time spent executing
auditctl.

> audit_add_rule() time:
>     before this patch: 3120ns.
>     after this patch: 7783ns, increase 149%.
> audit_del_rule() time:
>     before this patch: 3510ns.
>     after this patch: 8519ns, increase 143%.
>
> Analyse:
>     After this patch, rule change time obviously increase. But rule change may not happen very often.
>
> [1] CIS is a Linux Benchmarks for security purpose.
> https://www.cisecurity.org/benchmark/distribution_independent_linux/

-- 
paul moore
www.paul-moore.com

--
Linux-audit mailing list
Linux-audit@redhat.com
https://www.redhat.com/mailman/listinfo/linux-audit


^ permalink raw reply	[flat|nested] 2+ messages in thread

end of thread, other threads:[~2021-01-14 15:22 UTC | newest]

Thread overview: 2+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-01-14 13:25 Fw:Re:[RFC,v1,1/1] audit: speed up syscall rule match while exiting syscall yang.yang29
2021-01-14 15:22 ` Paul Moore

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).