Re: BIG performance hit with auditd on large systems (>64 CPUs)

From: Paul Moore <paul@paul-moore.com>
To: Klaus Lichtenwalder <klic@mnet-online.de>, linux-audit@redhat.com
Subject: Re: BIG performance hit with auditd on large systems (>64 CPUs)
Date: Tue, 30 May 2017 15:49:13 -0400	[thread overview]
Message-ID: <CAHC9VhS_XPnBLsU3b=3Wws1HmCO7sNrrGfqyvxi=SyNhVUFhjg@mail.gmail.com> (raw)
In-Reply-To: <cd40e2a5-6862-09a1-94c8-f7347dfd20bd@mnet-online.de>

On Tue, May 30, 2017 at 2:17 PM, Klaus Lichtenwalder
<klic@mnet-online.de> wrote:
>>>> your rules to put all the ones with '-F auid>=400' below a single
>>>> line rule
>>>> like this:
>>>> -a never,exit -F auid<400
>>>>
>>>> and remove the '-F auid>=400' from all of the rules below it.
>>>>
>>> ...
>>>
>>> I did this, and verified it, but there was absolutely no difference
>>> to unsorted rules having -S all also specified
>>>
>>> Still cpu %system up to 50% and run time of jobs 100% longer.
>>> This was on a vm with 72 cpus
>>>
>
> Just to give this story some kind of closure: we got a test kernel from
> $SUPPORT fixing a specifig bugzilla (which seems to be private) and %cpu
> system is in normal (low) ranges again.
>
> So thanks for your advices, they are still heeded!

For the record the core issue was fixed in f56298835036 ("audit:
acquire creds selectively to reduce atomic op overhead").

-- 
paul moore
www.paul-moore.com