Re: Perf Data on LSM in v5.3

From: Stephen Smalley <sds@tycho.nsa.gov>
To: Wenhui Zhang <wenhui@gwmail.gwu.edu>,
	John Johansen <john.johansen@canonical.com>
Cc: Casey Schaufler <casey@schaufler-ca.com>,
	Casey Schaufler <casey.schaufler@intel.com>,
	James Morris <jmorris@namei.org>,
	linux-security-module@vger.kernel.org,
	SELinux <selinux@vger.kernel.org>,
	Kees Cook <keescook@chromium.org>,
	penguin-kernel@i-love.sakura.ne.jp,
	Paul Moore <paul@paul-moore.com>
Subject: Re: Perf Data on LSM in v5.3
Date: Wed, 15 Jan 2020 09:09:01 -0500	[thread overview]
Message-ID: <2a984c76-a499-53f6-68c5-5d3604094ba4@tycho.nsa.gov> (raw)
In-Reply-To: <e7cfc960-32fb-7712-b21c-37999cf29430@tycho.nsa.gov>

On 1/15/20 8:40 AM, Stephen Smalley wrote:
> On 1/14/20 8:00 PM, Wenhui Zhang wrote:
>> Hi, John:
>>
>> It seems like, the MAC hooks are default to*return 0 or empty void 
>> hooks* if CONFIG_SECURITY, CONFIG_SECURITY_NETWORK , 
>> CONFIG_PAGE_TABLE_ISOLATION, CONFIG_SECURITY_INFINIBAND, 
>> CONFIG_SECURITY_PATH, CONFIG_INTEL_TXT,
>> CONFIG_HAVE_HARDENED_USERCOPY_ALLOCATOR, 
>> CONFIG_HARDENED_USERCOPY, CONFIG_HARDENED_USERCOPY_FALLBACK *are NOT 
>> set*.
>>
>> If HOOKs are "return 0 or empty void hooks", MAC is not enabled.
>> In runtime of fs-benchmarks, if CONFIG_DEFAULT_SECURITY_DAC=y, then 
>> capability is enabled.
>>
>> Please correct me if I am wrong.
>>
>> For the first test, wo-sec is tested with:
>> # CONFIG_SECURITY_DMESG_RESTRICT is not set
>> # CONFIG_SECURITY is not set
>> # CONFIG_SECURITYFS is not set
>> # CONFIG_PAGE_TABLE_ISOLATION is not set
>> # CONFIG_INTEL_TXT is not set
>> CONFIG_HAVE_HARDENED_USERCOPY_ALLOCATOR=y
>> # CONFIG_HARDENED_USERCOPY is not set
>> CONFIG_FORTIFY_SOURCE=y
>> # CONFIG_STATIC_USERMODEHELPER is not set
>> CONFIG_DEFAULT_SECURITY_DAC=y
>>
>>
>> For the second test, w-sec is tested with:
>> # CONFIG_SECURITY_DMESG_RESTRICT is not set
>> CONFIG_SECURITY=y
>> CONFIG_SECURITYFS=y
>> # CONFIG_SECURITY_NETWORK is not set
>> CONFIG_PAGE_TABLE_ISOLATION=y
>> CONFIG_SECURITY_INFINIBAND=y
>> CONFIG_SECURITY_PATH=y
>> CONFIG_INTEL_TXT=y
>> CONFIG_HAVE_HARDENED_USERCOPY_ALLOCATOR=y
>> CONFIG_HARDENED_USERCOPY=y
>> CONFIG_HARDENED_USERCOPY_FALLBACK=y
>> # CONFIG_HARDENED_USERCOPY_PAGESPAN is not set
>> CONFIG_FORTIFY_SOURCE=y
>> # CONFIG_STATIC_USERMODEHELPER is not set
>> # CONFIG_SECURITY_SMACK is not set
>> # CONFIG_SECURITY_TOMOYO is not set
>> # CONFIG_SECURITY_APPARMOR is not set
>> # CONFIG_SECURITY_LOADPIN is not set
>> # CONFIG_SECURITY_YAMA is not set
>> # CONFIG_SECURITY_SAFESETID is not set
>> # CONFIG_INTEGRITY is not set
>> CONFIG_DEFAULT_SECURITY_DAC=y
>> # 
>> CONFIG_LSM="yama,loadpin,safesetid,integrity,apparmor,selinux,smack,tomoyo" 
>>
> 
> Your configs should only differ with respect to CONFIG_SECURITY* if you 
> want to evaluate LSM, SELinux, etc overheads.  PAGE_TABLE_ISOLATION, 
> INTEL_TXT, and HARDENED_USERCOPY are not relevant to LSM itself.
> 
> Also, what benchmarks are you using?  Your own home-grown ones, a set of 
> open source standard benchmarks (if so, which ones?).  You should 
> include both micro and macro benchmarks in your suite.
> 
> How stable are your results?  What kind of variance / standard deviation 
> are you seeing?
> 
> It is hard to get meaningful, reliable performance measurements so going 
> down this road is not to be done lightly.

Also, I note that you don't enable CONFIG_SECURITY_NETWORK above.  That 
means you aren't including the base LSM overhead for the networking 
security hooks.  So if you then compare that against SELinux (which 
requires CONFIG_SECURITY_NETWORK), you are going to end up attributing 
the cost of both the LSM overhead and SELinux overhead all to SELinux. 
If you truly want to isolate the base LSM overhead, you need to enable 
all the hooks.