Re: [PATCH net-next] af_unix: dont send SCM_CREDENTIALS by default

From: Tim Chen <tim.c.chen@linux.intel.com>
To: Eric Dumazet <eric.dumazet@gmail.com>
Cc: "Yan, Zheng" <zheng.z.yan@intel.com>,
	"Yan, Zheng" <yanzheng@21cn.com>,
	"netdev@vger.kernel.org" <netdev@vger.kernel.org>,
	"davem@davemloft.net" <davem@davemloft.net>,
	"sfr@canb.auug.org.au" <sfr@canb.auug.org.au>,
	"jirislaby@gmail.com" <jirislaby@gmail.com>,
	"sedat.dilek@gmail.com" <sedat.dilek@gmail.com>,
	"Shi, Alex" <alex.shi@intel.com>,
	Valdis Kletnieks <Valdis.Kletnieks@vt.edu>
Subject: Re: [PATCH net-next] af_unix: dont send SCM_CREDENTIALS by default
Date: Mon, 12 Sep 2011 12:15:46 -0700	[thread overview]
Message-ID: <1315854946.2576.3122.camel@schen9-DESK> (raw)
In-Reply-To: <1315544777.5410.19.camel@edumazet-laptop>

On Fri, 2011-09-09 at 07:06 +0200, Eric Dumazet wrote:
> Le jeudi 08 septembre 2011 à 02:24 -0700, Tim Chen a écrit :
> 
> > Looking forward to the patch.  This should improve the scalability of
> > af_unix.
> 
> Here it is, based on top on previous one [af_unix: Fix use-after-free
> crashes]
> 
> Thanks
> 
> [PATCH net-next] af_unix: dont send SCM_CREDENTIALS by default
> 
> Since commit 7361c36c5224 (af_unix: Allow credentials to work across
> user and pid namespaces) af_unix performance dropped a lot.
> 
> This is because we now take a reference on pid and cred in each write(),
> and release them in read(), usually done from another process,
> eventually from another cpu. This triggers false sharing.
> 
> # Events: 154K cycles
> #
> # Overhead  Command       Shared Object                               Symbol
> # ........  .......  ..................  ...................................
> #
>     10.40%  hackbench  [kernel.kallsyms]   [k] put_pid
>      8.60%  hackbench  [kernel.kallsyms]   [k] unix_stream_recvmsg
>      7.87%  hackbench  [kernel.kallsyms]   [k] unix_stream_sendmsg
>      6.11%  hackbench  [kernel.kallsyms]   [k] do_raw_spin_lock
>      4.95%  hackbench  [kernel.kallsyms]   [k] unix_scm_to_skb
>      4.87%  hackbench  [kernel.kallsyms]   [k] pid_nr_ns
>      4.34%  hackbench  [kernel.kallsyms]   [k] cred_to_ucred
>      2.39%  hackbench  [kernel.kallsyms]   [k] unix_destruct_scm
>      2.24%  hackbench  [kernel.kallsyms]   [k] sub_preempt_count
>      1.75%  hackbench  [kernel.kallsyms]   [k] fget_light
>      1.51%  hackbench  [kernel.kallsyms]   [k] __mutex_lock_interruptible_slowpath
>      1.42%  hackbench  [kernel.kallsyms]   [k] sock_alloc_send_pskb
> 
> 
> This patch includes SCM_CREDENTIALS information in a af_unix message/skb
> only if requested by the sender, [man 7 unix for details how to include
> ancillary data using sendmsg() system call]
> 
> Note: This might break buggy applications that expected SCM_CREDENTIAL
> from an unaware write() system call.
> 
> Performance boost in hackbench : more than 50% gain on a 16 thread
> machine (2 quad-core cpus, 2 threads per core)
> 
> hackbench 20 thread 2000
> 
> 4.224 sec instead of 9.102 sec
> 
> 
> Signed-off-by: Eric Dumazet <eric.dumazet@gmail.com>

This speeds up the case where we're not passing SCM_CREDENTIALS nicely.

Acked-by: Tim Chen <tim.c.chen@linux.intel.com>