Fw: [Bug 194241] New: LXC - setns() performance issue with more than 1vCPU

Fw: [Bug 194241] New: LXC - setns() performance issue with more than 1vCPU

[Stephen Hemminger]

Begin forwarded message:

Date: Tue, 07 Feb 2017 02:18:18 +0000
From: bugzilla-daemon@bugzilla.kernel.org
To: stephen@networkplumber.org
Subject: [Bug 194241] New: LXC - setns() performance issue with more than 1vCPU


https://bugzilla.kernel.org/show_bug.cgi?id=194241

            Bug ID: 194241
           Summary: LXC - setns() performance issue with more than 1vCPU
           Product: Networking
           Version: 2.5
    Kernel Version: 3.6.57 , 3.16.7
          Hardware: All
                OS: Linux
              Tree: Mainline
            Status: NEW
          Severity: normal
          Priority: P1
         Component: Other
          Assignee: stephen@networkplumber.org
          Reporter: skarmarkar@sonusnet.com
        Regression: No

Created attachment 254411
  --> https://bugzilla.kernel.org/attachment.cgi?id=254411&action=edit  
sample program to reproduce the issue

Description of problem:
Facing a peculiar problem with setns () call. 
Trying to test the network namespace basic functionality and performance tests.
The basic test is to create two network namespaces (nstest1, and nstest2) and
enabled the loopback interfaces in them.
Then the test program called setns () function (Re-associate thread with a
namespace) to switch from namespace to nstest1, then from nstest1 to nstest2,
from nstest2 to ntest1 etc. (round-robin).


Version-Release number of selected component (if applicable):

As part of the tests tried on kernel 3.6.57 and kernel 3.16.7, and ran the same
setns() tests with only one CPU, and observed system call rates of several
million calls per second. The results were incredibly different - for kernel
3.6.57 it was 0.22 micro second per setns() call vs. 2.1 milli second (10,000
times faster; for kernel 3.16.7 it was 0.16 micro second per setns() call vs.
31 milli second (200,000 times faster).

1.       On kernel 3.6.57, the same test run on 1-CPU VM executed 100,000,000
setns () calls in 22 seconds (4,545,454 calls per second; 0.22 micro second per
call). 
2.       On kernel 3.6.57, the same test run on 2-CPU or 4-CPU VM executed
10000 setns() calls in 21 seconds (476 calls per second; 2.1 milli second per
call)


How reproducible:
easily and always reproducible

Steps to Reproduce:

Attached test program

Details on the program below-
test_netns.c – (attached).

  - Based on http://linux.die.net/man/2/setns
Build:
  $ gcc -g -o test_netns test_netns.c

Test setup:
  1. Create network namespaces nstest1 and nstest2, and bring up loopback (lo) 
     # ip netns add nstest1
     # ip netns add nstest2
     # ip netns exec nstest1 ip link set dev lo up
     # ip netns exec nstest2 ip link set dev lo up
  2. Make sure that the namespaces are seen.
     # ip netns
     nstest1
     nstest2
     # ls /var/run/netns/
nstest1  nstest2
  3. Get the PID of the current process. This PID is to be used in the test
command.
     # echo $$
     24821
     #
  4. Run the test. Need to replace [PID] from the Step 3; replace [ITERS] with
number of desired
     setns() calls.
     # ./test_netns /var/run/netns/nstest1 /var/run/netns/nstest2
/proc/[PID]/ns/net [ITERS] "/sbin/ip link"
  4.A Example for running the test for 1000 iterations for PID=24821 for three
namespaces: 
     two namespaces and process's own:
     # ./test_netns /var/run/netns/nstest1 /var/run/netns/nstest2
/proc/24821/ns/net 1000 "/sbin/ip link"

Actual results:


Expected results:


Additional info:

-- 
You are receiving this mail because:
You are the assignee for the bug.

Re: [Bug 194241] New: LXC - setns() performance issue with more than 1vCPU

[David Miller]

From: Stephen Hemminger <stephen@networkplumber.org>
Date: Tue, 7 Feb 2017 08:30:32 -0800

>     Kernel Version: 3.6.57 , 3.16.7

Yeah, we've made thousands of network namespace performance
improvements since then.