Re: [RFC][PATCH 00/13] percpu rwsem -v2

[Daniel Wagner <daniel.wagner@bmw-carit.de>]

On 06/23/2015 04:34 PM, Peter Zijlstra wrote:
> On Tue, Jun 23, 2015 at 11:35:24AM +0200, Daniel Wagner wrote:
>> flock01
>>                              mean   variance      sigma        max        min
>>                     4.1.0    11.7075   816.3341    28.5716   125.6552     0.0021
>>        4.1.0+percpu-rwsem    11.4614   760.1345    27.5705   132.5030     0.0026
>>                       tip     6.8390   329.3037    18.1467    81.0373     0.0021
>>          tip+percpu-rwsem    10.0870   546.7435    23.3825   106.2396     0.0026
> 
>> posix01
>>                              mean   variance      sigma        max        min
>>                     4.1.0   121.9020 27882.5260   166.9806   603.5509     0.0063
>>        4.1.0+percpu-rwsem   185.3981 38474.3836   196.1489   580.6532     0.0073
>>                       tip   129.2736 23752.7122   154.1191   474.0604     0.0063
>>          tip+percpu-rwsem   142.6474 24732.1571   157.2646   468.7478     0.0072
> 
> Both these tests are incredibly unstable for me (as well as for you it
> appears). Variance is through the roof on them.

Since on my test machine not all 4 socket have inter connection, I pinned the 
tests down to one socket to see if that reduces the variance. 

Expect flock01 and posix01 show now really low variances (3 runs):

[...]
flock02
                             mean   variance      sigma        max        min
                    tip-1    11.8994     0.5874     0.7664    13.2022     8.6324
                    tip-2    11.7394     0.5252     0.7247    13.2540     9.7513
                    tip-3    11.8155     0.5288     0.7272    13.2700     9.9480
       tip+percpu-rswem-1    15.3601     0.8981     0.9477    16.8116    12.6910
       tip+percpu-rswem-2    15.2558     0.8442     0.9188    17.0199    12.9586
       tip+percpu-rswem-3    15.5297     0.6386     0.7991    17.4392    12.7992


lease01
                             mean   variance      sigma        max        min
                    tip-1     0.3424     0.0001     0.0110     0.3644     0.3088
                    tip-2     0.3627     0.0003     0.0185     0.4140     0.3312
                    tip-3     0.3446     0.0002     0.0125     0.3851     0.3155
       tip+percpu-rswem-1     0.3464     0.0001     0.0116     0.3781     0.3113
       tip+percpu-rswem-2     0.3597     0.0003     0.0162     0.3978     0.3250
       tip+percpu-rswem-3     0.3513     0.0002     0.0151     0.3933     0.3122
[...]

So with this setup we can start to compare the numbers.

> I get runtimes like:
> 
> root@ivb-ex:/usr/local/src/lockperf# ./flock01 -n 240 -l 32 /tmp/a 
> 0.266157011
> root@ivb-ex:/usr/local/src/lockperf# ./flock01 -n 240 -l 32 /tmp/a 
> 139.303399960

Same here:

flock01
                             mean   variance      sigma        max        min
                    tip-1   242.6147  3632.6201    60.2712   313.3081    86.3743
                    tip-2   233.1934  3850.1995    62.0500   318.2716   101.2738
                    tip-3   223.0392  3944.5220    62.8054   318.1932   110.8155
       tip+percpu-rswem-1   276.5913  2145.0510    46.3147   317.5385   156.1318
       tip+percpu-rswem-2   270.7089  2735.7635    52.3045   318.9418   154.5902
       tip+percpu-rswem-3   267.8207  3028.3557    55.0305   320.2987   150.9659

posix01
                             mean   variance      sigma        max        min
                    tip-1    18.8729   151.2810    12.2996    37.3563     0.0060
                    tip-2    17.6894   140.9982    11.8743    37.2080     0.0060
                    tip-3    18.7785   145.1217    12.0466    35.5001     0.0060
       tip+percpu-rswem-1    18.9970   163.8856    12.8018    35.8795     0.0069
       tip+percpu-rswem-2    18.9594   147.3197    12.1375    35.4404     0.0069
       tip+percpu-rswem-3    18.8366   126.5831    11.2509    35.9014     0.0069