From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751846AbbGAGDr (ORCPT ); Wed, 1 Jul 2015 02:03:47 -0400 Received: from hotel311.server4you.de ([85.25.146.15]:48414 "EHLO hotel311.server4you.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750805AbbGAGDl (ORCPT ); Wed, 1 Jul 2015 02:03:41 -0400 X-Greylist: delayed 337 seconds by postgrey-1.27 at vger.kernel.org; Wed, 01 Jul 2015 02:03:40 EDT Message-ID: <55938163.7020606@monom.org> Date: Wed, 01 Jul 2015 07:57:55 +0200 From: Daniel Wagner User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.7.0 MIME-Version: 1.0 To: Ingo Molnar , Peter Zijlstra CC: oleg@redhat.com, paulmck@linux.vnet.ibm.com, tj@kernel.org, mingo@redhat.com, linux-kernel@vger.kernel.org, der.herr@hofr.at, dave@stgolabs.net, riel@redhat.com, viro@ZenIV.linux.org.uk, torvalds@linux-foundation.org, jlayton@poochiereds.net Subject: Re: [RFC][PATCH 00/13] percpu rwsem -v2 References: <20150622121623.291363374@infradead.org> <55884FC2.6030607@bmw-carit.de> <20150622190553.GZ3644@twins.programming.kicks-ass.net> <5589285C.2010100@bmw-carit.de> <20150623143411.GA25159@twins.programming.kicks-ass.net> <558973A7.6010407@bmw-carit.de> <20150623175012.GD3644@twins.programming.kicks-ass.net> <20150623193624.GH18673@twins.programming.kicks-ass.net> <20150624084648.GB27873@gmail.com> <558A75EA.40905@bmw-carit.de> In-Reply-To: <558A75EA.40905@bmw-carit.de> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Hi, I did a sweep over the parameters for posix01. The parameters are number of processes and number of locks taken per process. In contrast to the other test, it looks like there is no set which ends a nice stable result (read low variance). I have tried several things including pinning down all processes to CPUs to avoid migration. The results improved slightly but there was still a high variance. Anyway I have collected some data and I like to share it. Maybe it is still useful. All numbers here are without the above mentioned pinning. There are some runs missing (don't know the reason yet) and I didn't let it run till the end. So add some salt to these numbers. The test script and raw data can be found here: http://monom.org/posix01/ The tables reads: nproc: number of process started columns: number of locks taken per process Hardware 4x E5-4610, for this test all process are scheduled on one socket First the numbers for tip 4.1.0-02756-ge3d06bd. nproc 8 100 200 300 400 500 count 1000.000000 1000.000000 1000.000000 1000.000000 1000.000000 mean 0.075449 0.210547 0.340658 0.464083 0.590400 std 0.015550 0.024989 0.032080 0.043803 0.055003 min 0.021643 0.067456 0.211779 0.279643 0.327628 25% 0.065337 0.195664 0.318114 0.430040 0.546488 50% 0.075345 0.209411 0.338512 0.461397 0.591433 75% 0.084725 0.226517 0.364190 0.494638 0.626532 max 0.127050 0.281836 0.454558 0.607559 0.762149 nproc 16 100 200 300 400 500 count 1000.000000 1000.000000 1000.000000 1000.000000 1000.000000 mean 1.023660 2.463384 3.891954 5.312716 6.752857 std 0.105065 0.124916 0.136476 0.172906 0.207449 min 0.351199 1.527379 3.106403 4.157478 5.519601 25% 0.961098 2.397597 3.807098 5.201875 6.633034 50% 1.031460 2.467317 3.895824 5.321227 6.757502 75% 1.093412 2.539284 3.985122 5.432336 6.889859 max 1.278603 2.785901 4.369434 5.798982 7.324263 nproc 24 100 200 300 400 500 count 1000.000000 1000.000000 1000.000000 1000.000000 1000.000000 mean 3.460166 7.942193 11.898540 11.150066 11.060036 std 0.191564 0.232989 0.612868 0.680323 0.465967 min 2.748545 6.575510 9.977165 9.209685 8.937682 25% 3.325521 7.806847 11.440580 10.774070 10.912302 50% 3.493138 7.951859 11.852556 11.163595 11.074910 75% 3.596927 8.088036 12.443429 11.365197 11.243125 max 3.974884 8.589840 13.079780 16.341043 14.244954 nproc 32 100 200 300 400 500 count 1000.000000 1000.000000 1000.000000 1000.000000 1000.000000 mean 6.797286 13.943421 14.373278 15.857103 20.047039 std 0.366013 0.417859 0.625967 0.377463 0.302939 min 3.323312 12.266006 12.492706 14.451931 17.496059 25% 6.649401 13.719397 14.186790 15.738348 19.958001 50% 6.868362 13.862458 14.312992 15.870438 20.083564 75% 6.995801 14.027167 14.429383 15.984881 20.215722 max 7.369007 15.631300 21.587450 19.364991 20.755793 nproc 40 100 200 300 400 500 count 1000.000000 1000.000000 1000.000000 1000.000000 1000.000000 mean 11.156514 16.936808 18.930412 25.605206 32.334239 std 0.613158 0.614545 0.485336 0.344226 0.398747 min 5.609261 13.147398 16.930261 23.448985 28.992899 25% 10.999876 16.740775 18.788180 25.481274 32.188020 50% 11.251502 16.883100 18.946506 25.648879 32.369347 75% 11.439205 17.032133 19.105678 25.806715 32.565019 max 12.155905 24.116348 26.152117 26.502637 33.263763 nproc 48 100 200 300 400 500 count 1000.000000 1000.000000 1000.000000 1000.000000 1000.000000 mean 16.523705 18.214558 27.877811 37.703763 47.655792 std 0.974732 1.118383 0.357481 0.435081 0.472945 min 7.909358 16.279568 25.989797 35.308061 45.279940 25% 16.385582 17.960832 27.729399 37.555420 47.458123 50% 16.692900 18.137635 27.920459 37.767064 47.679325 75% 16.927355 18.311502 28.092018 37.950782 47.926311 max 17.720374 35.810409 28.721941 38.746273 49.333097 nproc 56 100 200 300 400 500 count 1000.000000 1000.000000 1000.000000 1000.000000 1000.000000 mean 11.567668 25.100333 38.603884 52.135564 65.716669 std 0.320771 0.369833 0.554834 0.534120 0.612844 min 10.123811 22.598875 35.668780 49.182148 62.504962 25% 11.394438 24.925338 38.389200 51.885988 65.441492 50% 11.593920 25.135043 38.641839 52.206010 65.771692 75% 11.789101 25.328558 38.895343 52.451819 66.068270 max 12.319346 25.948404 46.458428 53.605888 67.270679 nproc 64 100 200 300 400 500 count 1000.000000 1000.000000 1000.000000 1000.000000 1000.000000 mean 15.421295 33.254418 51.073912 68.936111 86.919074 std 0.398493 0.411222 0.551629 0.690891 0.694183 min 13.269859 30.900978 48.174802 65.549282 83.099271 25% 15.203732 33.037478 50.821702 68.619365 86.579749 50% 15.467885 33.279869 51.130972 69.001664 86.953804 75% 15.694466 33.514712 51.380860 69.361632 87.341084 max 16.347321 34.475095 52.507292 70.884752 88.807083 nproc 72 100 200 300 400 500 count 1000.000000 1000.000000 1000.000000 1000.000000 1000.000000 mean 19.762286 42.488827 65.167763 87.903430 110.666679 std 0.483660 0.480269 0.689872 0.828354 0.892759 min 15.506067 39.937453 61.196633 84.227403 107.014850 25% 19.519194 42.261548 64.834133 87.515837 110.225142 50% 19.809986 42.541263 65.265768 87.974049 110.747980 75% 20.083315 42.792858 65.603762 88.392599 111.223192 max 20.913434 43.830009 66.791452 90.184550 113.062344 nproc 80 100 200 300 400 500 count 1000.000000 1000.000000 1000.000000 1000.000000 1000.000000 mean 24.782285 52.853068 80.902314 109.112294 137.441640 std 0.523731 0.639160 0.799033 0.952619 1.091478 min 20.126615 47.813274 77.357915 104.033857 131.978443 25% 24.498501 52.547855 80.509926 108.606293 136.877050 50% 24.835766 52.918841 80.950773 109.197236 137.498470 75% 25.137887 53.244013 81.376380 109.723791 138.101133 max 26.161997 54.372957 83.266046 111.709888 140.419400 nproc 88 100 200 300 400 500 count 1000.000000 1000.000000 1000.000000 1000.000000 1000.000000 mean 30.196867 64.467080 98.710365 133.024282 167.330900 std 0.749476 0.691460 0.863908 1.033780 1.240237 min 16.647491 60.034797 94.053510 128.281171 161.778166 25% 29.896764 64.121607 98.290368 132.484092 166.711172 50% 30.271808 64.514222 98.742714 133.089852 167.429483 75% 30.627200 64.903154 99.262584 133.706735 168.086624 max 31.806051 66.343856 101.077264 136.143873 170.449596 nproc 96 100 200 300 400 500 count 1000.000000 1000.000000 1000.000000 1000.000000 1000.000000 mean 36.304100 77.194851 117.958001 158.820159 199.868940 std 0.712442 0.718565 1.009163 1.220813 1.462219 min 31.128111 73.850226 112.075970 152.910227 192.977453 25% 35.928427 76.811233 117.466922 158.151278 199.058411 50% 36.378220 77.209148 117.998878 158.879704 199.861157 75% 36.761744 77.636286 118.615380 159.583272 200.701769 max 38.069263 79.445286 120.878239 162.826438 206.826424 nproc 104 100 200 300 400 500 count 1000.000000 1000.000000 1000.000000 1000.000000 1000.000000 mean 42.731401 90.887253 138.815476 186.824953 235.055458 std 1.045572 0.742232 0.999065 1.298818 1.554890 min 23.734733 87.384048 133.462821 180.971966 227.475939 25% 42.353032 90.441055 138.213962 186.109237 234.169575 50% 42.861112 90.900274 138.836083 186.835884 235.084204 75% 43.236527 91.382487 139.460129 187.694247 236.011148 max 44.600281 93.394394 141.959512 190.171221 239.491909 nproc 112 100 200 300 400 count 460.000000 460.000000 460.000000 460.000000 mean 49.782729 105.468739 161.416099 217.385757 std 0.904312 1.011980 1.222772 1.475225 min 45.334285 100.711113 156.087707 210.639527 25% 49.394518 104.971028 160.743875 216.590612 50% 49.906665 105.604756 161.528712 217.437408 75% 50.363428 106.088852 162.187166 218.286111 max 51.800116 108.372299 164.614385 221.788613 And now the same tests for tip+percpu_rwsem: nproc 8 100 200 300 400 500 count 1000.000000 1000.000000 1000.000000 1000.000000 1000.000000 mean 0.285784 0.639623 0.935062 1.165287 1.457565 std 0.040458 0.089317 0.112704 0.094596 0.110337 min 0.118961 0.253775 0.351943 0.869095 1.026194 25% 0.263250 0.600806 0.858630 1.100281 1.376566 50% 0.287019 0.649395 0.930437 1.167166 1.461235 75% 0.312601 0.692013 1.013786 1.228887 1.533511 max 0.407264 0.860837 1.298671 1.460842 1.927867 nproc 16 100 200 300 400 500 count 1000.000000 1000.000000 1000.000000 1000.000000 1000.000000 mean 2.338683 5.219408 8.117279 11.050641 14.035433 std 0.146102 0.270400 0.392875 0.510692 0.576044 min 1.836110 4.179970 6.491748 8.998336 11.442838 25% 2.239374 5.042915 7.860587 10.728740 13.667630 50% 2.335801 5.217732 8.125243 11.052183 14.010561 75% 2.443152 5.404223 8.396037 11.404375 14.417740 max 2.798029 5.927344 9.172875 12.203548 15.444552 nproc 24 100 200 300 400 500 count 1000.000000 1000.000000 1000.000000 1000.000000 1000.000000 mean 6.399927 13.673487 20.729554 27.316864 34.125202 std 0.558388 1.157996 1.647191 2.066864 2.487975 min 4.961608 10.767524 17.145018 22.441426 28.566438 25% 5.987118 12.849801 19.555979 25.943463 32.399122 50% 6.388215 13.583983 20.533054 27.122120 33.959403 75% 6.915310 14.786835 22.252796 29.187176 36.308254 max 7.405319 15.823960 23.858206 31.754922 38.997955 nproc 32 100 200 300 400 500 count 1000.000000 1000.000000 1000.000000 1000.000000 1000.000000 mean 11.973832 24.885823 36.705614 48.036525 57.418669 std 1.270516 2.604583 3.963139 5.283237 6.441122 min 9.395066 19.958662 27.768684 38.247046 46.265231 25% 10.955417 22.708953 33.510437 43.613011 51.901209 50% 11.801515 24.556642 35.805816 47.315635 55.933447 75% 13.294692 27.520679 40.689642 53.139912 63.860584 max 14.217272 29.968337 44.409489 58.246754 71.045867 nproc 40 100 200 300 400 500 count 1000.000000 1000.000000 1000.000000 1000.000000 1000.000000 mean 19.307414 39.204462 55.768040 70.808627 83.830246 std 2.189803 3.982241 5.467692 6.737372 8.124025 min 14.450258 30.606836 44.342114 55.520218 64.704178 25% 17.418113 35.968251 51.341042 65.352697 77.744806 50% 19.067713 39.023460 55.548934 70.282785 83.374667 75% 21.479466 42.666118 60.379906 76.604241 91.158904 max 23.687483 47.019928 67.143361 85.084045 100.957011 nproc 48 100 200 300 400 500 count 1000.000000 1000.000000 1000.000000 1000.000000 1000.000000 mean 28.386773 55.462523 77.886706 92.579064 104.319703 std 3.231688 6.142373 8.633285 10.950222 12.510504 min 21.703659 42.486864 56.904221 66.605689 76.529646 25% 25.635256 50.575642 71.306694 82.931995 94.222776 50% 28.136694 55.235674 77.298409 91.993559 104.909015 75% 31.484979 60.645302 85.693462 102.195018 114.141212 max 35.713537 68.342796 96.065304 115.926497 130.916876 nproc 56 100 200 300 400 500 count 1000.000000 1000.000000 1000.000000 1000.000000 1000.000000 mean 39.037206 74.470404 97.900979 111.320283 135.943281 std 4.594741 8.940246 11.715321 13.823450 16.032080 min 29.532559 55.193557 65.590273 79.580482 98.565733 25% 35.212004 66.990273 88.066459 100.643871 122.864654 50% 38.796902 73.928176 96.771490 110.669216 136.199617 75% 43.154846 82.041731 108.937264 120.727216 147.769269 max 49.215714 92.181542 125.188702 141.113117 170.961264 nproc 64 100 200 300 400 500 count 1000.000000 1000.000000 1000.000000 1000.000000 1000.000000 mean 51.099012 93.028015 114.649700 145.944300 178.043572 std 6.310777 12.719401 14.675830 18.019135 21.084448 min 36.770938 54.620852 80.837116 98.765936 126.207980 25% 45.955694 84.078285 103.452854 132.127548 160.746493 50% 50.275929 93.031565 114.333533 144.951788 177.105994 75% 56.955477 104.656181 128.418118 163.865640 197.275452 max 63.369715 120.360706 146.542148 182.482159 218.814651 nproc 72 100 200 300 400 500 count 506.000000 506.000000 506.000000 506.000000 506.000000 mean 64.905270 108.760098 138.811285 179.277895 222.584001 std 8.784532 16.293281 18.160401 21.203767 25.904456 min 43.035451 64.762288 96.401934 127.995159 162.341026 25% 58.658290 98.438247 126.035692 162.944645 202.228444 50% 64.756854 109.608197 139.190635 181.413255 223.359111 75% 72.488483 123.608470 152.745541 195.549278 245.454358 max 83.424516 139.214509 172.538610 218.677815 270.799895 nproc 80 100 200 300 400 500 count 61.000000 61.000000 61.000000 61.000000 61.000000 mean 76.727789 124.438489 174.095378 225.855798 272.416390 std 9.757928 18.034325 20.216132 24.868596 29.384832 min 55.988043 83.842137 130.842940 173.596051 208.508169 25% 69.218268 116.679810 162.149179 207.015727 252.194955 50% 75.392969 125.378519 173.117425 225.071270 276.188038 75% 83.748328 136.689138 192.392097 245.019530 296.407232 max 97.004966 165.172805 206.391629 266.751069 318.089290 nproc 88 100 count 157.000000 mean 90.337638 std 15.239911 min 53.393662 25% 79.648088 50% 91.075065 75% 103.530939 max 120.680507 And an attempt at visualization: http://monom.org/posix01/sweep-4.1.0-02756-ge3d06bd.png http://monom.org/posix01/sweep-4.1.0-02769-g6ce2591.png Let me know if these numbers help or not. I start to get better in running those tests tough they take quite some time to finish. So if they are useless I sleep well without doing this :) cheers, daniel