From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1750978AbcGLSRE (ORCPT ); Tue, 12 Jul 2016 14:17:04 -0400 Received: from mail-by2nam03on0109.outbound.protection.outlook.com ([104.47.42.109]:64182 "EHLO NAM03-BY2-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1750720AbcGLSRB (ORCPT ); Tue, 12 Jul 2016 14:17:01 -0400 X-Greylist: delayed 97587 seconds by postgrey-1.27 at vger.kernel.org; Tue, 12 Jul 2016 14:17:00 EDT Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=waiman.long@hpe.com; Message-ID: <578533FE.9050606@hpe.com> Date: Tue, 12 Jul 2016 14:16:30 -0400 From: Waiman Long User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:10.0.12) Gecko/20130109 Thunderbird/10.0.12 MIME-Version: 1.0 To: Juergen Gross CC: Peter Zijlstra , , , , Pan Xinhui , , , , , , , , , , , Subject: Re: [PATCH v2 0/4] implement vcpu preempted check References: <1467124991-13164-1-git-send-email-xinhui.pan@linux.vnet.ibm.com> <20160706065255.GH30909@twins.programming.kicks-ass.net> <5783B6D7.7020903@hpe.com> <57846F02.7000306@suse.com> In-Reply-To: <57846F02.7000306@suse.com> Content-Type: text/plain; charset="ISO-8859-1"; format=flowed Content-Transfer-Encoding: 7bit X-Originating-IP: [72.71.243.165] X-ClientProxiedBy: BY2PR1001CA0064.namprd10.prod.outlook.com (10.164.163.32) To DF4PR84MB0313.NAMPRD84.PROD.OUTLOOK.COM (10.162.193.27) X-MS-Office365-Filtering-Correlation-Id: 8ef90d0f-c054-4547-8841-08d3aa80b716 X-Microsoft-Exchange-Diagnostics: 1;DF4PR84MB0313;2:wShmknDWOH78LaxY3MHZKgqMNiQgWCJAZNIy/BxZnycSjY2MbZ7t73eSo+5hg/UjpBGVECZRZ9BTD8rqNCSOcV4cIFhnpQB2o1snI63Sxxhiy9qnZRmzpBPcAI8VQSC00KvWU4XLsj0yxIXG2BTWc/qUhFwvb/txK1aPeiFCjNTbvGX6HCmObYnc3xX8uTTJ;3:P5YRMwaS4sf54/7Vd7xhnhgbo/hF7FQ0BLS6EVVE5NMkYA6DaOfi8pzRVgIpgN0dq2ZI4PCGRgAE9asyZwzHqh5mejNnQJNYkU+0ZHEUSaLlf9dIDEmUZzkniRRyaRSn;25:l0giOA0U1MBwjNFh/Ymrn2jA+o//j081pecyZctg/2k+u7F0FvJUP1on799XBtNcR3VWtLy6kndcYLmOfk9LIw+pJLVaiPj1AYyXVGq2itsYu7SFoebD89So0vQcKZTaEkx5oDo2qTuXlWJxv/GwMa+7bXOl3TIzoapRePZubj8ZElR/kCFQGqjOcQzakHyrt1QZKARUxSISjRukJhCFkv/2GCl5kUP9/s4M6wf3nsHAh3o3nSen0T7l05unvb7T5ihK11OdAH0DodBDbaW6iv4m2jbXrbo2Xk8+5bLKp1wUtgXELZOuHqIyY6WuDRZKXWkAoJB5bz3Br4+cdE7giDvk9A+tMDAuhsveiyvqeyd1cmIHA7jVZzVzcrP3hWaM4kxeZOwwitfwDUQzcTGPLvwa0/LR9arMzWBzNe3Pg5o=;31:S2l84zdEIvNkDGJlRopeaUwoV6GtWdXXekIZeZOXZxAiEBHO2a31mFJtBxQJNm7OR0JNFsE2mN2zlvFJDlrG6I3a6jBJ44B3Ydn+d7Lvk+AyIxLkqMWaoGVZEHTT5E3d+XnVxUX3SDRw5rXPNk6rEHqc7A38Ei+7P5sQGZCSzgus45uNa23Cv1N65lOvXngoFHexYI8XL3kOtUksdJfa5w== X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:DF4PR84MB0313; X-Microsoft-Exchange-Diagnostics: 1;DF4PR84MB0313;20:Mccr6bPuQ1Dap3HDTCCiKaW9PKn/zAlS7oB8GsXf7nk4JnbfGKLnsRMOso6qDc4LNhwEEQLzzFK6md3b8XOLhxwXWJEEV1vai08qxQitAPr00bsv3r5dacDL7ez9RKqG3dAEzcxLjQugwkFofMrmxpuGISHE6ilMjWan+lYDwk5JHdLFqKlR9d8klHx+V2vHYO8zceMgpDt0i6LzgpyCss0iwRvtcq02aPDvG3M/yImhBKyA7tgW4aPiKCITSceUpKpeo2QqNwz+lVq7+eAY09HHBDy+V4Ts5FLraKGAgfC8awX5H4homwBH2GMMLyxj2+KXRZrWkFgatXtXTkhp3w9DMnpLgC+myssJYic/q0sre4ARO6jCJAVralURSqRm8ejo0fMPeFNP4BH0doMazntSTi7wkIdtEnGkln8dhWgwv11eq48pmcZTbjgl51FP0N1G+rUaMYLGDzHk1Fts6OTJ/dgHLPQWgJzg0AZWzxB4dE55CrcHMrJOjek3onZO;4:O6shIvfiK86bNRa9TT2PofRvsGQLlgz+9/M+rq02wb2tnhiYelHLjWWqhMP1ESC2KUo31wqvx3AlG2MMQzgcMP+02JTR6VNH6bxxGKffkrmsuaC33mIsv+S8XisEfmNdl8L6O8sJ6LEIdX3RkrFZAUXbbQJUBkg6g4MX8c5+kGtIkeizaWsokb9IuQ0aHeDe9jk/CqVIYGh4uCMmwh5YZbnAJWt9rxyZkA+Jqh1uUCcKZZ3Vq5gCC+9CuXg7/giqgSaOZWdGmcwwOZU3/Nid81ufkUTQTKuZFOb/vzyEJ+0ogOaQKmIyXRcFoRAWrn64xZCVvhSoQgcU2ncQZiIljOPx31QeTxj5SNofV2fa3bMM3+RitI+3k5aLCOIHy/5t X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(601004)(2401047)(5005006)(8121501046)(10201501046)(3002001);SRVR:DF4PR84MB0313;BCL:0;PCL:0;RULEID:;SRVR:DF4PR84MB0313; X-Forefront-PRVS: 0001227049 X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10019020)(4630300001)(6049001)(6009001)(7916002)(24454002)(51914003)(199003)(189002)(377454003)(99136001)(59896002)(117156001)(3846002)(65816999)(101416001)(2950100001)(586003)(19580395003)(305945005)(6116002)(80316001)(54356999)(42186005)(92566002)(106356001)(23756003)(2906002)(4326007)(93886004)(77096005)(230700001)(50466002)(65806001)(47776003)(66066001)(65956001)(64126003)(33656002)(86362001)(105586002)(8676002)(36756003)(76176999)(50986999)(87266999)(68736007)(83506001)(4001350100001)(81156014)(81166006)(110136002)(97736004)(189998001)(7736002)(7846002);DIR:OUT;SFP:1102;SCL:1;SRVR:DF4PR84MB0313;H:[192.168.142.166];FPR:;SPF:None;PTR:InfoNoRecords;A:1;MX:1;LANG:en; X-Microsoft-Exchange-Diagnostics: =?iso-8859-1?Q?1;DF4PR84MB0313;23:+iru9VRt8zQikn/q2qTCMU2wezPblGn6Rdw7zDy?= =?iso-8859-1?Q?UDTdzB88cyG9uEhtH5aYFGWEdwS8THU8vfe7M2Ao40kU8u0ESn7Yx9IhAI?= =?iso-8859-1?Q?Mse7el2o3JKkbAoI76DWAX3b+bIIazxSbMTF8VRqlP/UC9/TQEAMrlQL9u?= =?iso-8859-1?Q?ZtJg8M8TQ6Axf0MxUd0Gaz+53j5TPb4SJYSxEDF1e1meMbjYx6x26CbGui?= =?iso-8859-1?Q?3MP7QsHlP11NpM52WJem3B7tq6pXf4h9kKGyywCCthO/MFp4H4xwg3bfMI?= =?iso-8859-1?Q?6lPXVQH1IMs5hX8bFW3Hrsek1EfpNXx+6jPOupGnLpoT/1xeTnejZlsreV?= =?iso-8859-1?Q?+ANEEdZKSUpbfdHFUR7dnh0dUQtcQwOutBHeTDjI2DpXcyaSsiPdjbSXdK?= =?iso-8859-1?Q?zX/BE3e5xSgVvV6nBPcHk0byz17r2MNUXbD4TE6Kq0Q+MjLd2R//oMcsE9?= =?iso-8859-1?Q?kR0faK5Bgdy7UifBSjmC4hdHqGDZPO5MjhnY+B7kYkdZhpMEuAjvxyJHSb?= =?iso-8859-1?Q?rTEKEwmDw7HNc9SJ+db1MQIB8Qg6cgr2jcQwP8D+IG2ZJ2o+o5iIZT2fVg?= =?iso-8859-1?Q?Q7xdF1zdYx0ojKhjfNOx7T7Do5fSob0dEeJaEVsp1SIAlLyH5f8tnD5wEN?= =?iso-8859-1?Q?oAfyE01pKbai3MgkWe6aXxFutawCKohWnbELwpX73etMYK46Fshfklkbu1?= =?iso-8859-1?Q?+3fmdPUis4kYHYMy65Fdpk13ChhpJlbtaxBnqW8v44ypPbufa61l14UiaA?= =?iso-8859-1?Q?LgiY17MbL4yIKi0dAaJTfb22J/RpPOLiss0UzvFoqTp3AXv893OIsEbM3M?= =?iso-8859-1?Q?EUlO5DzMdOOMkH0LDo16eqwGE+BDgcfJO/bm7tYQNO1szJMNZCSC4Me3lJ?= =?iso-8859-1?Q?7TxtXIkKcycksbGtq/SX91d1ttYdTffGHD+NlxNG6KKYTDm3bi2XXhMHjh?= =?iso-8859-1?Q?nFM5xl2J53qxA6v8fvbKPG7zKT183QwQUVfvogAv+QHeIt5SqtoINVdXd4?= =?iso-8859-1?Q?ehEiPNv+bsRCdM6YXmugimtgAng9wBpEYI5w/IY7HiWw9xj986Iho/XCFr?= =?iso-8859-1?Q?R7H09nMQqHoBpN8mM3MuJ7NIOYtZ5BKKZopd1FDUOAlxiaBZrBiMzQ11JE?= =?iso-8859-1?Q?8jFq10exP+NqqkbOlK9uwuOm6/NhYPMcLyvWJCrTPnKsnGXqibqIZQoql4?= =?iso-8859-1?Q?gtQrgMbMEaW0aUCyZwawNbJgsfhe+JtZ4Yl5OdjNz9LMxwG5SglLop5XcV?= =?iso-8859-1?Q?lFyOD/In3niAHan1x9cUj4Nr3QOZxscnPMJLO7mmIuY6/T/3BdRULz4YMv?= =?iso-8859-1?Q?vHsVS4Z3CKDbqwn+4uzfrjjSQemOsBmNbmJ3ewFxwd302Gx671FjuaEBjJ?= =?iso-8859-1?Q?VYdUm5Ady+jOL2vQVGl5f1Rkr1AD17CkWHFZ1rEeKierQQFtdhg=3D=3D?= X-Microsoft-Exchange-Diagnostics: 1;DF4PR84MB0313;6:JI5N6bC+cJMmebB+HECE2IVm1qNvZXzuKH/5diTwUwyXCBuwu0sHVwKAIqQBV44FFso/27bL5H0wyDoSmvEatssEtrRDHKnjULoGWT9dbXg8llWFSMZwztNoD7OJDmRaW92cGUqPzWnOsHSt6wcJGVRnF2GwkyhoxEAADdhPmzS8TBLWK4+uC/JONdtMW5mrUnIrAPGdK2R+11rNP079F2PrMVe3KCboKsyjID47UfxCI7VkCSmokWzfw8GSq9Da3aA3A6xquoOEqOpm6WbwQmOk+/g7SeslAsFj55oalAI=;5:gPAXqPWBUDgEBobiUMd+fl9cNR1KloT1aKw3VLV70LuXqu4NOnd3tgHRtwtRkphAezPWGNYbacGURsb7coL3SvpG2K3eZcMF3j9sjhBKBWOgm+u6wb/7W38c6kgh6DNOeGLRln+qQFcvalzOZrv5qQ==;24:6wfyD2KHrYqb8soKfqDST/1/9hZnBlC5voEDjumudOKIbSyDXWZr7hiuLDzfTlq87bRSydTfc8E0fNppVbLly9jYPfgGMqwqu7PWozifxSU=;7:qnl99Gg5uIjt3ufJfIqZQNwXs1q2TpiRZX/FEr26lK0tkHhbvWDVzPqoxK+UBpedzrBeQZ4qQnR1pSTEvZZG6jC4c9VBfH5HjV6IGa8fLnCUWPeZ+jSeQ3N9SlmK98z2+rZFFa2n96BxRESa/zeCS/7PBWF0W2TXxda8tlir5TYBwtoemJLYwH+cZBkqACzUTkbP9jxc+Mg2MISgQDUHppvFHVAjR6xjWYECHFsDTKh+6GAK/dKU05RXD2CEoC3H SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: hpe.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Jul 2016 18:16:55.7231 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: DF4PR84MB0313 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 07/12/2016 12:16 AM, Juergen Gross wrote: > On 11/07/16 17:10, Waiman Long wrote: >> On 07/06/2016 02:52 AM, Peter Zijlstra wrote: >>> On Tue, Jun 28, 2016 at 10:43:07AM -0400, Pan Xinhui wrote: >>>> change fomr v1: >>>> a simplier definition of default vcpu_is_preempted >>>> skip mahcine type check on ppc, and add config. remove dedicated >>>> macro. >>>> add one patch to drop overload of rwsem_spin_on_owner and >>>> mutex_spin_on_owner. >>>> add more comments >>>> thanks boqun and Peter's suggestion. >>>> >>>> This patch set aims to fix lock holder preemption issues. >>>> >>>> test-case: >>>> perf record -a perf bench sched messaging -g 400 -p&& perf report >>>> >>>> 18.09% sched-messaging [kernel.vmlinux] [k] osq_lock >>>> 12.28% sched-messaging [kernel.vmlinux] [k] rwsem_spin_on_owner >>>> 5.27% sched-messaging [kernel.vmlinux] [k] mutex_unlock >>>> 3.89% sched-messaging [kernel.vmlinux] [k] wait_consider_task >>>> 3.64% sched-messaging [kernel.vmlinux] [k] _raw_write_lock_irq >>>> 3.41% sched-messaging [kernel.vmlinux] [k] mutex_spin_on_owner.is >>>> 2.49% sched-messaging [kernel.vmlinux] [k] system_call >>>> >>>> We introduce interface bool vcpu_is_preempted(int cpu) and use it in >>>> some spin >>>> loops of osq_lock, rwsem_spin_on_owner and mutex_spin_on_owner. >>>> These spin_on_onwer variant also cause rcu stall before we apply this >>>> patch set >>>> >>> Paolo, could you help out with an (x86) KVM interface for this? >>> >>> Waiman, could you see if you can utilize this to get rid of the >>> SPIN_THRESHOLD in qspinlock_paravirt? >> That API is certainly useful to make the paravirt spinlock perform >> better. However, I am not sure if we can completely get rid of the >> SPIN_THRESHOLD at this point. It is not just the kvm, the xen code need >> to be modified as well. > This should be rather easy. The relevant information is included in the > runstate data mapped into kernel memory. I can provide a patch for Xen > if needed. > > > Juergen Thanks for the offering. We will wait until Xinhui's patch comes through before working on the next step. As for the elimination of SPIN_THRESHOLD, the queue head may not always have the right CPU number of the lock holder. So I don't think we can eliminate that for the queue head spinning. I think we can eliminates the SPIN_THRESHOLD spinning for the other queue node vCPUs. Cheers, Longman