From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1755438AbcH2RN4 (ORCPT ); Mon, 29 Aug 2016 13:13:56 -0400 Received: from mail-db5eur01on0083.outbound.protection.outlook.com ([104.47.2.83]:9248 "EHLO EUR01-DB5-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1753029AbcH2RNw (ORCPT ); Mon, 29 Aug 2016 13:13:52 -0400 Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=cmetcalf@mellanox.com; Subject: Re: [PATCH v15 04/13] task_isolation: add initial support To: Peter Zijlstra References: <1471382376-5443-1-git-send-email-cmetcalf@mellanox.com> <1471382376-5443-5-git-send-email-cmetcalf@mellanox.com> <20160829163352.GV10153@twins.programming.kicks-ass.net> CC: Gilad Ben Yossef , Steven Rostedt , Ingo Molnar , Andrew Morton , Rik van Riel , Tejun Heo , Frederic Weisbecker , Thomas Gleixner , "Paul E. McKenney" , Christoph Lameter , Viresh Kumar , Catalin Marinas , Will Deacon , Andy Lutomirski , Michal Hocko , , , , From: Chris Metcalf Message-ID: Date: Mon, 29 Aug 2016 12:40:32 -0400 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.2.0 MIME-Version: 1.0 In-Reply-To: <20160829163352.GV10153@twins.programming.kicks-ass.net> Content-Type: text/plain; charset="windows-1252"; format=flowed Content-Transfer-Encoding: 7bit X-Originating-IP: [12.216.194.146] X-ClientProxiedBy: DM5PR18CA0005.namprd18.prod.outlook.com (10.175.218.143) To HE1PR0501MB2762.eurprd05.prod.outlook.com (10.172.125.16) X-MS-Office365-Filtering-Correlation-Id: d9260855-cb22-4556-8198-08d3d02b39c1 X-Microsoft-Exchange-Diagnostics: 1;HE1PR0501MB2762;2:uBm9+GUYfEtKkha8uufUxr3T5HqgDKijrIu+DD3lTgBCQlJa1C5HvMK08ahbcfzluyObEeIaHLEaQuNepuwxpeYhHqVbh0xl7oYPNHyQ0qKgP8clpplZ0pMlNx8FDDxb8b7eMgcYGUSEiYQby1Mvyw+g05STxswtN93z4HXXKx9ezP/JPPsR8SCjGD5UJ+dU;3:V+qoiKVl8FSNXUa9npn8Yg3x54GNe+hvL0xuUy0aYfHCKF2Prom9AEgJ7sIhMCiAoYgltvtq3PebxNwiOTrP1N1bY6C4nL3DD74DMXiaYB8YuEM2NzsL8LKrq5F0vdH/;25:cRvaxuIij4mrPrkhfaBvydMXP71yrjVg6HcONSeN5P7gNoqMOKvCEmdM5CVGq61Z+riM8HahkkoLSYNi4Ditw5wHex7BuwciyU91/ekJYudEBf6XtKC0QHPl3hyu7lr64eg1padatBvj0PIkskg02cESklE428f/WTO0YmC6nC9yCQrU8EQXk87Kjwyehp1VQ6RH+Ihr4ItSJJWj9mBO4zynl9nF90kYpsdj49jj9/4UuM93P3zXWWv3D5Kbn3uUDhd3/cMThP6aW2RiCdUA16ZZu9fz7Sqtw4zZ1t5Gbzh8NJWXUXJSNsdgwRiS4PSwNEN8G+HvltKmtPQuA9aCIRvA24m9T0VWWyp20Y8po5DffBsB8AqlpdU2VR1+cBI8mnQrcaYsEP41YRtyjozJjbNQnTqh7X40Gkktvq5t1/M= X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:HE1PR0501MB2762; X-Microsoft-Exchange-Diagnostics: 1;HE1PR0501MB2762;31:z9qXcId1shYu3Dqt24P31JXYBpYNq/6znAKb6QwGOnIlXJwxfoXYWHzco0X+6c9FygYfF+5fpQ7KgqYoymXqT583dVsBU7Z9zd9fYa9AUWearwrh4fYtoVHKxGnKAoN18oPk+A2EAJ0QEgHvrfBQR4JRhdv6ltZT5RQpLYNbPIbjOj0H633GPn3oN9tnPzYuqHMUWyI6G2lxkYx7x6B9ile8Y+0nUa1CG6Vo6Y3vovk=;20:7zy0mtKRApJkwy4KRbJFEvk+9qcFRi+nDY3COEkaj5qzqROFhuVMvhP+BCzpTVuwF6ZgwghtrlIoxODf4cVjzLBajf4UF/k+R4vF/Q2oayk+M3HoemV+ySUHVfbfF3F+y9YQuSC0DR3AhThmgXLK2i2/fvbG030EMevziIxATdyNbeXUUcF9OA+P7UvqtwR36wvbMzp52XJjI0h9/AJ0mB8lvpMO1FHOvR63NiQ0rDMPPjNtkK1XmWaxx5KIuTtqh8t5PpGMF4xWglIQdQUG21UppflLjnjqfa1wXBrgYKYnKmA6HiY0o7U77bSEVlU7c+Qe24awQAyQloAJJcncg/xumb79tZdqUoODg60fcBcWIodn9w+wC1GjJDEO547HUSAG92tv6S3LV6In5bohMsYlR2vZjgNZM9Pp4m9B2XAeTIbLugmfm56KglYX1eKtsM2rUv0/TxF8TP8GchWFBpYKFTwtVH7xKFpshZ0/JXaK7j0hT4dRG3X+KcuUKmPB X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(171992500451332); X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(6040176)(601004)(2401047)(5005006)(8121501046)(3002001)(10201501046)(6055026);SRVR:HE1PR0501MB2762;BCL:0;PCL:0;RULEID:;SRVR:HE1PR0501MB2762; X-Microsoft-Exchange-Diagnostics: 1;HE1PR0501MB2762;4:5BB09LUrKyw8e2Ck2oau4Vcirf35wtUPAhgNkVj/x4fwEOqXmsnoLRzsBq/A/50S+ViI+nGJU2fTAmXKOtgXe1IIbXoet8I0HTNOBIDtA1bA8YLaw/Rd8nKAaSTpokxvRPAyh3Ydsn/NeturSxEhT4hKM0sBFO2cw3eABvKWE5CLRQLV6UfISJcrCp5+7/6gHqL4M/s9/rJzu8whN0VhxhFQ3Muwp5lrFojelhBwcW+H1SHQwaRhxErQpc3Ttcq0/K4KaolyU6dDRQOQiE7X/4FIXxNkeDUnP9cN0RDg7YAPUI8bgAwLz4TwYrUeJFIzXHm/PnFTrUuCv64eZnzHha0JcIaDVGf+QS9s57GIltXsyZ4erb/auYIULUZbQdf6xcNKCb2u54+9TlC6ivQFTbXv2fwRJg8H+S6R/cq+sxr9IWhTJb11aZvNDtQZG6lMFraK64VoMEcmdx5su6zzOg== X-Forefront-PRVS: 0049B3F387 X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10009020)(4630300001)(6049001)(6009001)(7916002)(377454003)(189002)(24454002)(52314003)(199003)(23746002)(4326007)(586003)(101416001)(47776003)(6116002)(81156014)(65806001)(68736007)(50466002)(81166006)(42186005)(65956001)(64126003)(189998001)(2906002)(66066001)(110136002)(8676002)(106356001)(3846002)(4001350100001)(97736004)(7416002)(19580395003)(105586002)(86362001)(7736002)(54356999)(7846002)(83506001)(76176999)(36756003)(31686004)(31696002)(50986999)(33646002)(230700001)(92566002)(15975445007)(2950100001)(77096005)(5660300001)(305945005)(65826007)(18886065003);DIR:OUT;SFP:1101;SCL:1;SRVR:HE1PR0501MB2762;H:[10.15.7.181];FPR:;SPF:None;PTR:InfoNoRecords;MX:1;A:1;LANG:en; X-Microsoft-Exchange-Diagnostics: =?Windows-1252?Q?1;HE1PR0501MB2762;23:dJH5DpfRFvXp0qiLiS0UukCg+PvIl0dFI+t?= =?Windows-1252?Q?8Sj+HLOxgmN2GyiOxh4XOmVgr4E3Pw7WwlK5Q0EsPXkxCLdNf3HpmyDr?= =?Windows-1252?Q?+KJ4BpA/itwVhhDdGDYS+Ou0sQ1K7Rb9SY/nKZ2JxwzY6KxEW0JIPDEh?= =?Windows-1252?Q?wraCF+pkQo84euUO8WfqaRzFy1CZOGnlQAu8pt2VXdKIFaoz+sm/T24j?= =?Windows-1252?Q?bFD05tCUFOIL98sibMQjwiAKZsfEmFNW0HmlWI5ZEWdXbhnnFJv9ULXo?= =?Windows-1252?Q?9z1aTdWvUbS/4brGZXBd7Gi4CJMCZTsb4zSNLCv29pO7HBXL3GsKqxfS?= =?Windows-1252?Q?w+d4ZtvWXUnVKjw9f6CM8vvi6OqgvHU9BbSJ80DFN6VjoT3TZx1sZUqC?= =?Windows-1252?Q?iBjaLWPJRdI9Xwt5nAFb2cALjyR2Lc3t6x/TyLofnRRLHOP3yuRZFLoE?= =?Windows-1252?Q?1Tcf9pt0NJj7Aowx9JF/PhoZvpZKFrEpF2uk/VECbMjVSVp6jBDCV37s?= =?Windows-1252?Q?5kwjJ3Qjc2EbhRBC+ZUtcAz4y5OvuppzpgriYr6327TxQ3A8c4LPAeDa?= =?Windows-1252?Q?D2VZOLT3LkjTE0V4uxa6Di3Rn36RBMrQclEMi9SwFRCPbelo+R966qHq?= =?Windows-1252?Q?hyz/BbYzlJrV/DqWLUBN7WFi4IaufQWcCIiAYdZVNvsSXE1PJn2DTtmR?= =?Windows-1252?Q?5aVf7jtcyIELuANyJW/NT4mPvmVyhQ2WXPon6I7VLhsIi2wUFbzZo52q?= =?Windows-1252?Q?9JRwa+rzrJfhnZf0ofkACoddhyIrut3rVDdU0Guim5wJLb03gMsEuh52?= =?Windows-1252?Q?j8ecxr19Ca/FYymc+gmAB1jm/0xq636+D4AByW2jBYTPG/eewOqTzzq0?= =?Windows-1252?Q?y4UAmdtYMhGQx7CCVu+nxujBU7lrH5DKfyG/Hoklx5fXYOT5nQWFEGcD?= =?Windows-1252?Q?CGpp/LOezF1Cxa7kbEGAqx80TttF9gzBCm6KzebHf6GhlFJogx5tnhVH?= =?Windows-1252?Q?ZATwsk0cP/6Ft0dyOk7E1mq2dXA16+9w6g3DrWLvLihS5UDE8bPrRbE2?= =?Windows-1252?Q?s8nTDWcwjWtJGyDxJzcjqam694m47+2knkDPlQCIG2xKZvVn+u0pSqCi?= =?Windows-1252?Q?Hql73tdDyeTyV0TXg/VqK9rNWawjeHg/dP5AEw6U3LOOsKZ+Ky4kzlfY?= =?Windows-1252?Q?KHSrN4HTZjXffwMPI//xtUEB6CA/Nq2HBlcCX4NegQQuqUuAwjt+t/Eg?= =?Windows-1252?Q?eBe+jpjaxSfAjpyBJ9KDf4RH8YOC6zS9zAReW0UuKs7hw5sHN75DRq9X?= =?Windows-1252?Q?WjMN/HUV2ysPLEWehc1yIq5hUvAyAlg1VH4yyIxuRbKSouCjuFusZHZk?= =?Windows-1252?Q?di6VlScweVSQ2O48r36yW01IaoVXLQxkXgXhye3/bfYjkpEuA4gHTVk0?= =?Windows-1252?Q?fo4lfFsZhosMvJiAR3JXKQwvY7uag//u2MbvV2ZCLhw=3D=3D?= X-Microsoft-Exchange-Diagnostics: 1;HE1PR0501MB2762;6:PiqncKJDHecwNV05PhXnF7F3yVbCw1hDRZ/wafDsq4x/u/4QlDx6GUQPtcyg4lgjCb7SKXIoX1/kn/Y/27JFPdkoWz+LzJe7QuLDxl5Z5+CQUwecB5Q8Wv/+S2/2lgkFnN2QcKpSf7RLNoBVSgVhdbrzzU5ySeSrfT9VmhaieTMpCUUXG0Ndb1NZr/9AfCIclAq9gGw4TsKJSCthu02QIwAqIsYS9Aj1TV7ov88GitsNeriahBISVUmIjpJN6+2zPlZ4EGK03tuYt9CC7cyoaqzG1anCLS8I8Ukl5JFVPSHvcFG6UHrWDSvtgXKrJHa7625n6t7ikG1xy58x9gnjIg==;5:JRWu8Yp1ZOKObzfDdoYUbPX9WuarEtuPyWeYsYUy+aWIc9RZ0rC3YtopEwKpVcQa4VSxRXmxjI5FXpKtPjl4KEayQZ2JTT1UmeaYIO9+TzA0SDv6VPoV1ryjRHoxZXoRUKYZyMFBK5GmifBkSdCIeg==;24:OPIbNyXUT6ZQMVuOq9HndH5zSee1ySva52Sv6iASDqm0scDXTLxy1A2gUbKpb9wIczl7yKA+is7l1FegaiDL60rm/ENBlIknboHm+VT8XBI=;7:s66fczfBNnaQTjh/Us04HtGX5peXeLE5LdsJCOQlbpEsSPRgTUWdj8N9Rc3399ICJwfDaZbwf+P0bM/IApFDRPiBwOInC2DCBUtjQpwptdrXllf6tjDeAkKXEqdn36Rv0cUeQ+UMIWwE6VrvGb09aotDKEfPPn3270A7Mlv70Fx3kcU+HWFPFyQkTXPoLR0b2Abg2yCi3XuDIT5ai/AWhds5t+uCpu6PJ+lAPhkzKn9V70LekaggxWm7Z0wmnalj SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Aug 2016 16:40:41.6489 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: HE1PR0501MB2762 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 8/29/2016 12:33 PM, Peter Zijlstra wrote: > On Tue, Aug 16, 2016 at 05:19:27PM -0400, Chris Metcalf wrote: >> + /* >> + * Request rescheduling unless we are in full dynticks mode. >> + * We would eventually get pre-empted without this, and if >> + * there's another task waiting, it would run; but by >> + * explicitly requesting the reschedule, we may reduce the >> + * latency. We could directly call schedule() here as well, >> + * but since our caller is the standard place where schedule() >> + * is called, we defer to the caller. >> + * >> + * A more substantive approach here would be to use a struct >> + * completion here explicitly, and complete it when we shut >> + * down dynticks, but since we presumably have nothing better >> + * to do on this core anyway, just spinning seems plausible. >> + */ >> + if (!tick_nohz_tick_stopped()) >> + set_tsk_need_resched(current); > This is broken.. and it would be really good if you don't actually need > to do this. Can you elaborate? We clearly do want to wait until we are in full dynticks mode before we return to userspace. We could do it just in the prctl() syscall only, but then we lose the ability to implement the NOSIG mode, which can be a convenience. Even without that consideration, we really can't be sure we stay in dynticks mode if we disable the dynamic tick, but then enable interrupts, and end up taking an interrupt on the way back to userspace, and it turns the tick back on. That's why we do it here, where we know interrupts will stay disabled until we get to userspace. So if we are doing it here, what else can/should we do? There really shouldn't be any other tasks waiting to run at this point, so there's not a heck of a lot else to do on this core. We could just spin and check need_resched and signal status manually instead, but that seems kind of duplicative of code already done in our caller here. So... thoughts? -- Chris Metcalf, Mellanox Technologies http://www.mellanox.com