From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751120AbdAMQjf (ORCPT ); Fri, 13 Jan 2017 11:39:35 -0500 Received: from mail-sn1nam02on0089.outbound.protection.outlook.com ([104.47.36.89]:36146 "EHLO NAM02-SN1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1750923AbdAMQj3 (ORCPT ); Fri, 13 Jan 2017 11:39:29 -0500 Authentication-Results: spf=pass (sender IP is 74.221.232.54) smtp.mailfrom=sandisk.com; fb.com; dkim=none (message not signed) header.d=none;fb.com; dmarc=bestguesspass action=none header.from=sandisk.com; X-AuditID: ac1c2133-4386e98000013ebf-80-58797d825c26 From: Bart Van Assche To: "linux-kernel@vger.kernel.org" , "hare@suse.de" , "linux-block@vger.kernel.org" , "axboe@fb.com" CC: "osandov@osandov.com" Subject: Re: [PATCH 08/10] blk-mq-sched: add framework for MQ capable IO schedulers Thread-Topic: [PATCH 08/10] blk-mq-sched: add framework for MQ capable IO schedulers Thread-Index: AQHSbbuXZWDHNybfxkyT/kG6RrQC2w== Date: Fri, 13 Jan 2017 16:39:21 +0000 Message-ID: <1484325542.2679.3.camel@sandisk.com> References: <1484170803-9311-1-git-send-email-axboe@fb.com> <1484170803-9311-9-git-send-email-axboe@fb.com> In-Reply-To: Accept-Language: nl-NL, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [172.28.1.254] Content-Type: text/plain; charset="iso-8859-1" Content-ID: <4C9C98C573530E4AAC9C08DF7A07111C@sandisk.com> MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFlrDIsWRmVeSWpSXmKPExsWyRobxn25zbWWEQfMmRov/e46xWexZNInJ Yu8tbYvLu+YAeUfOMDqwekxsfsfuMefPQWaPzaerPT5vkgtgieKySUnNySxLLdK3S+DKePLm L1vBa8GKp5+bWBsYP/F2MXJySAiYSOw/uoapi5GLQ0hgCZPExLtdLBDOJUaJP+t/sYNUsQkY ScyesAcsISJwgFGi6ckfoBYODmYBfYkrn/RAaoQFQiWmNu5mArFFBMIkJjYtYoWw9SSm334D NodFQFXi1+YmsBpeAUOJOe1XoDYvYJRom3yfDSTBKWAtcWF/GxvIfEYBWYmW19wgYWYBcYlb T+YzQVwtILFkz3lmCFtU4uXjf6wQtoLE5xX/2CDq9SRuTJ0CZVtJ3N31GMrWlli28DUzxA2C EidnPmGZwCg2C8mKWUjaZyFpn4WkfRaS9gWMrKsYxYoTk4tz01MLDE30ihPzUjKLs/WS83M3 MYLjUNF4B+O/De6HGAU4GJV4eE+EV0QIsSaWFVfmHmKU4GBWEuHl+wsU4k1JrKxKLcqPLyrN SS0+xCjNwaIkzit/YHeokEB6YklqdmpqQWoRTJaJg1OqgTH1hwHjhoP/ts5nUlFfwrg/ddHu 5G2r3+3+HjEt4VmXXz+Hl9a2YqlX8zgFMmRuPBc2WfMmxfi4x5wjd6J78+eenSm4nd/06MoA lQeP1exyLmkaR33w9FTIj5W/1/eobuJPm/tWG8vVOVq+RnMuvmTousqDvzd4+ZeqRSd//Hj0 /of/suV/ztxTYinOSDTUYi4qTgQAPhoBPb8CAAA= X-EOPAttributedMessage: 0 X-MS-Office365-Filtering-HT: Tenant X-Forefront-Antispam-Report: CIP:74.221.232.54;IPV:NLI;CTRY:US;EFV:NLI;SFV:NSPM;SFS:(10009020)(6009001)(7916002)(39860400002)(39850400002)(39840400002)(39450400003)(39410400002)(2980300002)(438002)(24454002)(377454003)(189002)(199003)(377424004)(97736004)(5001770100001)(2906002)(7736002)(33646002)(8746002)(4326007)(36756003)(230783001)(38730400001)(8936002)(229853002)(626004)(305945005)(69596002)(2201001)(2501003)(356003)(8676002)(81166006)(86362001)(189998001)(81156014)(2950100002)(102836003)(76176999)(68736007)(2270400002)(3846002)(103116003)(5660300001)(6116002)(50466002)(2900100001)(50986999)(106466001)(54356999)(47776003)(92566002)(23756003)(106116001)(11001)(7099028);DIR:OUT;SFP:1101;SCL:1;SRVR:CY1PR02MB1288;H:sacsmgep14.sandisk.com;FPR:;SPF:Pass;PTR:InfoDomainNonexistent;MX:1;A:1;LANG:en; X-Microsoft-Exchange-Diagnostics: 1;BN1BFFO11FD026;1:mIRdznvoUMm88XYQgEeFjhew7tAp4vTZ3FWYB5D85n+s5J8rwTj+AfLiRuA6bFz2fCS8JsNjNn+Y0UFgvzZlDxQwx62mmX1WUTnfBSYJIv3ONRTXURPx78Ea5RiJzllEoH+ZqE/LMJlQ9P1dC6ltT1AtX5L7UA4ppDJODw98l7o7huv6IsyuON6GKq1Ax0WYcub9yIXn+5ZLpO8C5O076ie4SyCDcDPfEEzM5fjcwqYlkNHhqiyCp19j+5rhnz5uC4qezFJIt1uexu+iZ5tbGayu0/pRWF9rwkK7L/0t+6U12z8T/FrTToRt8cSU6iLxtkZoudbVIq+rsvovmooWOjacV1S5hU4Z0sUr9VD9qtUpmEPk3s3orHMQkv72O3voEDl43pwJTQeHN6QrqYzl/f+G3NR4l4LchDgjeheH39rSBe21ldTgm9Z84WUUEm27QfY261irWQug07rzUOLox6gC6biLZOSIbWuKEfikIiCOIRfK8c4OfFYKKRgxqFkfrtjByqdChifw8zzjdxmsmL3JArAwWJ6pt6HmeHJGVMM7NPgAuA/nJFzKRJntm5SW X-MS-Office365-Filtering-Correlation-Id: e9798250-89f9-4c18-a243-08d43bd2bcd9 X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:(22001)(8251501002);SRVR:CY1PR02MB1288; X-Microsoft-Exchange-Diagnostics: 1;CY1PR02MB1288;3:loHBZkRYB289rLQytxJkhmADUNI4UvZzsSR17A1ndPQxjjKCOOjBRlZAapnpTJeGQc+T9JmlCDCHPYv+yZWqRCZj9QrbPyeLAXEr2CRd2ZQX77xSGD0tydqXi7iKSbjI5/hrmzB9T6J7R5pCO5cUbqKSNO/NqbYP3yy+dZlAL3z0KelsrGEtg/sQRZuEz6+NoktBE1oxsZDOboIb00aFoOe0i+r2qqHWSuQL6pL9UtW47gCrWivt4qAwFgcHWZag6vsaABNS/5PdNH4yX9NPz+Qia2q0gwv9sjkILhJP0TfM7WmvrjNPLbz8KhxWsj0XRHWd3f+htRljsx83hLP9D/qDei0sHezYDvErL1K+Bw/JwFQHdr0r9mK44pWEE78QMHQXgvSjlSf8hOEBm8TDag==;25:EALXzLgVwxu3WQ5zimM/nY7iBpWNpXic3XNyYYtYfHG0KgMYs8fm5hXEJP2aCKI2of1TiEe79F0tIrL6IujKPmnMV8/rdCYqXdMfhJXgZFC8jDH5vK3zfWcKKfd1J8LFTmNl1UD+uKHTmmNJMyjrry6F9HBsL+FK+6TcRdTtWvILpWPfmwtD4CCW4CkSXxouv+VmBNY7JabFAj/6ve5fGhhaqlijxji2PDY/8A7bADhOxHpB7e16+NyIOm9zr2Y03RC5YKQQEd2dhpO+2OdxRpjAmbFMl5dgjsRPZKoax3KZ17xq6VVxgchC60Ep23veKMNRqzODPEicXrlWc3CCcpl32hJjv9KIi7l/4vmVMHqmxVRSdEpAuzDTmC1HF//6eMgIHx0z/gKo9dQHKjkIDecOQh8ig8W9vGbczeu817cvmStFrJKqPxQefZWnABOCOFNtdntDRXPdlqEVBMPQLA== X-Microsoft-Exchange-Diagnostics: 1;CY1PR02MB1288;31:KNiK/kRjBbI/+9PnK9p6+bnj++6C2S15HyRhCArAHlvh6R755SPKhYnfKyfQJozFJ9sn0ra5OhM0HZuwr0RD4TnhLM9uwylIH+d+JfTtrlO+p3CmQh2gunHLZVFTHCgYasTyL4J7gogWy5Y76oUPeT3DP1/qR5FIjnz86Mw4lTv6Z1Gkp0qe5jFHShVHOmjFvwg/HSwSAn5/ECd4raW7k7NoPLD/c4YTE87Bb+w1YFAEM+jqIdQQbaDooBuyV7+ueM5eIvv/ZicSfGhKAUwaxdvchnH8h2tziA3BDdrC4nU=;20:6uACnMYFgiRTatvYSo7iz3Mlk34U2ZxfC2OnrJYcR4sHzcr5ykIiKcwz/ltlZbrd1Gmpu/b0q3d90hv1KROv8Rty58Ozkz9L0ORTdo44GvfdJRLiv7vl/qxyXl/EL9sy/rJ5kvebY7H2ISkx5pqtsBPpAq7zRua70HHBP0DeV62Lkb117hmOh7/EcLXkv2RlP8eo7Si0dajugskcCrchpV3/QHoofWs2CUmL6fuw589SrTj9qlkVmxrLaQkjVSpwAu42wjxE17zfB781DWmn+i8hbnEnZ61ORJi3BjL2psLCbuCSiCpwI/CXLrKHXa8ZPKcm2YNMPgFZKXh0MWtCOnKouUEz16Y0v+Ot16sMQrkbiGiOEJ7T6eempn039QSLbzo95TBvNurPjxC9hqJXIQccUpaWMrN/bOvjuzPohwWqGzRqI9GBYsoiQuv8Orn/TwCXqyIg/q7Zxd1CZnNKyBz1DCkutAHpE85OV0IWOBh8k/EpcAtiIB7oqpbAQFhg X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(67672495146484); X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(6040375)(601004)(2401047)(13023025)(13017025)(13015025)(13024025)(13018025)(5005006)(8121501046)(3002001)(10201501046)(6055026)(6041248)(20161123562025)(20161123555025)(20161123564025)(20161123560025)(6072148);SRVR:CY1PR02MB1288;BCL:0;PCL:0;RULEID:;SRVR:CY1PR02MB1288; X-Microsoft-Exchange-Diagnostics: 1;CY1PR02MB1288;4:iUOZWS2lWrTtgZplElJnsa/BD7H1bAaqTQ5QcO2nk1df2XpEPaErzXfK6lVLVxup8/SBMJk1qSKl4nRVXqUcP5LnyKlQJabohzo4enr3BuDved6L5NEABiqyWKjAPhAQuVpWC/aJpgXe+IhA+L9nCCtXNRbcU+VYm07LT/RluAkH1uEEIgCxFyJ4mpUN6ttJsHfXsGsruJrFUk7VpVC5gkuEucMJIQFeE9uDi+V2AqzoeEiqGm1n+zmbZEaZSCCwwBz0DUFU4rj04F8jsbtvdtpMTujOaslgTCWBiJExKSFZOq8AszhXEOFxiAaGun/Ye4GOVnlGSJWvxXVTaQa4ABJC9mZVvvWdXrVMNMKTbjN7t40m3LRXI8ja4ZCkHgzTW6ZYqG2m6g7qT6XBvfzPZAreQbL6dlbAkR2jj+8HzHQrtkypEyVpkHp7oCRzWbpSLYUKAMR3/Q/pYqffbQqY+z2mgAsfH5R3d8oK2ydSZqOtVbIp4ZO6zLYCK9zl3oTfPXeOwPQlF6H59+hh3r3s0BW557VDnznwawM65x1JUim8O99crvQZlGzQXDgZs10lA0V8fzj6MsnTEkIp0sIjQzfA6jFWLwr51Cwl3xUvHJXTQJueC1QhDCs10yMxAv/b4TI554Jrc0F5v9ajIVU2WzfDftBMLWEcD5Tz/l0J+09zupZn8QNYk18+AFyUSahCWph7GACrktAzImrNNqaEqHquKTSy2Fzpx8x3mXxarL0= X-Forefront-PRVS: 018632C080 X-Microsoft-Exchange-Diagnostics: =?iso-8859-1?Q?1;CY1PR02MB1288;23:Dr4AAHMF7XmFGB0ap9Zr1b0sRzvjYE5zmHqdTFS?= =?iso-8859-1?Q?3ZPVjUul8DF94XgrWeO4G6YolpumsHynisbKaNymqVPR2HK3I9ZyDtj2sV?= =?iso-8859-1?Q?FJgGNcj0qNabE6nrV2KW7w+a/PFHwI3wOeKDx07CWMusXPYugfe7rZqbiW?= =?iso-8859-1?Q?FLRvratgtsawjdaatQsMVB9ghCrJFrRfJ8VV78JLupnyUnKw0EWhmb/N2r?= =?iso-8859-1?Q?+v9yfaxvrvVgZL+nV9pTZQ9JXX8uwXAPZ+2FvULflCROc2eNC0cOwIHp3l?= =?iso-8859-1?Q?+Y8h5Zb/SmMQDI3hrxixtZKKdNRfTVGRnxoFqsM1VXH660wEGoRQVK4VVv?= =?iso-8859-1?Q?6Olfb4ifk2z8bYYOCtGuSSKbyXEtv5oL8fBrzwTgEsp2Au2DR0freSWkyc?= =?iso-8859-1?Q?sIW1EnXq3WE/+tuH/AeXwGwLC6G64Nr+mSbNZW7+wDB7lX1m0zV+MD+p/4?= =?iso-8859-1?Q?fjFKwiUq2w9XFsRsiegwv86rg36bF4vdSxxlJkXDO+XeZODuFpmSJJ5s+j?= =?iso-8859-1?Q?oSLEj22H2hf0W9PBJPEfzUug5hnrzB2lTW5pjKSzmuwQNSvQvwvQZnSAhV?= =?iso-8859-1?Q?LY/pWGLi5e3mkKjcisc4frReo1ZqAYwWbkFla9d2K3oo4VT2uiLlt5opqb?= =?iso-8859-1?Q?aqPZNyLv2mxiW+k90n4aklC93pJ36NGT/3PDXoltlkaWhker3MizWWAqcj?= =?iso-8859-1?Q?sAo7QdklfTTDZhaaI5BkZDLF7AIIouKRYMwsvRDy4pLn4gZrPg7PzrHIKk?= =?iso-8859-1?Q?0yTTTZmsOBi5JPpSrVjX/xmAxKRn5kzxzTVoE7WA4VmyOeMnbTXCggefqJ?= =?iso-8859-1?Q?2cP+7Vv9g0bemcau8YLvRYH6166nLpOZBZQ726rx+bzQ2qlNIL8kp8NaUM?= =?iso-8859-1?Q?R3sn8CqDCllOOE/fVKwSVVNlhQB1VFY7w1QbK0p9s5YiXPp5S5leDxgO5s?= =?iso-8859-1?Q?Nj1wNEYAVY7r11ylWabsyMh2zHsrsYHoITZsG2C3BqMW7wTcv0mEHfmVSR?= =?iso-8859-1?Q?7H4Ma/QV9eFnFuVm3VRCs7w8V3FfMf4b0hJJqKGTDHlo7Te0D9yjHU4IK2?= =?iso-8859-1?Q?w70gcFe3eCECePWhgQVQdtoDhaFp7WuVJGYWwkR4+U8vpjjfUw/4mz5ZN3?= =?iso-8859-1?Q?TX5v8oGBn0jpjXqfU5c66Wn6zhUblBR/o7pFVDCp4+ko/jGjW/jNsIKzHj?= =?iso-8859-1?Q?mloMn9rqY792hAIsVNzzbGuxt4AbYW9+pt1rm+Q5Te9kWKh51+mgBpSuRa?= =?iso-8859-1?Q?FBNlGfJ3X3pBAAtKgXX1yv+ZYoMcQM7WEh7feYCbG//10XMJaaCyRyTCzB?= =?iso-8859-1?Q?0anwGwbZB548r+ZnDiZ7ky7F9GSdpvcmamIQ6yM8neGKpdTRU+gf7jBYWE?= =?iso-8859-1?Q?v8sepmEz64LpI+Qd2JvBmd+/amI0lklPTF5dMw6DswaNyiZV2UA=3D=3D?= X-Microsoft-Exchange-Diagnostics: 1;CY1PR02MB1288;6:Or1/cB5Pxikr5A00JaXJKTCyMUI1QIBsCjYtytb3JPLKhuJWUmM95GWd4A6YqWJhfeqBZJnMn5E+dJvX/6Rypyr3cTn/WnKw8UZXPdnlVVa5s0M7FdES245SMIWgFnjmNNCFKy5kwf+k8vNXIVigMOVBXovIaph5Z02aKjALFbanjMIt/cYYBXQLThxQKFhW/1nDZ99YVA8Pfc47hJ5A87DBJsmSxLC4TBSbsEyYRR2RRz/0QhxK4GWyeE5Uu6+7qhIw53gH9+sYq9BAErt6OD3FRsWOCW5Vwt3D1ry5fRsFc3ssE5dJglG55dC7lkwYR1hza3Ouvwd8feh13BmOOkJrZEfBXaaePhlc9cdPIpRYrAmqXKm8KRP6YejkV4Y1oQbENELbxhWPFnVYZLl7iIa1ir69o+MWWbwKZudfvM81s31QGtJFUHjnqXAE4hJWjPYBA4PC3+L7obT5NAq9LQ==;5:7xtcnJsZAYI4CeQrg2SMyO7cyuzQZz7kp3rhblGNEIUImjHnOsFriN3MrOzkUAtBn2BnDXKrLVM9VtSXwhehGCdmSgVbx467tI6YWqk9KFaS6Eh8Kl5lWZrTM9F0quANPRoZ4w+1Mkfc5GEczx4F/AD7ZvupqLq7pnZpxzRGi9c=;24:rAEsACDnKh06HR3dKQEOa85KwUyS49bFkYGcIThYDAq60FjxtWeN3mHqRvUAqSYfIEvEU3hxGzC3Oe81N2Wxqy57M8oJCg7VjvjqrL+ueQM= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1;CY1PR02MB1288;7:C6qkZh1ioRq30bNsMX+0wpneBNujUZRbIvrnOAdrUWFVPmD/FaLckJqz1mwu9LDnNut3N3edl5jV5oEw8vaCTX5MlWtbyCU03081pYjiIAOC7V1anwPSBH+ZuJ3CnEcbKjPdljBhFR3xa30Ww0CsKmCPTQ5ZAqwyyGnANeoT4GsqpbJWS5X6VruQ1+DX2UZenTHoM2yUDWyfreHc1ur65F9BP0CJY1lhhqv6sIxDkyY4sGjf8YrhUb6bC3EuVj6T9b2j9damSDn7Mjt8TOANy/0b1qVmctteJZSSCWQYxYW5+E2lPg+M8wcKDKN57nlo61NaPZ2Ub/iqk5Gl0JLX4VUFg7Pkn2BQN23oIiyAQw/lxiF2A8LQK1tU+hweaY54EF5rasFHr84bb7hmtmb1Ck8siQjIESkD4EQGsG8BhrpkOhc+D2m7znn4CPgLGzCJeFQWl+X0LXK65yU/d1yZzg== X-OriginatorOrg: sandisk.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Jan 2017 16:39:25.7549 (UTC) X-MS-Exchange-CrossTenant-Id: fcd9ea9c-ae8c-460c-ab3c-3db42d7ac64d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=fcd9ea9c-ae8c-460c-ab3c-3db42d7ac64d;Ip=[74.221.232.54];Helo=[sacsmgep14.sandisk.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY1PR02MB1288 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by mail.home.local id v0DGeUbN030531 On Fri, 2017-01-13 at 12:15 +0100, Hannes Reinecke wrote: > On 01/11/2017 10:40 PM, Jens Axboe wrote: > > This adds a set of hooks that intercepts the blk-mq path of > > allocating/inserting/issuing/completing requests, allowing > > us to develop a scheduler within that framework. > > > > We reuse the existing elevator scheduler API on the registration > > side, but augment that with the scheduler flagging support for > > the blk-mq interfce, and with a separate set of ops hooks for MQ > > devices. > > > > We split driver and scheduler tags, so we can run the scheduling > > independent of device queue depth. > > > > Signed-off-by: Jens Axboe > > [ .. ] > > @@ -823,6 +847,35 @@ static inline unsigned int queued_to_index(unsigned int queued) > > return min(BLK_MQ_MAX_DISPATCH_ORDER - 1, ilog2(queued) + 1); > > } > > > > +static bool blk_mq_get_driver_tag(struct request *rq, > > + struct blk_mq_hw_ctx **hctx, bool wait) > > +{ > > + struct blk_mq_alloc_data data = { > > + .q = rq->q, > > + .ctx = rq->mq_ctx, > > + .hctx = blk_mq_map_queue(rq->q, rq->mq_ctx->cpu), > > + .flags = wait ? 0 : BLK_MQ_REQ_NOWAIT, > > + }; > > + > > + if (blk_mq_hctx_stopped(data.hctx)) > > + return false; > > + > > + if (rq->tag != -1) { > > +done: > > + if (hctx) > > + *hctx = data.hctx; > > + return true; > > + } > > + > > + rq->tag = blk_mq_get_tag(&data); > > + if (rq->tag >= 0) { > > + data.hctx->tags->rqs[rq->tag] = rq; > > + goto done; > > + } > > + > > + return false; > > +} > > + > > What happens with the existing request at 'rqs[rq->tag]' ? > Surely there is one already, right? > Things like '->init_request' assume a fully populated array, so moving > one entry to another location is ... interesting. > > I would have thought we need to do a request cloning here, > otherwise this would introduce a memory leak, right? > (Not to mention a potential double completion, as the request is now at > two positions in the array) Hello Hannes, Have you noticed that there are two .rqs[] arrays - tags->rqs and sched_tags->rqs[]? .init_request() loops over sched_tags->rqs[]. The above assignment applies to tags->rqs[]. Bart.