From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1758040AbcLTPqi (ORCPT ); Tue, 20 Dec 2016 10:46:38 -0500 Received: from mx0a-00082601.pphosted.com ([67.231.145.42]:45644 "EHLO mx0a-00082601.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1764413AbcLTPqT (ORCPT ); Tue, 20 Dec 2016 10:46:19 -0500 Subject: Re: [PATCH 6/8] blk-mq-sched: add framework for MQ capable IO schedulers To: Paolo Valente References: <1481933536-12844-1-git-send-email-axboe@fb.com> <1481933536-12844-7-git-send-email-axboe@fb.com> <4A1753A0-121C-4A01-8821-6CDD99C98896@linaro.org> CC: , , , From: Jens Axboe Message-ID: <24dc9b88-f846-f6e2-0568-f5b612575651@fb.com> Date: Tue, 20 Dec 2016 08:45:16 -0700 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.5.1 MIME-Version: 1.0 In-Reply-To: <4A1753A0-121C-4A01-8821-6CDD99C98896@linaro.org> Content-Type: text/plain; charset="windows-1252" Content-Transfer-Encoding: 7bit X-Originating-IP: [66.29.164.166] X-ClientProxiedBy: SN1PR17CA0031.namprd17.prod.outlook.com (10.169.33.169) To DM5PR15MB1196.namprd15.prod.outlook.com (10.173.209.14) X-MS-Office365-Filtering-Correlation-Id: 9b0819c8-c733-47e9-ad58-08d428ef3539 X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:(22001);SRVR:DM5PR15MB1196; X-Microsoft-Exchange-Diagnostics: 1;DM5PR15MB1196;3:Jjb1nwHqv05hzFcDwJx1/+G0pwH3nXSbAVvD4PaPYUyYYjpFDIOiZmJxJ7Pj8ygSMK4vn94iOK4GtjU5S+lqH1ZkvRU5x+BWkcGFjoQfwvj26ZfIP9R1c9X9uJUhtjg+pNQkidNh3D6Y+iGncbdhyEATue/SStW6mucd+abmtT4HjdsAlY0lA1XFNG7k9dySykILfrp8a6RbUSdgeKHThQiGNHaBQsEF8LTl089OKYuWA4AbuXAHLxWLYk2jTBBNhzLvNtFg/wr9xy6EP1Zcdw==;25:mr8EAATbjt6ABX3eTqe6Z/rWBnzAMnReAnfQKJtnaDdOECyoLd2Bd57/2y5J9NBnikWFadAI+hLnVrkY/KpJI2hF11GzYeru45kXHQ3vBEEOrytQPix62k7yNn3f9i+zWGY556TX+Jqw7vcbZZ8TaKEUEmz3kuNyBnoT+uuCo32Q6X/tW7H5YnZQSqzWXtinRrURltOFJNiD4YqeDmL5NAkBoSid1CmdCfHUXxgksuYtu34S09ZBgy+0CUlV4II968iDPtA+TOshT4sN4rI7eG2P5t7dXkgMIktHugdXeZKZub9eJISBxJru5tp9OGKj8tmc9VeNhweQwErbe3wYTMNFVTE25UDLWbNfTE3rOCMgoENvOX/ecvC3548VZRYGa5BPo6YH3A5rNh3QK/lHX1w5jhSBCA6icGI4aPSbpFqEzWyXgZdmO5euyCJcWemNneUCYC7+w7xV5PM9Y1G1AQ== X-Microsoft-Exchange-Diagnostics: 1;DM5PR15MB1196;31:ylxicsni/d9SSMiZRNIb7tMGfkjgJZJ/JfvZV+269BHjOsX4Amo69uXFDyvNcBztLRfktPj91GUopVVPcbtrn46tWdraj42MzHB7NtfD1EXEl8iGBPHPP5sDjRMPn30+WbCgm4TQ5DPyhQSwj10RSLSuvFYctkb1nFlJ8ewkdY8tZwFgY7fakW01efrjwvfVk9u8Cv7YYhjPOHhR/2xibZWT/NYrwOh3nk7+XHKzQrY/YkhOj7+/3l00xFezTSbuBLH7S5Jt8F8Hx+T3OeXbZA==;20:/5eiqgxs9/LgO6N0MIvXGYt9Ljx6fXJtGUpIL8zkikDRYTNQcFHewD976MFAA0qHT6at3l/uslYAUxXv7Vf2+pR+s14KH7dnNzPrAYH9hrDa9tJB3uobOLvNOhAZn/MVytYuPTnkdvGB2+2bu8RUtu55bzWHNNxQgakApPufUohixJSeEl9/T8XEDjQCRlVXyZLyvPQRfTUqJE/wjEEMyBQjhGBGbUailIwSy8fDY0VyGdHJ5IE/zk6shoVNEt2BKwGjlBNIK6bWHZEUj9W+Qmy+GrYHdkRNkTqtsSvJniyyWO14x69oPpA7xY3koVpK7NcpQQVgc2JAHmIgCvEeGMB/L+Sye2FLhOxE5MfXURky0BIbf44ZPxUMJD0t5lEXOG3rF5BorWiuiot5IBlNzBploAbraSIR22mOWI9z+hv+VVYDxRYeTEXUY4YbsiGpM47wLXaFKN5hnxFyEYEgKttZixcOXwrffKCc0tl0kYfQp24rwevIc+WY/a7/113j X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(211171220733660); X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(6040375)(601004)(2401047)(5005006)(8121501046)(10201501046)(3002001)(6041248)(20161123560025)(20161123564025)(20161123562025)(20161123555025)(6072148);SRVR:DM5PR15MB1196;BCL:0;PCL:0;RULEID:;SRVR:DM5PR15MB1196; X-Microsoft-Exchange-Diagnostics: 1;DM5PR15MB1196;4:Y8wXP6mBf2ea0X19FV7jBYM5Z/GaVrHVVC4r88Wvxb+seErQsG+RafDy5dx8Vk8REUPqm7Xg7UWp4Geo9oOO7frsUCYN1b9WINuxnuRtn+HIiZlg1DpbdjVRxW5LlvyBWsJ+uRE0yzr1CEb4wQzlnCL0krd4DUZpVD56x5XN1B9cReXxR8nefI66kaHUk8T3gho6fFH995tag/dlinVlM2nCXEQr813d50slI3eep5MMtYw+3UD2Q0c/bK0CLcKIkjD2gP/NoxWnQcxL/PJnRa+wBuJJPfzaoiC1/knZHHFWhqjsaZjWUyz6jt2dQiL7xg7sglLh586dxakEDiGAOqCu+KNRsoq5anb82xdsenwr2qWmtEgLd0zNfyiNYoMbjxzYW2grIJ3Y6xrGeMKHXML5Qv+ny22MYBTFZTdYaR6Jau+WpnBWT0wYAdnQFzy8N9oh9WdXB3NMwXWC2pyDKDv5kdDM/tvZpMYqyuuusXwiWgtVS42RjSIh0a4tAAbc1K2I3bvEI8JuiRT7grdu3sm6iMJUADUAxUIqbcdcHKo1Ol3J6bSEo3p6pv2leSxBwzfYmz/bGTcEnCi0egHCZrpkNo/DgQth7PJul4zeuIf0t7efBLk1y7zY1KXn3WYj X-Forefront-PRVS: 0162ACCC24 X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10019020)(4630300001)(6049001)(6009001)(7916002)(39450400003)(39830400002)(39410400002)(199003)(24454002)(377454003)(189002)(77096006)(36756003)(101416001)(42186005)(4326007)(86362001)(97736004)(65826007)(2906002)(117156001)(65806001)(31686004)(65956001)(110136003)(8676002)(92566002)(33646002)(66066001)(47776003)(31696002)(6486002)(230783001)(5660300001)(25786008)(90366009)(23746002)(76176999)(229853002)(3846002)(38730400001)(81166006)(54356999)(6116002)(50986999)(305945005)(230700001)(4001350100001)(81156014)(68736007)(2950100002)(106356001)(83506001)(105586002)(6916009)(50466002)(64126003)(6666003)(7736002)(189998001);DIR:OUT;SFP:1102;SCL:1;SRVR:DM5PR15MB1196;H:[192.168.1.176];FPR:;SPF:None;PTR:InfoNoRecords;A:1;MX:1;LANG:en; X-Microsoft-Exchange-Diagnostics: =?Windows-1252?Q?1;DM5PR15MB1196;23:tFp9sKXqJ99V0iLhcx+rxYs9gbJkwGKtW5Jl+?= =?Windows-1252?Q?nFt2ztYLKFx8NM27/yy9nyWZWT++a3j3CSPb69K7wcZHK6L0b++BPFv+?= =?Windows-1252?Q?6nl2P2/Faa0LbjmFCFOmsksz2a4211w/tTwzDFkLiavJTF5NwLK0xbsj?= =?Windows-1252?Q?9wdev0bBdmDA67AzXrAD8LDC4uRMLGxWBFeRl1IUxUR3qWbDzqFO6BWR?= =?Windows-1252?Q?mAhmFsBWpxY/DY3lxoKyPgQXzmolkv8yFp4Xq225YwEbppgkrwAhzW8F?= =?Windows-1252?Q?YSmBjpx+mfRDY7/yPRTMY3gH8/l6ImOB7jOHEqa4oihWSnY9+atrBTlT?= =?Windows-1252?Q?BPCm0YSC8yGhbshCR7rqohmvTVz/Ml/YrlJOlkoelUNxt++g4ntabdvE?= =?Windows-1252?Q?8T917jugOxQn4oZH95iBCchHRysFYvXPCECVk8Z0v81LgCWKpLvpWB/k?= =?Windows-1252?Q?DHD9QfSIEBMu4SgyvtXJxBrKfwiBLunOQHZBISYHjoT/i2/PdFzlpbbf?= =?Windows-1252?Q?+qPZ9Zp3VFsBxVmAfTrZhkSyhv2lNMM3wjUtFTUdU5IA1T6qhAsFouix?= =?Windows-1252?Q?tq20r1CAxhV63DSK1/8+XFCnUlI/gTOHEiD/pMj0kjw4oVdZ80B/zEoN?= =?Windows-1252?Q?rQstHdlz7lnMfGp2fsePkoMv7eN7FBQpON3q7JOC3m1IJCb3d3qgS39z?= =?Windows-1252?Q?kKoGN8ozJXtFWBzzT2LU1XVlkDYfqDpYMeE7BG5EkhPrwhfgCNjLSrQO?= =?Windows-1252?Q?twAg7nI2AjW3TNt6tNiXCAbQZoRzytUcAXB9/1nq/lcvw+9UEdM/1QqU?= =?Windows-1252?Q?ye48kBoE19LYByxGer6VdMH59S2YdtG3G5aLA8g3XP1HwkTT/a64WdrM?= =?Windows-1252?Q?jjZr9UlRqY/pSQyzjv6XjB/Q/wYlnPI5ssbDxoUqBBlqnsMBxzRLzeQb?= =?Windows-1252?Q?+XSiCVOAgoTWujH6et76ZW8PaBOAzbfxV+74tLIVupSRPg8MLbfvq8+5?= =?Windows-1252?Q?719pvqYr5vzkrI2775nNzk9ri67nLkNP8lN6UysZ5FJBkC26rKVc9WUk?= =?Windows-1252?Q?XBlzCHIM1XTKJl7nc8SXMXYZi8InhqUojW2C+u2nMp7cM06YAHTe44jS?= =?Windows-1252?Q?O2OD3rwHjyhBkaxQOP5HrlcWoWvo2XeoMZLnTck7EqQldXzqKvABwg7j?= =?Windows-1252?Q?u2E6HCv789wvPkWIUwkpBmAVVl1tvf6ph1A0xJaZp53hk175XNEIkDmg?= =?Windows-1252?Q?VPmwd02tNCcCB6kstg7Ibfg6pQxDOAxx/1g2yCgvGqjdOPqirKuHrCt4?= =?Windows-1252?Q?Zz2C4OyfND892MBg4hJvdd1EcixArIq5Lr61xjNolR//27P3Gg/4Q+uu?= =?Windows-1252?Q?DrkZBSpU1r9k+qaysFX6cxm7l2UweGzGlstNOv6MLyFENWoM0kpnHGCO?= =?Windows-1252?Q?GxnX0bhIeB6lhJ5TZlhyB3BeN6IKtJ+Mqnm8Tx7+BTUAdDtGnQ/lqAfL?= =?Windows-1252?Q?Oqx72wMAz6Qv7e6PrGAcRmb0ME646qNr12A/NrgSsg2iuUzciCs8ltrU?= =?Windows-1252?Q?zNvGhf3xhPau+M=3D?= X-Microsoft-Exchange-Diagnostics: 1;DM5PR15MB1196;6:s9eLPbRgu5CBXfLMJpmoUmfgrg7hSXUECMlMnBcxnsMpjdxqfKhHPSDfC3Tv+a62mHeAdcPdhIVU6ZPoZmFqRlDuUCXYfncPdrXR1pajS+bCtqas29lBcp6+4Fzbxs6pRVG3sQ1mE1Z2fezL2+B4O9cvExCtHAiq+udJXJH/3DCGfumIwqO3QlghbG29CipQv4mwCZ3iXFbBgGyEdORJUZbxok/oWWHSuaIKWTpdNKZRtO1qcIjqNPUSZvcsqVF7AHZuA0PTQYqyN1iMZ0lndpkIBhv9HU1KLjN/HLzhKT6+ZdcHXrJggvOsS1G4yobfhyF9lAHyqmwz7IPYTr6MN3n4+AlzV61ogv87V99M8uW8YhllUZq0eq0DvWGpB4aFoo++TLpukjf3cyihPkRNSN15r4vHEmE0Yfs2ULHggw8=;5:DGJHloIxFW8cRHqSm9z7KxbVKEPxJZLolJ6SbB8JNQPAyRyZi0CxU81Fe9NZNrYbzNo6Pdj1AaqJ6ssx/Eczdb3/MkvegFsrzCprYb0EB3gVqU5oOvHuqycV6JJQMoLSfULI6kABXPfkL+lP4mkWRy29EMqUy8FvVCdhi9xN9Iw=;24:/a2LozSmGekMxov+r5Uf2i7MtXsEVE6RxIUcFnu1CE8+um79KxyLoTb/0Zaav951HLpFWCoCwGuGtKP8PxpJWACovK2QsRR5kQm+yw5yCWY= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1;DM5PR15MB1196;7:3LqvT/RMIj92hzjnDiBxTc1pIOVEisfDqbyry/rcSCA92IRYvjpxdZN76udri3aLLlE08i8HlzWnfOkOCtU1e1L8DP54jrZ9CvIrJophUV9i7/HQJzjQKzI5dUe7ngvpRZQlW5uKEJSE52tSi9SMQB8IR/PkHXLjQ2Av9hbYSDzA66uuShggrUuJ41QlfXDdoYLCX5x2Ue54m688eWqHROA2KthmLbFlq0+L7K8eVIqYafshqCHU0aFqMi8n3uKTcvjB89RZTKAylnOxe73ekrq/iixvjIN/VjkzWcbSVhUcODZKU3SNAQ4mkKapZXAwszAZX7ZdZoi87kVlBkH9kPyHjMqlOn2oqF3u9kaKYPELt9kcIW00GsWjesxDDprqDED9HClV27A9MfNe+TDNcz+peVd9yZNSGKIEFSe+B5vqnR7k8Cp2Zg6APAEHqDIGs5XigH31mERsjsLHAyBUBQ==;20:LkUeKM28EWVDTDn/go1WYecwGNbJehKauQUjZJhG4ceoc28su+LfOoujA57cQ8UGJ48WDLvyMpxBK650CGc0USy8yRPgsqS90krC91DggJwwf9yRWruE3+iIDR++L9tfXiMKzeB8AjW54pVgF//kYTqFXk6p8Fb1VuyLsdoUAys= X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Dec 2016 15:45:21.2492 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR15MB1196 X-OriginatorOrg: fb.com X-Proofpoint-Spam-Reason: safe X-FB-Internal: Safe X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:,, definitions=2016-12-20_10:,, signatures=0 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 12/20/2016 04:55 AM, Paolo Valente wrote: >> +struct request *blk_mq_sched_get_request(struct request_queue *q, >> + struct bio *bio, >> + unsigned int op, >> + struct blk_mq_alloc_data *data) >> +{ >> + struct elevator_queue *e = q->elevator; >> + struct blk_mq_hw_ctx *hctx; >> + struct blk_mq_ctx *ctx; >> + struct request *rq; >> + >> + blk_queue_enter_live(q); >> + ctx = blk_mq_get_ctx(q); >> + hctx = blk_mq_map_queue(q, ctx->cpu); >> + >> + blk_mq_set_alloc_data(data, q, 0, ctx, hctx); >> + >> + if (e && e->type->ops.mq.get_request) >> + rq = e->type->ops.mq.get_request(q, op, data); > > bio is not passed to the scheduler here. Yet bfq uses bio to get the > blkcg (invoking bio_blkcg). I'm not finding any workaround. One important note here - what I'm posting is a work in progress, it's by no means set in stone. So when you find missing items like this, feel free to fix them up and send a patch. I will then fold in that patch. Or if you don't feel comfortable fixing it up, let me know, and I'll fix it up next time I touch it. >> + else >> + rq = __blk_mq_alloc_request(data, op); >> + >> + if (rq) { >> + rq->elv.icq = NULL; >> + if (e && e->type->icq_cache) >> + blk_mq_sched_assign_ioc(q, rq, bio); > > bfq needs rq->elv.icq to be consistent in bfq_get_request, but the > needed initialization seems to occur only after mq.get_request is > invoked. > > Note: to minimize latency, I'm reporting immediately each problem that > apparently cannot be solved by just modifying bfq. But, if the > resulting higher number of micro-emails is annoying for you, I can > buffer my questions, and send you cumulative emails less frequently. That's perfectly fine, I prefer knowing earlier rather than later. But do also remember that it's fine to send a patch to fix those things up, you don't have to wait for me. -- Jens Axboe