From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from esa4.hgst.iphmx.com ([216.71.154.42]:36693 "EHLO esa4.hgst.iphmx.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751023AbdCQUjX (ORCPT ); Fri, 17 Mar 2017 16:39:23 -0400 From: Bart Van Assche To: "tom.leiming@gmail.com" CC: "linux-kernel@vger.kernel.org" , "hch@infradead.org" , "linux-block@vger.kernel.org" , "yizhan@redhat.com" , "axboe@fb.com" , "tj@kernel.org" Subject: Re: [PATCH v1 3/3] blk-mq: start to freeze queue just after setting dying Date: Fri, 17 Mar 2017 20:36:58 +0000 Message-ID: <1489783004.2826.13.camel@sandisk.com> References: <20170317095711.5819-1-tom.leiming@gmail.com> <20170317095711.5819-4-tom.leiming@gmail.com> <1489771915.2826.4.camel@sandisk.com> In-Reply-To: Content-Type: text/plain; charset="iso-8859-1" MIME-Version: 1.0 Sender: linux-block-owner@vger.kernel.org List-Id: linux-block@vger.kernel.org On Sat, 2017-03-18 at 02:32 +0800, Ming Lei wrote: > On Sat, Mar 18, 2017 at 1:32 AM, Bart Van Assche wrote: > > + /* > > + * Avoid that the updates of the queue flags and q_usage_counte= r > > + * are reordered. > > + */ > > + smp_wmb(); >=20 > atomic_inc_return() in blk_mq_freeze_queue_start() does imply a > barrier(smp_mb()). Hello Ming, It's probably a good idea to mention that in a comment. The implementation of blk_mq_freeze_queue_start() namely could be changed in the future such that it uses another atomic operation that doesn't implicitly perform smp_m= b(). Thanks, Bart.= From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751343AbdCQUjh (ORCPT ); Fri, 17 Mar 2017 16:39:37 -0400 Received: from esa4.hgst.iphmx.com ([216.71.154.42]:36693 "EHLO esa4.hgst.iphmx.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751023AbdCQUjX (ORCPT ); Fri, 17 Mar 2017 16:39:23 -0400 X-IronPort-AV: E=Sophos;i="5.36,178,1486396800"; d="scan'208";a="4108149" Authentication-Results: spf=pass (sender IP is 74.221.232.54) smtp.mailfrom=sandisk.com; gmail.com; dkim=none (message not signed) header.d=none;gmail.com; dmarc=bestguesspass action=none header.from=sandisk.com; X-AuditID: ac1c2133-99bff7000000c960-e7-58cc48ec3f8a From: Bart Van Assche To: "tom.leiming@gmail.com" CC: "linux-kernel@vger.kernel.org" , "hch@infradead.org" , "linux-block@vger.kernel.org" , "yizhan@redhat.com" , "axboe@fb.com" , "tj@kernel.org" Subject: Re: [PATCH v1 3/3] blk-mq: start to freeze queue just after setting dying Thread-Topic: [PATCH v1 3/3] blk-mq: start to freeze queue just after setting dying Thread-Index: AQHSn145Mk+PEYhJp0SjoHMAamSd2g== Date: Fri, 17 Mar 2017 20:36:58 +0000 Message-ID: <1489783004.2826.13.camel@sandisk.com> References: <20170317095711.5819-1-tom.leiming@gmail.com> <20170317095711.5819-4-tom.leiming@gmail.com> <1489771915.2826.4.camel@sandisk.com> In-Reply-To: Accept-Language: en-US, nl-NL Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [172.28.1.254] Content-Type: text/plain; charset="iso-8859-1" Content-ID: <79C798E6482AA148ACD4533E19EC7525@sandisk.com> MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFjrIIsWRmVeSWpSXmKPExsWyRobxn+47jzMRBmvfMlv833OMzeL0hEVM FntvaVtc3jWHzeLX8qOMFu9/XGe32HLlLpsDu8fE5nfsHjtn3WX32LxCy2PTqk42j/f7rrJ5 fN4kF8AWxWWTkpqTWZZapG+XwJWxfck1poIO1ooTM5cxNzA+Ze5i5OSQEDCROLZzMVsXIxeH kMASJonrj7ezgCSEBC4xSuy/wgliswkYScyesAcsLiJgLPHp9TNGkAZmgWlMElM67zGBJIQF QiTu3v7PBlEUKjH9zkRWCFtPYsG+SYwgNouAqsT/BQfBbF6gofd69rJAbL7NKHHiwGKwZk6B QIkphw6yg9iMArISi6e3gC1gFhCXuPVkPhPE2QISS/ach3pBVOLl43+sELaCxOcV/9gg6vUk bkydAmVbSTy+fYsdwtaWWLbwNTPEEYISJ2c+YZnAKDYLyYpZSNpnIWmfhaR9FpL2BYysqxjF ihOTi3PTUwsMTfSKE/NSMouz9ZLzczcxgiNW0XgH478N7ocYBTgYlXh4bzw5HSHEmlhWXJl7 iFGCg1lJhJfJ/UyEEG9KYmVValF+fFFpTmrxIUZpDhYlcd6Y2VMjhATSE0tSs1NTC1KLYLJM HJxSDYydW+p80+bYBNbExmuf5T/swXZ9tpP3pCXbHxmvUfs0tcLoURq39+I4Cya7L1Onyc5n jfHPelhwwik4IuCklsIMU+0T36o3GMiekf+i+1ZmSyO3QesEgUMmCVs38q58+lhkXVayKdPR Za9Tn/iq/zXRvJx7+9c/vtfszN9Mf+7yqLo8N0m9w1CJpTgj0VCLuag4EQAweSgi1AIAAA== X-EOPAttributedMessage: 0 X-MS-Office365-Filtering-HT: Tenant X-Forefront-Antispam-Report: CIP:74.221.232.54;IPV:NLI;CTRY:US;EFV:NLI;SFV:NSPM;SFS:(10019020)(6009001)(39840400002)(39850400002)(39450400003)(39860400002)(39410400002)(2980300002)(438002)(377424004)(377454003)(24454002)(199003)(189002)(9170700003)(229853002)(2906002)(7736002)(2501003)(305945005)(5640700003)(33646002)(23756003)(50466002)(38730400002)(81166006)(47776003)(356003)(8676002)(8746002)(103116003)(6916009)(8936002)(5660300001)(2950100002)(54906002)(110136004)(2900100001)(3846002)(36756003)(86362001)(93886004)(6246003)(76176999)(50986999)(102836003)(39060400002)(53936002)(189998001)(2351001)(6116002)(54356999)(4326008)(106466001)(53546008);DIR:OUT;SFP:1102;SCL:1;SRVR:CY4PR04MB0742;H:sacsmgep14.sandisk.com;FPR:;SPF:Pass;MLV:ovrnspm;A:1;MX:1;PTR:InfoDomainNonexistent;LANG:en; X-Microsoft-Exchange-Diagnostics: 1;BN3NAM04FT013;1:gSdDfurS/LiRge4SCWz+U5T72oyj3fmDbvhJ4W+v44NCwcGoVNUUzP1WbpBFmcgv2P1b/iwrgwwCCdilxYa4/BrKHsgvEKJpooAL9Grqblk3GfuDLZTXo9V0YJ5hftdThKzUWiKs2BbQ5TuR4sQzgGShsUDammuQEIcAwYpTXOFpIUUzK8675Cyqe3BOyZm5Ve825905S55pAIe6a6RVJD/gjmMxJM7RThlZlc/rUyYmtY/sm+NPCKdfEQ5ODhHVYEdXjgdt4mPT1iudncXj4yHP560P8Eq4txJDBBX1w1ceKP9dkziWnHJ/dhgOPrcOGWrnIOLjTP9vIRx92S4NFzkw4Z08+ckSndSURK4KkxMI7CFT2XJ+Gj9KB6UcOYPgL8e+WkdPW7m2luXPYCXUeucu71z8SkWvypw8ybNm0p/ce7nZ0qMrQj1LpGEyH0ECFh2iRB+rsj2b2tstMQervd/u/rwEc2S6Hb1lthR5GhsKNywJsboQ3AGlDYDDRnHI6BaXlLODT96Ihi2zAAILeQ== X-MS-Office365-Filtering-Correlation-Id: 3774b034-b636-437a-f8f9-08d46d755e69 X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:(22001)(8251501002);SRVR:CY4PR04MB0742; X-Microsoft-Exchange-Diagnostics: 1;CY4PR04MB0742;3:mvhTcPZEqOaXuwgL4JCPV+InDE5vk5x8Gcr1u/6KaXo0cGPYJeh8bAsihTWoI3W7+FSiSl5mzOtsXl2x6hS3+Pqq3W4t8OapUMQkN24Sjq5ForZmVA0x+LEx+grVr2t1pq1paQdalFcqribLrdAuYibSf40kCUNEm35ybyyF9H6P0ICGHerfFKck2VL+fBJtgTmzIHNjYj8pdWRQ9ATcV24UMSSYP+0Oa/DgBsY+3FHSRIN8LY+1SO3mwNIVR9mP+0VM2gqKfaok+5SKFPmyaVs+vRC/kCDVhwddu3D2e5daV5iS1sd1yLlzK7xr+1MGT6GXUUVqoI7uVAXaGYzkdafLDnqMOk5yhrraYn4QJqE8DZGIXJcxbNqn0u0Pawwdx2Sga06r0DcK5x7hREF5XA==;25:zjYQAN2QcOtg5J2KeIQC6ocSeEHeETcJzuzNOYX/PYq5ED5ZfMhd2eDkPuLMTmWuDiM0ADnoFbUwsOmWKi01xuEu9mtvVYWBM/+4hpmkex/6yiVKS/DyOYrx5lmDzYbeAhqTvlPL9lkpZIpy62K5vcymBfVS6wNOu7lcd2b6eqpA4ZZjESaxDw9td9qGUIZrdOLKcClQLz74ZS6pxNn7eKE0XNUUk6GpiXxc+e9jWO5+rUDu7nvWMuYGTwEVg1hQmOnDWyUwzEFLgd9Kv4zT/3pYb1pCpkx4mxG5tg9UxZaKyenzaXPn7O0r7Hf77nvr7M0Uw3C7EzDdEvMteX+cwFMJoKv9nxpiJ/V2jzDhHWRzhUYCFwo6wi7fYlduzLkNBEoAaocTSMBXIIOWOHLfa+a1ZitduGd152sDXi5yRqjw0e454GXCmliMQVV03LufJslJryy2mTSoxLsiibx0AA== X-Microsoft-Exchange-Diagnostics: 1;CY4PR04MB0742;31:gSne9+/1BIDMF17/3wEcQ6fsWZl1SKXDkxHrhE1eTWrazYr9vk5zdExcAEI9V0awDIbHQo/fjBl+ppFzx6Yd59ApDtWcV8R/JgIjiKBJGVDigBXRcSfTLmYE/s8R517pTNL7ysUsDjOVPuRbjYWlYMXjiQ76fwnrpqck1cm8pXsvfTVzGDLgzL88wUxB1bEHnpAoMEbG3vlp/2BVAjQO/9OVOEXvmu0mRBKRUWS91uYwMa76Q/b+G/spZW6p7ktMUbjf7PUU4BsI2RaB/DOm6Q==;20:KfLI3I0NPpUncDOrHUOoqdSFp5Uwme+N7VVvPo04mnv0uqmW8lG+hDYrhej01V+stqacgHBirWOgmgrTA57ZM6i0wZgIy87wpFslDMkpd1w/oAi14Lik4K5M70rMkzxN3AnExqxGjissMuQTVBM4vBl2iDWJ0JrJTzZ6Rr60Etgmdw0xLgSDyJ5mUsGaRHaoKMV7jnCBNJpWbye+fNSkHpQbIliUo1riLs4LLEPEafG3FVTXvgB+LfOrxnrI9S/RlX9mRbByr+ncfNYlbQHG7HAho2tuxvoJHHxIQ068D7+SrRpY63fvvWNq7KKe/oj206lptkk39IdRjq0HIVyenjHbrhdBvsHmdDKJFZ0oUUBHvtYdI0XOosIMJPgnGHta6wL5PVkpxH3iZDRa85yE/bqa8f2weWkzL9ciCxH8UkJv6s6GMamE9RaWo0mk4R1bChkx2+yoPh5BOwZltpd/80Y8GPp1LR0uFOtK5ArHZotBomIIz32Ot6/AVU8Q3h8D WDCIPOUTBOUND: EOP-TRUE X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(6040375)(601004)(2401047)(5005006)(8121501046)(13018025)(13016025)(13024025)(13023025)(3002001)(10201501046)(6055026)(6041248)(20161123560025)(20161123562025)(20161123564025)(20161123555025)(20161123558025)(6072148);SRVR:CY4PR04MB0742;BCL:0;PCL:0;RULEID:;SRVR:CY4PR04MB0742; X-Microsoft-Exchange-Diagnostics: 1;CY4PR04MB0742;4:B5v1dlPHex2quMw0ybCK8enFuPsfZ3MFNbij7dAr56Nh8BSLipV+HSldZCMf+/XDPNWrh77N+GXClNBK5ei/yQ5yyt31kcnLu4bqnm6R1ow+dYv/4GaAiItZinQJF+ibFD2V2jkuWciYNJMZLao23dpTE1g56f/ti8vbxa3AJFLusU8zwhaANK/jtXQ9o0r3j6PLEf5hoZL56csOiWu+IVgGoCiMoHSwe32/N5M+Uxvj742z+bxql2NwVbnrHbWPRb7Wlar9niZnbAHlHHh3XAoB1s4X5fOeBy+L33nBOX4QlJRiQtNi7OfGy+Tus2VEj8A1gad7e7xn1xsAtngVBNg8OJZRIwZzbE5QDoszPKC03DR27UdPNV7WILMRnIs49IggTYAg0qVjKSYp+YOfJBY8jJzZbMuXSfEX6A8S5Kg0ZcHwCoqt3zTr31UBZBF1bCcG4Fudr1W9jaxy1eI9an8hRD+yIiq5TJvjS4+4bFgvLvyzCGc6Rsmlk2FoZ5DZ6cdDG4S1+ommm0uGXSSzRvyJOaK7S2NpcN3sBFJVmQvjcbUBFL2Zk4MLfw32FXxLR04thtyCIq0bpgvPz1KBgyI9HTHm30x+4ZAwY1VIgxCTE5Je5PXGGqhufjYjRQWP3pju/IE5pef0Kc5VC1+YeRpGLhkoGkVDpn1AmLzmIbN3lnZiqE0vzup82l8+leYHZF7znqZogXMO4B/KcZPuYg== X-Forefront-PRVS: 0249EFCB0B X-Microsoft-Exchange-Diagnostics: =?iso-8859-1?Q?1;CY4PR04MB0742;23:jTWHCNSzlIT2+aK4Ro8Oa5BCweJOrvk/bgxtAbm?= =?iso-8859-1?Q?hkd47doahSkBcZRpDVpWBzrQaEJNHAQbyFpB4Kmne3+Fs0aH9WXtCh1EYd?= =?iso-8859-1?Q?e7NPOupKhllzq7bE4FmkILZ9be2jaoakLgIG/JRJgu9HD+YiS3hdoH3drq?= =?iso-8859-1?Q?32DfccRy+hedMUYYJretvaLci3zLoJ1z4ODrkco+hleShLxMdFrOZ49vsE?= =?iso-8859-1?Q?WeUlPxouP+W12hiK2fCWm2G4hyBGd7buhM7m2RnU+/YzwR2IhdTdDQJKn1?= =?iso-8859-1?Q?0wtQpvtF3pYBDuHFWS0jt/KITjFpulvr91RSAOlycPZfnRswE/e8rBdAtb?= =?iso-8859-1?Q?bWGfJCpIcYDplwmMJ1fcSCLoO9Lb0oWYZ0Yn5K0tdE7LAM/LTaaEATGrvD?= =?iso-8859-1?Q?1dn0mpPaXJvtrxs3KeX/6eZrqMHp3XNTJYrEZlyjDtp2Myn/5XWTDPnBXn?= =?iso-8859-1?Q?MaYFH8pAWeOQQweWVQU3SESPRLMQXBGn/X98ZZsh8Yf4HLosgeFOBnBk1H?= =?iso-8859-1?Q?m9bZixQPHo0wFC+WH/STUvrQE3jkk6SYin6RFp1GL6HQNgeBBguCn+Rqyp?= =?iso-8859-1?Q?lhOxXRhsmf3Ydy9X0KkpoEhCqn/ahmQg8WXL5tBW0oYl3pKSlA4I65/Wdh?= =?iso-8859-1?Q?0AS87tW7WrWpr5BwthXKOlTExrbPN1UAzCgvgt0RrDp52y2En5MQRbu5El?= =?iso-8859-1?Q?QFW+jw28ls4BlSHtYgvOwMZMRpxNk0zNnHn/yX/+PeQSadzpnkoDlY06Cp?= =?iso-8859-1?Q?epYv87ziw6aiRyde5RfA2Nv5dfA7/7kYhP07b0BqmfdYrJGkdtZ5sByYbK?= =?iso-8859-1?Q?VGRNojJlOo63PStULud/CLTsC6y+/pCUtYzrzcREZ4piAEQFFbQMIah0sb?= =?iso-8859-1?Q?1MsCJZiUov0ZKsjg5/qaM53mpe69RL5OQFlOie4KAzqhn+2QDwj6V6D7Gr?= =?iso-8859-1?Q?MEOeSZshKdGQIG+HYqvYu5xkIxn7+5z1JsZkvr5Z6wWse4cekk72v23lv0?= =?iso-8859-1?Q?P87ZJ6Iy6QSNDH2ECcA5S3h3y08/ZZnXG4wtbQIGzWffnknNZttPkGTUQn?= =?iso-8859-1?Q?QDNdmX1jVrvRf0uL/XMgh8NCchTRiMEU8ziU6QaBzjCN6H/bFOGRA7dWRH?= =?iso-8859-1?Q?D0Yqmjko3Xb8mLCm/7WqsTRKebgWQhlu2GslYzUNa5du5NUlmKESpYYyMO?= =?iso-8859-1?Q?iBy3Ye2y6GwA/zwd6AgG+dQraXSsosbinjZUx3nTclGPuQB+6CM+1VofWU?= =?iso-8859-1?Q?eZyVixYDuVk4ke9W11nwv4XkVSmnBTMPOoIsiSIH2ddwZ7Ep7/hhXbEz4n?= =?iso-8859-1?Q?o/x53kX7VDGkgpB0tKv5+Of3qHDsettmUaN/YAC4qTTWQig5ib5FV7icIZ?= =?iso-8859-1?Q?V50iZY7Q=3D?= X-Microsoft-Exchange-Diagnostics: 1;CY4PR04MB0742;6:NktGv4mLDjMA+AxN/HNp39DRxd01dbVDa2BU3V9Aj1xLnzvSR/3pBjpvbcP3IADjhfcLNRsEU3E9T34rKuCYvBBq13TgIFr3IgtCjwEjAaIpZ5KEfyvNboOAdXckJRvz5bRRAX4Fyd9VlWkCaifkGl2gegpGKRsfadf22U9LDElP63F4xhWH0qxrarJeo/pe/tXf+FnwI0gs3DUyxR76Knl2mRBzYl+eAyg61KIs6p8MirVBpB2FLeHSLl6wF8HyOP78Bh0K0zXeq5Yom37AaqBHJwBQdvspM8z77WFB6Eu0U5jh3Goka4y6bp8RaEnpJCCE2Sy6ySdbdvPKxWsIi49eGkCgvvFtgfkSf4o9yDBwVW0fbjm9oD4xrerzKhxjT7MqHD/YoTKlesZiU2EWRqnAc2NdhIslnMT/MBazUV6H35I7cZvz5vwq0M0w8tSS;5:8i0B1xC1vREDq/s6X+3v0mVVlrZTeUCzuUrEYZLCDEMIv88Q8lfY/gp59WKbrV8xyDlbmkaeIbpl20WWYf8Y2UKDvEI3eMtzMB5yc+X8vPjtZ2oE1MmWueTyE+uTYWOipfYnDFHKPtN3y+8HvnQ35g==;24:Pf9kuvobdgmDK8woxKcXsg52+OHtloQrEsPOy/Ngxpn+DhHp8DhBB1K6Hnkvbo+5CRys1ED9jlq1wdMoxonHm5VuZbEsBbKw+Hm1VYy612I= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1;CY4PR04MB0742;7:1v8i+s6aC6dR9VBGljbz+S+AO3kmtkHb1auJC3G8ZwBKhYCg5WL2j8wvJjWxhedE7xygl9M0TQPcOzYjlZPLa0Yf+9OXraudrNE9vAp1gBKYwU1tT0o78gyturawYTj99MgAwC1ukkE+UCRZiZ405NE3+4iIPqQmlyJPSKVwt90uYX6HMDDq/roVbqB8pRx7f9Q/L42+UkBEfRnYKspFUWS9dHSjO6wwERdsYTp2fy9tnK3s0nlF1SLAjPxuSwyUMrcqyOp5bNOElr3ocPe0nNqCHHk9PvNqrNoakBkAawzfzZlRn+ax3CuR12hO89MTiYaieuJCdh3y1sIW2fVcvw==;20:o999YKCiXVtl+bpOeVjS08edNuV6/ybsO58BqvLomWvZuz4StMO6oEvOgHzETCJSz55AK25VFlT+h35KBzZ6mEQZMj5oDxBnsXadzoGFh/bYnAk/VTQuCE0bDDajy7gtjbMEfU8hQBQ6F+c/9IAztKyfhH6aOuVWtnR0F2luVIs= X-OriginatorOrg: sandisk.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Mar 2017 20:37:02.2124 (UTC) X-MS-Exchange-CrossTenant-Id: b61c8803-16f3-4c35-9b17-6f65f441df86 X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=b61c8803-16f3-4c35-9b17-6f65f441df86;Ip=[74.221.232.54];Helo=[sacsmgep14.sandisk.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY4PR04MB0742 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by mail.home.local id v2HKdgZU004310 On Sat, 2017-03-18 at 02:32 +0800, Ming Lei wrote: > On Sat, Mar 18, 2017 at 1:32 AM, Bart Van Assche wrote: > > + /* > > + * Avoid that the updates of the queue flags and q_usage_counter > > + * are reordered. > > + */ > > + smp_wmb(); > > atomic_inc_return() in blk_mq_freeze_queue_start() does imply a > barrier(smp_mb()). Hello Ming, It's probably a good idea to mention that in a comment. The implementation of blk_mq_freeze_queue_start() namely could be changed in the future such that it uses another atomic operation that doesn't implicitly perform smp_mb(). Thanks, Bart.