From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.0 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5DBD5C43381 for ; Wed, 27 Mar 2019 20:11:23 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 25F682087C for ; Wed, 27 Mar 2019 20:11:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729218AbfC0ULW (ORCPT ); Wed, 27 Mar 2019 16:11:22 -0400 Received: from metis.ext.pengutronix.de ([85.220.165.71]:54201 "EHLO metis.ext.pengutronix.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727832AbfC0ULW (ORCPT ); Wed, 27 Mar 2019 16:11:22 -0400 Received: from gallifrey.ext.pengutronix.de ([2001:67c:670:201:5054:ff:fe8d:eefb] helo=bjornoya.blackshift.org) by metis.ext.pengutronix.de with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.89) (envelope-from ) id 1h9EtK-00053v-10; Wed, 27 Mar 2019 21:11:18 +0100 Received: from [IPv6:2a03:f580:87bc:d400:845f:6e2c:f24d:6770] (2a03-f580-87bc-d400-845f-6e2c-f24d-6770.ip6.dokom21.de [IPv6:2a03:f580:87bc:d400:845f:6e2c:f24d:6770]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-384) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits)) (Client CN "mkl@blackshift.org", Issuer "StartCom Class 1 Client CA" (not verified)) (Authenticated sender: mkl@blackshift.org) by smtp.blackshift.org (Postfix) with ESMTPSA id 57D193EF362; Wed, 27 Mar 2019 20:11:13 +0000 (UTC) To: Cong Wang Cc: Jiri Pirko , Linux Kernel Network Developers , Dave Taht , Jamal Hadi Salim , kernel@pengutronix.de, linux-can@vger.kernel.org, David Miller References: <20190327165632.10711-1-mkl@pengutronix.de> <20190327165632.10711-2-mkl@pengutronix.de> From: Marc Kleine-Budde Openpgp: preference=signencrypt Autocrypt: addr=mkl@pengutronix.de; prefer-encrypt=mutual; keydata= mQINBFFVq30BEACtnSvtXHoeHJxG6nRULcvlkW6RuNwHKmrqoksispp43X8+nwqIFYgb8UaX zu8T6kZP2wEIpM9RjEL3jdBjZNCsjSS6x1qzpc2+2ivjdiJsqeaagIgvy2JWy7vUa4/PyGfx QyUeXOxdj59DvLwAx8I6hOgeHx2X/ntKAMUxwawYfPZpP3gwTNKc27dJWSomOLgp+gbmOmgc 6U5KwhAxPTEb3CsT5RicsC+uQQFumdl5I6XS+pbeXZndXwnj5t84M+HEj7RN6bUfV2WZO/AB Xt5+qFkC/AVUcj/dcHvZwQJlGeZxoi4veCoOT2MYqfR0ax1MmN+LVRvKm29oSyD4Ts/97cbs XsZDRxnEG3z/7Winiv0ZanclA7v7CQwrzsbpCv+oj+zokGuKasofzKdpywkjAfSE1zTyF+8K nxBAmzwEqeQ3iKqBc3AcCseqSPX53mPqmwvNVS2GqBpnOfY7Mxr1AEmxdEcRYbhG6Xdn+ACq Dq0Db3A++3PhMSaOu125uIAIwMXRJIzCXYSqXo8NIeo9tobk0C/9w3fUfMTrBDtSviLHqlp8 eQEP8+TDSmRP/CwmFHv36jd+XGmBHzW5I7qw0OORRwNFYBeEuiOIgxAfjjbLGHh9SRwEqXAL kw+WVTwh0MN1k7I9/CDVlGvc3yIKS0sA+wudYiselXzgLuP5cQARAQABtCZNYXJjIEtsZWlu ZS1CdWRkZSA8bWtsQHBlbmd1dHJvbml4LmRlPokCVAQTAQoAPgIbAwIeAQIXgAULCQgHAwUV CgkICwUWAgMBABYhBMFAC6CzmJ5vvH1bXCte4hHFiupUBQJcUsSbBQkM366zAAoJECte4hHF iupUgkAP/2RdxKPZ3GMqag33jKwKAbn/fRqAFWqUH9TCsRH3h6+/uEPnZdzhkL4a9p/6OeJn Z6NXqgsyRAOTZsSFcwlfxLNHVxBWm8pMwrBecdt4lzrjSt/3ws2GqxPsmza1Gs61lEdYvLST Ix2vPbB4FAfE0kizKAjRZzlwOyuHOr2ilujDsKTpFtd8lV1nBNNn6HBIBR5ShvJnwyUdzuby tOsSt7qJEvF1x3y49bHCy3uy+MmYuoEyG6zo9udUzhVsKe3hHYC2kfB16ZOBjFC3lH2U5An+ yQYIIPZrSWXUeKjeMaKGvbg6W9Oi4XEtrwpzUGhbewxCZZCIrzAH2hz0dUhacxB201Y/faY6 BdTS75SPs+zjTYo8yE9Y9eG7x/lB60nQjJiZVNvZ88QDfVuLl/heuIq+fyNajBbqbtBT5CWf mOP4Dh4xjm3Vwlz8imWW/drEVJZJrPYqv0HdPbY8jVMpqoe5jDloyVn3prfLdXSbKPexlJaW 5tnPd4lj8rqOFShRnLFCibpeHWIumqrIqIkiRA9kFW3XMgtU6JkIrQzhJb6Tc6mZg2wuYW0d Wo2qvdziMgPkMFiWJpsxM9xPk9BBVwR+uojNq5LzdCsXQ2seG0dhaOTaaIDWVS8U/V8Nqjrl 6bGG2quo5YzJuXKjtKjZ4R6k762pHJ3tnzI/jnlc1sXzuQENBFxSzJYBCAC58uHRFEjVVE3J 31eyEQT6H1zSFCccTMPO/ewwAnotQWo98Bc67ecmprcnjRjSUKTbyY/eFxS21JnC4ZB0pJKx MNwK6zq71wLmpseXOgjufuG3kvCgwHLGf/nkBHXmSINHvW00eFK/kJBakwHEbddq8Dr4ewmr G7yr8d6A3CSn/qhOYWhIxNORK3SVo4Io7ExNX/ljbisGsgRzsWvY1JlN4sabSNEr7a8YaqTd 2CfFe/5fPcQRGsfhAbH2pVGigr7JddONJPXGE7XzOrx5KTwEv19H6xNe+D/W3FwjZdO4TKIo vcZveSDrFWOi4o2Te4O5OB/2zZbNWPEON8MaXi9zABEBAAGJA3IEGAEKACYWIQTBQAugs5ie b7x9W1wrXuIRxYrqVAUCXFLMlgIbAgUJAeKNmgFACRArXuIRxYrqVMB0IAQZAQoAHRYhBJrx JF84Dn3PPNRrhVrGIaOR5J0gBQJcUsyWAAoJEFrGIaOR5J0grw4H/itil/yryJCvzi6iuZHS suSHHOiEf+UQHib1MLP96LM7FmDabjVSmJDpH4TsMu17A0HTG+bPMAdeia0+q9FWSvSHYW8D wNhfkb8zojpa37qBpVpiNy7r6BKGSRSoFOv6m/iIoRJuJ041AEKao6djj/FdQF8OV1EtWKRO +nE2bNuDCcwHkhHP+FHExdzhKSmnIsMjGpGwIQKN6DxlJ7fN4W7UZFIQdSO21ei+akinBo4K O0uNCnVmePU1UzrwXKG2sS2f97A+sZE89vkc59NtfPHhofI3JkmYexIF6uqLA3PumTqLQ2Lu bywPAC3YNphlhmBrG589p+sdtwDQlpoH9O7NeBAAg/lyGOUUIONrheii/l/zR0xxr2TDE6tq 6HZWdtjWoqcaky6MSyJQIeJ20AjzdV/PxMkd8zOijRVTnlK44bcfidqFM6yuT1bvXAO6NOPy pvBRnfP66L/xECnZe7s07rXpNFy72XGNZwhj89xfpK4a9E8HQcOD0mNtCJaz7TTugqBOsQx2 45VPHosmhdtBQ6/gjlf2WY9FXb5RyceeSuK4lVrz9uZB+fUHBge/giOSsrqFo/9fWAZsE67k 6Mkdbpc7ZQwxelcpP/giB9N+XAfBsffQ8q6kIyuFV4ILsIECCIA4nt1rYmzphv6t5J6PmlTq TzW9jNzbYANoOFAGnjzNRyc9i8UiLvjhTzaKPBOkQfhStEJaZrdSWuR/7Tt2wZBBoNTsgNAw A+cEu+SWCvdX7vNpsCHMiHtcEmVt5R0Tex1Ky87EfXdnGR2mDi6Iyxi3MQcHez3C61Ga3Baf P8UtXR6zrrrlX22xXtpNJf4I4Z6RaLpB/avIXTFXPbJ8CUUbVD2R2mZ/jyzaTzgiABDZspbS gw17QQUrKqUog0nHXuaGGA1uvreHTnyBWx5P8FP7rhtvYKhw6XdJ06ns+2SFcQv0Bv6PcSDK aRXmnW+OsDthn84x1YkfGIRJEPvvmiOKQsFEiB4OUtTX2pheYmZcZc81KFfJMmE8Z9+LT6Ry uSS5AQ0EXFLNDgEIAL14qAzTMCE1PwRrYJRI/RSQGAGF3HLdYvjbQd9Ozzg02K3mNCF2Phb1 cjsbMk/V6WMxYoZCEtCh4X2GjQG2GDDW4KC9HOa8cTmr9Vcno+f+pUle09TMzWDgtnH92WKx d0FIQev1zDbxU7lk1dIqyOjjpyhmR8Put6vgunvuIjGJ/GapHL/O0yjVlpumtmow6eME2muc TeJjpapPWBGcy/8VU4LM8xMeMWv8DtQML5ogyJxZ0Smt+AntIzcF9miV2SeYXA3OFiojQstF vScN7owL1XiQ3UjJotCp6pUcSVgVv0SgJXbDo5Nv87M2itn68VPfTu2uBBxRYqXQovsR++kA EQEAAYkCPAQYAQoAJhYhBMFAC6CzmJ5vvH1bXCte4hHFiupUBQJcUs0OAhsMBQkB4o0iAAoJ ECte4hHFiupUbioQAJ40bEJmMOF28vFcGvQrpI+lfHJGk9zSrh4F4SlJyOVWV1yWyUAINr8w v1aamg2nAppZ16z4nAnGU/47tWZ4P8blLVG8x4SWzz3D7MCy1FsQBTrWGLqWldPhkBAGp2VH xDOK4rLhuQWx3H5zd3kPXaIgvHI3EliWaQN+u2xmTQSJN75I/V47QsaPvkm4TVe3JlB7l1Fg OmSvYx31YC+3slh89ayjPWt8hFaTLnB9NaW9bLhs3E2ESF9Dei0FRXIt3qnFV/hnETsx3X4h KEnXxhSRDVeURP7V6P/z3+WIfddVKZk5ZLHi39fJpxvsg9YLSfStMJ/cJfiPXk1vKdoa+FjN 7nGAZyF6NHTNhsI7aHnvZMDavmAD3lK6CY+UBGtGQA3QhrUc2cedp1V53lXwor/D/D3Wo9wY iSXKOl4fFCh2Peo7qYmFUaDdyiCxvFm+YcIeMZ8wO5udzkjDtP4lWKAn4tUcdcwMOT5d0I3q WATP4wFI8QktNBqF3VY47HFwF9PtNuOZIqeAquKezywUc5KqKdqEWCPx9pfLxBAh3GW2Zfjp lP6A5upKs2ktDZOC2HZXP4IJ1GTk8hnfS4ade8s9FNcwu9m3JlxcGKLPq5DnIbPVQI1UUR4F QyAqTtIdSpeFYbvH8D7pO4lxLSz2ZyBMk+aKKs6GL5MqEci8OcFW Subject: Re: [PATCH 1/2] net: sch_generic: add flag IFF_FIFO_QUEUE to use pfifo_fast as default scheduler Message-ID: <7a25d800-aed1-bec6-0ff8-38c06c3b8fb5@pengutronix.de> Date: Wed, 27 Mar 2019 21:11:02 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.5.1 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="q6VZhPsvsIxZNhfozWL3PZh8kaOmYrAFm" X-SA-Exim-Connect-IP: 2001:67c:670:201:5054:ff:fe8d:eefb X-SA-Exim-Mail-From: mkl@pengutronix.de X-SA-Exim-Scanned: No (on metis.ext.pengutronix.de); SAEximRunCond expanded to false X-PTX-Original-Recipient: netdev@vger.kernel.org Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --q6VZhPsvsIxZNhfozWL3PZh8kaOmYrAFm Content-Type: multipart/mixed; boundary="xjg0BCi14C39uMYDPAuKvLMIbuCUp2G4y"; protected-headers="v1" From: Marc Kleine-Budde To: Cong Wang Cc: Jiri Pirko , Linux Kernel Network Developers , Dave Taht , Jamal Hadi Salim , kernel@pengutronix.de, linux-can@vger.kernel.org, David Miller Message-ID: <7a25d800-aed1-bec6-0ff8-38c06c3b8fb5@pengutronix.de> Subject: Re: [PATCH 1/2] net: sch_generic: add flag IFF_FIFO_QUEUE to use pfifo_fast as default scheduler References: <20190327165632.10711-1-mkl@pengutronix.de> <20190327165632.10711-2-mkl@pengutronix.de> In-Reply-To: --xjg0BCi14C39uMYDPAuKvLMIbuCUp2G4y Content-Type: text/plain; charset=utf-8 Content-Language: en-GB Content-Transfer-Encoding: quoted-printable On 3/27/19 6:14 PM, Cong Wang wrote: > On Wed, Mar 27, 2019 at 9:56 AM Marc Kleine-Budde = wrote: >> >> There is networking hardware that isn't based on Ethernet for layers 1= and 2. >> >> For example CAN. >> >> CAN is a multi-master serial bus standard for connecting Electronic Co= ntrol >> Units [ECUs] also known as nodes. A frame on the CAN bus carries up to= 8 bytes >> of payload. Frame corruption is detected by a CRC. However frame loss = due to >> corruption is possible, but a quite unusual phenomenon. >> >> While fq_codel works great for TCP/IP, it doesn't for CAN. There are a= lot of >> legacy protocols on top of CAN, which are not build with flow control = or high >> CAN frame drop rates in mind. >> >> When using fq_codel, as soon as the queue reaches a certain delay base= d length, >> skbs from the head of the queue are silently dropped. Silently meaning= that the >> user space using a send() or similar syscall doesn't get an error. How= ever >> TCP's flow control algorithm will detect dropped packages and adjust t= he >> bandwidth accordingly. >> >> When using fq_codel and sending raw frames over CAN, which is the comm= on use >> case, the user space thinks the package has been sent without problems= , because >> send() returned without an error. pfifo_fast will drop skbs, if the qu= eue >> length exceeds the maximum. But with this scheduler the skbs at the ta= il are >> dropped, an error (-ENOBUFS) is propagated to user space. So that the = user >> space can slow down the package generation. >> >> On distributions, where fq_codel is made default via CONFIG_DEFAULT_NE= T_SCH >> during compile time, or set default during runtime with sysctl >> net.core.default_qdisc (see [1]), we get a bad user experience. In my = test case >> with pfifo_fast, I can transfer thousands of million CAN frames withou= t a frame >> drop. On the other hand with fq_codel there is more then one lost CAN = frame per >> thousand frames. >> >> As pointed out fq_codel is not suited for CAN hardware, so this patch >> introduces a new netdev_priv_flag called "IFF_FIFO_QUEUE" (in contrast= to the >> existing "IFF_NO_QUEUE"). >> >> During transition of a netdev from down to up state the default queuin= g >> discipline is attached by attach_default_qdiscs() with the help of >> attach_one_default_qdisc(). This patch modifies attach_one_default_qdi= sc() to >> attach the pfifo_fast (pfifo_fast_ops) if the "IFF_FIFO_QUEUE" flag is= set. >=20 > I wonder if we just need to allow arbitrary default qdisc per netdevice= > while you are on it. A private flag is simply a boolean, perhaps in the= > future other type of devices wants other default qdiscs, so that could > make it more flexible. =46rom my point of view there is networking hardware that use protocols that work with (i.e. benefit from) fq_codel (hash flow/queue/head drop). The silent head drop is the most prominent reason why it doesn't work on CAN. I haven't dug deep enough into the code to see if skb->hash is used or what the flow dissector will do on CAN frames. So reordering of CAN frames (if something else than skb->priority is used) might be a problem, too. =46rom my point of view, if your networking hardware and the protocols on= top don't like re-ordering or silent head drop, than pfifo_fast is probably a good default choice. I discussed the problem a bit at netdev 0x13 and one point someone mentioned is that if there is a generic set this qdisc function people might start to add this to network drivers to "optimize" them for their special workflow or test case. Doubts aside, how should an arbitrary default qdisc per netdevice look like? Add a string "default_qdisc" to the netdev? Lookup qdisc by string during DOWN->UP transition? What do if that qdisc is not compiled into the kernel? Or rather use an array of qdiscs with one of sch_generic defaults? Marc --=20 Pengutronix e.K. | Marc Kleine-Budde | Industrial Linux Solutions | Phone: +49-231-2826-924 | Vertretung West/Dortmund | Fax: +49-5121-206917-5555 | Amtsgericht Hildesheim, HRA 2686 | http://www.pengutronix.de | --xjg0BCi14C39uMYDPAuKvLMIbuCUp2G4y-- --q6VZhPsvsIxZNhfozWL3PZh8kaOmYrAFm Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- iQEzBAEBCgAdFiEEmvEkXzgOfc881GuFWsYho5HknSAFAlyb2NYACgkQWsYho5Hk nSDTkgf+O47Vm/J3p0yXQFYRp6m1TBBbJw0pMa+52ShLZU1nxP1nA+l/ZhfzMmlw ZteO6PHpDf37VpCSrAG739v417xxIincVwULGa7aRMMxP/HPqEP7WEaKyRXOAboN lWfACxOCzAWLcXj/FpOS0OLRA0jN5GI0FY5P6IJIhnXMpGPwniEugO+H+lMcC5bR Dd9Of0w3WF3nzQrkkjEj+MS/1NBGWiiKvs4xP2ykPGc0piaRO4hk6PL1/i1Rsuen TvNqG3etEqGecri65W5yPVQ0cBdp8TlM9108hLbpSMlrtLMYTu0bON6vwlnx8nOo 2dDC4hQi4q6RcT+vhIHjxhLwAZlAJg== =uGS2 -----END PGP SIGNATURE----- --q6VZhPsvsIxZNhfozWL3PZh8kaOmYrAFm--