From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1755403AbcIKIkO (ORCPT ); Sun, 11 Sep 2016 04:40:14 -0400 Received: from mail-db5eur01on0086.outbound.protection.outlook.com ([104.47.2.86]:60544 "EHLO EUR01-DB5-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1752735AbcIKIkH (ORCPT ); Sun, 11 Sep 2016 04:40:07 -0400 Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=matanb@mellanox.com; Subject: Re: [PATCHv12 1/3] rdmacg: Added rdma cgroup controller To: Jason Gunthorpe , Christoph Hellwig References: <1472632647-1525-1-git-send-email-pandit.parav@gmail.com> <1472632647-1525-2-git-send-email-pandit.parav@gmail.com> <61101e8b-5776-c0bc-b3ea-d8b984eebabf@mellanox.com> <20160831211618.GA12660@htj.duckdns.org> <9b6a346d-af4c-1e5f-0144-f68fb8e46c27@mellanox.com> <20160901084406.GA4115@lst.de> <20160910161442.GC29259@lst.de> <20160910170151.GA5230@obsidianresearch.com> CC: Parav Pandit , Tejun Heo , , , "Linux Kernel Mailing List" , , Li Zefan , Johannes Weiner , Doug Ledford , Liran Liss , "Hefty, Sean" , Haggai Eran , Jonathan Corbet , , , Or Gerlitz , Andrew Morton , From: Matan Barak Message-ID: Date: Sun, 11 Sep 2016 11:07:57 +0300 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.2.0 MIME-Version: 1.0 In-Reply-To: <20160910170151.GA5230@obsidianresearch.com> Content-Type: text/plain; charset="windows-1252"; format=flowed Content-Transfer-Encoding: 7bit X-Originating-IP: [193.47.165.251] X-ClientProxiedBy: AM4PR01CA0011.eurprd01.prod.exchangelabs.com (10.164.74.149) To DB5PR05MB1735.eurprd05.prod.outlook.com (10.165.7.17) X-MS-Office365-Filtering-Correlation-Id: c9c7ee91-1a4d-4a79-c4a7-08d3da1ac11d X-Microsoft-Exchange-Diagnostics: 1;DB5PR05MB1735;2:mH+CcCvVk0k+4t4Wr4ZezHOCU/jdZBwKkeESd4zBh7+8eVcvaKmnVuXvp1I5xKOlejvmb0x/hbTTHEcjBHRQdDPZbRkVcUyjCR8luIwxtHB3qQqgiOiewMR5xgm+3df6D4XO5Mafj9pezg37ocq6W4qIACEFLPD8fVm7Z+sq28biJYPBo7kbD9PmqrJfExjH;3:U4wP7l6Ai4VOUQVsxmUsFIw7UkGuuBr8+ue1mKthxq/GxZ5pBuUqq1bzuFPIzH9r/B8eAFmKne+AxkbeM1KhNh81oEmbDtiHTo68Q93JIXFsUieC51ddpIbLmAIgq0lD X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:DB5PR05MB1735; X-Microsoft-Exchange-Diagnostics: 1;DB5PR05MB1735;25:a3V/SSrtd8RSKhIiINgUcjsf/YmHlWm7NoToO1cH4ZiPmh5Un0f++mUofN0B740s2lIy+KCYam1ZlQNr+E00QGpvrsJrcXIIyxcTzQjC6veYb0k26aADOZTsi+Z/gE6T4gYmgCPmjvEM0bCl9FcUYvuFBsEFKkJVmnOVlj0bzPnwVUS6oS4eno6SsxIoYb3gpjxGSgl4ZeV/OhkJcrnEUxz8Mvyw0T1nh/L57TVBUobxNlrF4sz4NoJYfl4accbOohUdCa8NUwqI0m9GA493eSh0tloMWeHbfwNaNYFqx1zpgKRwq4aE/YCCIxYdTlEWU/t8Ne5DWtJnT1Xv2GEirz1r+5dxgKUIHBus7OVKFLEtwDcewcKT5Lb65qby64XiQaZHVo5Inr+DyQfSukQGMZbaZf2hMF57Mrz6dyR/oN13j/QYvMEzUcS4hipMUUpRKJtURXpfRkRKB2smqZU2laXVoYnGT4X7EO7hfjoYxpSzi92wOuvHtkWOI5TcmoBbt/yCdaOHvbAOQMqink51bLE3NUR9lx2sy/OnoIGt48HXtUgLmr4lg9cFPW1j8IOeiffuP7Sv0aU1AUYw+d3LoaLoZwLwnQtV7jxnQ0EleJMtptilxvSZx+pnxYjgZj1l98OHE8fLrxL71qHirhUN7Nx2gVlXVqnScz2xf2/xAl6eF4NObFdy2++zk8H4UoLFW8xaWCBx1jAhCOhhAjjI3A==;31:LzOBKWotizyhqmcCt9Dm9rAQ89sg1ajMZXOU1uhjxswc1Y9rWhPZurokdSa+/O7IWqvu2+9JDTOqXvAVBUlpy0ePPIPVr7F/hQu6bBHffssXF+fCGuUe8itppOdRrq4qI/BtcDoqHbd6nPXaYUkh8UffnIqPP2LGWieO/92mNplq5D1q3OGeBEFXjMWArhlO7dTx0KEgVzY3r40ZV8m6s8q/awHYA4EErg72fIRdgaI= X-Microsoft-Exchange-Diagnostics: 1;DB5PR05MB1735;20:e8PJH6kq4QnUh1HihjqBQZ84bKrJDo+hpnAE60d2iBqX6P+ileFNdCVQCfsZjtPy2wS7qG5Y8ZB4dT4GBjQzCLvDVcPNnbeFqYSe5QFpvDoP2/3t7Hx5cgjuJKSlKI6cWVP25nsY1j8eYOXiSMdTU3LHaOR92RXnIvca0+nY8oggtlOmRWc5IOZK1+5cEh/xfRKrOoTmFPr7rTd6COEg4mcPeCeKoYy1VdxxqBL/NTBvQEgH112ceQS87HFlxL9Zirm4H0VRSykGJ+22db1dE9UxRrmgmys0dMfbHdouCQPteDRZbwcdatFhDaE9vO+hSKCLjCUFqaZA613pVQSl/A7BBdAusxXX9d/DM0zlG+HE5x9ubPoi9jMHBfX1j24LNFCfg+8qNnbur+NG4SIAbRr5pnSKFQ7G8xnUh8jtIVr+4SesbWnrSyGiTwNqMQW3v7HKbvAYu7FgaY3N6e3eTEF43nmZ4M4ISzr3xCp+mJrqUMxN+HI1IiT0JkeM0m+Q;4:bQvFC4Q8BPQSuyo9Dd0CnY3YrYY4nQNtqPgn02VyUKvNKvoCd1ZxrX/sA8V6u+ZbaJLgpCC0q4ukY/dl1UfCoZUZe679rJ3D912rmmpgaODulk0xXQOwAmXSRZpSeDuR4vL63Ws8Hvu6RRi8g3XHDVDiuY8+FUe7r1TTU06RO9fHkH0WxbDYjAz/4Ru4K4EA7odgxYeYlnp9zu7uLLonoXTGPT5dA7P8iTMyl7RM6/SYyhw/tCbPLy8bDePcM8UNQxL4AfuclNS2fitv0+iYHzgguPoVchKJzoovTJ+ZlgJjhxb7ObTHWrCJ4CPQD0PczX2JqFHcF11pBWNpDgKT7Z0vXXPyvckGCq9k+BvVTXMTYY5+GnHOpAFRAd8JBg8SXLr+9XestvPyBYMJrd9hIg== X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(6040176)(601004)(2401047)(8121501046)(5005006)(10201501046)(3002001)(6055026);SRVR:DB5PR05MB1735;BCL:0;PCL:0;RULEID:;SRVR:DB5PR05MB1735; X-Forefront-PRVS: 0062BDD52C X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10009020)(4630300001)(6049001)(6009001)(7916002)(24454002)(189002)(199003)(4001350100001)(36756003)(23746002)(4326007)(2906002)(5001770100001)(586003)(6116002)(3846002)(68736007)(83506001)(305945005)(7736002)(86362001)(33646002)(230700001)(97736004)(7846002)(92566002)(64126003)(189998001)(8676002)(77096005)(65826007)(81166006)(81156014)(31696002)(106356001)(7416002)(42186005)(50466002)(65806001)(65956001)(66066001)(31686004)(50986999)(54356999)(76176999)(2950100001)(101416001)(5660300001)(47776003)(105586002)(93886004);DIR:OUT;SFP:1101;SCL:1;SRVR:DB5PR05MB1735;H:[10.223.3.146];FPR:;SPF:None;PTR:InfoNoRecords;MX:1;A:1;LANG:en; X-Microsoft-Exchange-Diagnostics: =?Windows-1252?Q?1;DB5PR05MB1735;23:Ds0MSx02Dr/mDqMSQxM6KMDN+NodLbhMPQFFA?= =?Windows-1252?Q?NEtuEpl6RXvXUhoV46kDT32fYMA7CGKPXgg5p7et4ypVGfwJM2Gbhh0t?= =?Windows-1252?Q?1eGhAB8vN/0OQ6iRjFRaM6LJBwNcJka/sYNvzZ8ph8E1Qa7q22MuRAZd?= =?Windows-1252?Q?Pj9x5iKNlDW1JG3/V2ZjszK7BuzLHwUfLmUqzP2HRh81uOVYd/tvhGk5?= =?Windows-1252?Q?3yxO85SaM2iB5E/l7cgrrdHsuYL8xpeyZ7DB5J1HjMtQcac7gj4vYPcR?= =?Windows-1252?Q?DR9gfCjmN2Hb0fQEWEXd+POXQaWzjNZcQuwmbAmhMvRXh+EMRG7My7GD?= =?Windows-1252?Q?IYMhkCnCngK4+FL+ON/UMvKAUznidU3W3aOAcJwEbTecOCEHYoAP7vM/?= =?Windows-1252?Q?Ek8ZvLnCShDf5whB1iHwViGzwydh2x10Co3YE4edjd2iguBmZl7NSJNv?= =?Windows-1252?Q?pE5H21F8Yk1EJ4Sye0gyfR3B21ldFsQbONwPIq+FNmoh1yy46XuHQsAs?= =?Windows-1252?Q?aDqwQCEyP3esbN4s8zVf1r+B7D7isXo3gKxoGdrr+1Ol+t1WDddjOkRO?= =?Windows-1252?Q?1BNwNJ3/k+w5hji3D9gnhBoLFV5viuf3Ionh6gcbR/TMOJn0k9dpJtE4?= =?Windows-1252?Q?RFWaE9zv5VHJfUEc+d/jxcuPUKIkshbeJCG3LuOTUOla/0+GsDpCiRNU?= =?Windows-1252?Q?wemq9lJhHqbCkdMqKYMmVN3jSmVK3tgrrXQ03Uc8sztXHxdKS9V/3c5h?= =?Windows-1252?Q?XfuNTxIzUTX+tuKjOyfohAkt8T3qo84YKNczjbD0qtRwUk9vdMmhGZvd?= =?Windows-1252?Q?v1ysfcz/IC0Nq7RrgOBFoLl6uWpVkpBycNsbzvB2C+hoDP8lvXtbjO/N?= =?Windows-1252?Q?5HWcBMQNwR/nRAn8MzxWcK5wLFaugBo4xBbBykkZFndf9bfLAqwnl+Jz?= =?Windows-1252?Q?3Xh6/fDdUsTS35kJSlv9z4C4MOYHhE77GTNnJYNWw/oapaCRT3dVCFUS?= =?Windows-1252?Q?q04RULEcjubdFqcHomnxa402BhIAgejXVjmF+rzm+8/kfB08FIGG/4yx?= =?Windows-1252?Q?N3gEtOZ7fsAEY373z4R7VcZ3rqALg+P1l9ZXN1OdL7DdcAH1SmETrhMf?= =?Windows-1252?Q?a206PdOzELetCuO8I+3RjsBboR0M0go/pPlGbEQr/zc1FQLYvuAWNhYC?= =?Windows-1252?Q?43JutIlz9irbivqmn03WRthkWOsw3i/k91XqB/jUKbQ7AG8+vdEDEfZf?= =?Windows-1252?Q?9cdLT41nmq1ogy3ieHtKYPCrVc48CPFdCIxuasVGj2WcglMcXdc6u6bq?= =?Windows-1252?Q?Ei8LDKsySFcXFYVibdIGi7YmSFzVvw8oYxkzzY80khhN8mrC/yCPxbBJ?= =?Windows-1252?Q?Ppa2ZE69uya?= X-Microsoft-Exchange-Diagnostics: 1;DB5PR05MB1735;6:06UCgNzE1+7KgrjmZIb07T4ztH9HjWR11mJiCQNcdHCE0DDyC1iSTI5VIQBMhutcMbcpJXbRhbQt83COvNWLTiAWm6yWaF5cQyaNXbIsXd+fRoxMk9Rf114dKCWRwm49DPIQFgp9Al2KZniBLLX9Hnxfw2i8ePg9lYAdiqkd6Yli2rQ1fX5Nad4FzDiZyozr7F33BO9sIEmzxPs0j/dH+AZyL3OK4HHaOSOPTL/0hDs7D5dBYkN0BZWrspKoIjPI8i4AkdOfWFLdNJmLUrcniCRDo2VM3GIek/B4r69rUzzPCwchISseKm5iL3r+8c4hykJfBe0SHRWPEp0BbYjPpQ==;5:7NUlnwkpXZRIjvffI2WJ5qF+k1pJRC8yHmtRr+BCbzvLq5odR5Fzz/1ERFOWd5HGCd7e6yTOzzXv3m1nTqUtYt58a+r/MMdWDlM2W+Su8pV0opmEBLrd8WgH+EiuaKpf5iSgvePr2dorovpYMwxdgg==;24:nTv7fCndulHWZ20QMWfUaDjlyE/7/hinEasZw6bogRgWpBXIfo+07Wg+FuIw7SYhRIO24W4ziFyAzKdVCUyIa+JXMZ6iUX2Y94uuanD6xdg=;7:davWgAsQaajJL3QutRDGJt9K90G0ejK5r86l0Uqp2BkIVBr3a23gTUa5BfbEdvtIAHBU9Qge8ScfapIbi51CV1//vUubGYz1jf4oAod2sRcoM5gz70Bj51sEDH1K5r1IVp5MzI3Edp703+WceCh8A27cY1ZHPTxtZDG2P92jd9owoQEydjPxZEMVkY9LVpVm6jmS1CZvsoY7/r6+8BlDilJEOL9O4+SrpP+u8I4Z6zhu4uKzO2P9dxsgWAmiZ89b SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Sep 2016 08:08:01.4344 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB5PR05MB1735 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 10/09/2016 20:01, Jason Gunthorpe wrote: > On Sat, Sep 10, 2016 at 06:14:42PM +0200, Christoph Hellwig wrote: >> OFVWG meetings have absolutely zero relevance for Linux development. > > Well, to be fair there are a fair number of kernel developers on that > particular call.. > >> More "flexibility" for drivers just means giving up on designing a >> coherent API and leaving it to drivers authors to add crap to their >> own drivers. That's a major step backwards. > > Sadly, it isn't a step backwards, it is status quo - at least as far > as the uapi is concerned. > > Every single user space driver has its own private abi file, carefully > hidden in their driver, and dutifully copied over to user space: > > providers/cxgb3/iwch-abi.h > providers/cxgb4/cxgb4-abi.h > providers/hfi1verbs/hfi-abi.h > providers/i40iw/i40iw-abi.h > providers/ipathverbs/ipath-abi.h > providers/mlx4/mlx4-abi.h > providers/mlx5/mlx5-abi.h > providers/mthca/mthca-abi.h > providers/nes/nes-abi.h > providers/ocrdma/ocrdma_abi.h > providers/rxe/rxe-abi.h > > Just to pick two random examples: > > struct mlx5_create_cq { > struct ibv_create_cq ibv_cmd; > __u64 buf_addr; > __u64 db_addr; > __u32 cqe_size; > }; > > struct iwch_create_cq { > struct ibv_create_cq ibv_cmd; > uint64_t user_rptr_addr; > }; > > Love to hear ideas on a way forward that doesn't involve rewriting > everything :( > Yeah, unfortunately, the RDMA ABI is more driver specific ABI than a common user-kernel ABI. I guess this will become even worse, as the RDMA subsystem is evolving to serve more drivers with different object types. For example, I would like to hear how hfi1 are going to define their user-kernel ABI (once they leave the custom ioctls). >> They should not be using the code in drivers/infiniband. usnic is such >> an example of a driver that should never have been added in it's current >> form. > > +1 > > Jason > Matan