From mboxrd@z Thu Jan 1 00:00:00 1970 From: Jason Gunthorpe Subject: Re: [PATCH 2/2] IB/mad: Use IDR for agent IDs Date: Mon, 11 Jun 2018 10:19:18 -0600 Message-ID: <20180611161918.GF5815@mellanox.com> References: <20180608174218.32455-1-willy@infradead.org> <20180608174218.32455-3-willy@infradead.org> <20180610063028.GH12407@mtr-leonro.mtl.com> <20180610104305.GA9284@bombadil.infradead.org> <20180610122505.GM12407@mtr-leonro.mtl.com> <20180610203027.GF5560@mellanox.com> <20180611043425.GA21382@mtr-leonro.mtl.com> <20180611044203.GA32562@mellanox.com> <20180611091914.00007858@dev.mellanox.co.il> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Return-path: Content-Disposition: inline In-Reply-To: <20180611091914.00007858@dev.mellanox.co.il> Sender: linux-kernel-owner@vger.kernel.org To: jackm Cc: Leon Romanovsky , Matthew Wilcox , hans.westgaard.ry@oracle.com, Doug Ledford , Matthew Wilcox , linux-rdma@vger.kernel.org, =?utf-8?B?SMOla29u?= Bugge , Parav Pandit , Pravin Shedge , linux-kernel@vger.kernel.org List-Id: linux-rdma@vger.kernel.org On Mon, Jun 11, 2018 at 09:19:14AM +0300, jackm wrote: > On Sun, 10 Jun 2018 22:42:03 -0600 > Jason Gunthorpe wrote: > > > Er, the spec has nothing to do with this. In Linux the TID is made > > unique because the core code provides 32 bits that are unique and the > > user provides another 32 bits that are unique. The driver cannot > > change any of those bits without risking non-uniquenes, which is > > exactly the bug mlx4 created when it stepped outside its bounds and > > improperly overrode bits in the TID for its own internal use. > > Actually, the opposite is true here. When SRIOV is active, each VM > generates its *own* TIDs -- with 32 bits of agent number and 32 bits > of counter. And it does it while re-using the LRH of the host, so all VMs and the host are now forced to share a TID space, yes I know. > There is a chance that two different VMs can generate the same TID! > Encoding the slave (VM) number in the packet actually guarantees > uniqueness here. Virtualizing the TID in the driver would be fine, but it must virtualize all the TIDs (even those generated by the HOST). Just blindly assuming the host doesn't generate TID's that overlap with the virtualization process is a bug. > There is nothing wrong with modifying the TID in a reversible way in > order to: a. guarantee uniqueness b. identify the VM which should > receive the response packet Sure, as long as *all* TID's sharing a LRH are vitalized like this. > The problem was created when the agent-id numbers started to use the > most-significant byte (thus making the MSB slave-id addition > impossible). It hasn't always been this way? What commit? Jason