From mboxrd@z Thu Jan 1 00:00:00 1970 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: base64 Subject: UAS: fix alignment of scatter/gather segments From: Alan Stern Message-Id: Date: Tue, 30 Apr 2019 10:39:25 -0400 (EDT) To: David Laight Cc: Oliver Neukum , "gregKH@linuxfoundation.org" , "linux-usb@vger.kernel.org" List-ID: T24gVHVlLCAzMCBBcHIgMjAxOSwgRGF2aWQgTGFpZ2h0IHdyb3RlOgoKPiBGcm9tOiBBbGFuIFN0 ZXJuCj4gPiBTZW50OiAyOSBBcHJpbCAyMDE5IDE4OjU1Cj4gPiBPbiBNb24sIDI5IEFwciAyMDE5 LCBPbGl2ZXIgTmV1a3VtIHdyb3RlOgo+ID4gCj4gPiA+IE9uIE1vLCAyMDE5LTA0LTI5IGF0IDEy OjA4IC0wNDAwLCBBbGFuIFN0ZXJuIHdyb3RlOgo+ID4gPiA+IE9uIE1vbiwgMjkgQXByIDIwMTks IE9saXZlciBOZXVrdW0gd3JvdGU6Cj4gPiA+ID4KPiA+ID4gPiA+IE9uIE1vLCAyMDE5LTA0LTI5 IGF0IDE1OjA2ICswMDAwLCBEYXZpZCBMYWlnaHQgd3JvdGU6Cj4gPiA+ID4KPiA+ID4gPiA+IEJ1 dCB0aGUgc3RhdGVtZW50IHRoZSBvbGQgY29tbWVudCBtYWRlIGFyZSBubyBsb25nZXIgY29ycmVj dC4KPiA+ID4gPgo+ID4gPiA+IFBlcmhhcHMgRGF2aWQgd291bGQgYmUgc2F0aXNmaWVkIGlmIHRo ZSBjb21tZW50IHdlcmUgY2hhbmdlZCB0byBzYXkKPiA+ID4gPiB0aGF0IF9zb21lXyBVU0IgY29u dHJvbGxlciBkcml2ZXJzIGhhdmUgdGhpcyB1bnVzdWFsIGFsaWdubWVudAo+ID4gPiA+IHJlcXVp cmVtZW50Lgo+ID4gPgo+ID4gPiBJdCB3b3VsZCBzZWVtIHRvIG1lIHRoYXQgZXZlcnkgY29udHJv bGxlciB0aGF0IGRvZXMgbm90IGRvCj4gPiA+IHNjYXR0ZXIvZ2F0aGVyIGhhcyB0aGlzIHJlcXVp cmVtZW50LiBJbiBvdGhlciB3b3JkcywgdGhpcyBpcwo+ID4gPiB0aGUgdHJ1ZSByZXF1aXJlbWVu dCBvZiBVU0IuIEl0IGRvZXMgbm90IGNvbWUgZnJvbSB0aGUKPiA+ID4gY29udHJvbGxlci4gSXQg Y29tZXMgZnJvbSB0aGUgcHJvdG9jb2wncyBuZWVkIHRvIG5vdAo+ID4gPiBzZW5kIGEgc2hvcnQg cGFja2FnZS4KPiAKPiBUaGUgaGFyZHdhcmUgcmVxdWlyZW1lbnQgaXMgdGhhdCBwYWNrZXRzIHRo YXQgbmVlZCB0byBiZSBqb2luZWQKPiB0b2dldGhlciB0byBtYWtlIGEgbG9uZyByZXF1ZXN0IG11 c3QgYmUgJ2Z1bGwnLgo+IEluIG1hbnkgY2FzZXMgdGhpcyBtZWFucyBhIHplcm8gbGVuZ3RoIHBh Y2tldCBtdXN0IGJlIHNlbnQgdG8KPiBjb3JyZWN0bHkgdGVybWluYXRlIGEgcmVxdWVzdCB0aGF0 IGlzIGEgbXVsdGlwbGUgb2YgdGhlIHBhY2tldCBzaXplLgoKV2l0aCBtYXNzIHN0b3JhZ2UgdGhp cyBoYXBwZW5zIG9ubHkgd2hlbiB0aGVyZSdzIGFuIGVycm9yLiAgSW4gb3RoZXIKY2FzZXMgdGhl IHNlbmRlciBhbmQgcmVjZWl2ZXIgYm90aCBrbm93IGJlZm9yZWhhbmQgaG93IGxvbmcgYSBidWxr CnRyYW5zZmVyIHdpbGwgYmUsIHNvIHRoZXJlJ3Mgbm8gbmVlZCB0byBtYXJrIGFuIGVhcmx5IHRl cm1pbmF0aW9uIHdpdGgKYSB6ZXJvLWxlbmd0aCBwYWNrZXQuCgo+IFNpbmNlIHRoZSBzb2Z0d2Fy ZSBoYXMgdG8gYWRkIHRoZSB6ZXJvIGxlbmd0aCBwYWNrZXQgdGhlcmUgaXMKPiBubyByZWFsIGRp ZmZlcmVuY2UgYmV0d2VlbiBhIHNpbmdsZSBzY2F0dGVyLWdhdGhlciB0cmFuc21pdCBhbmQKPiBt dWx0aXBsZSBzaW5nbGUgcGFja2V0IHRybmFzbWl0cy4KCkluIGZhY3QsIHplcm8tbGVuZ3RoIHBh Y2tldHMgaGF2ZSBub3RoaW5nIHRvIGRvIHdpdGggaXQuICBUaGUgc291cmNlIG9mCnRoZSBkYXRh IGRvZXNuJ3QgbWF0dGVyOiBJdCBjb3VsZCBiZSBzY2F0dGVyLWdhdGhlciwgYSBzaW5nbGUKY29u dGlndW91cyBidWZmZXIsIG9yIGV2ZW4gYSBPdWlqYSBib2FyZC4gIEFsbCB0aGF0IG1hdHRlcnMg aXMgdGhhdCB0aGUKZGF0YSBzaG93cyB1cCBhdCB0aGUgcmVjZWl2ZXIgYXMgYSBzZXJpZXMgb2Yg cGFja2V0cyBpbiB3aGljaCBhbGwKKGV4Y2VwdCBwb3NzaWJseSB0aGUgbGFzdCkgYXJlIGZ1bGwg c2l6ZS4gIFRoZSByZWNlaXZlciBkb2Vzbid0IGNhcmUgCmhvdyB0aG9zZSBwYWNrZXRzIHdlcmUg YXNzZW1ibGVkLgoKPiBGb3IgVVNCMiBidWxrIHRyYW5zZmVycyB0aGUgcGFja2V0IHNpemUgaXMg NTEyLCBhbmQgZm9yIFVTQjMgMTAyNC4KPiBUaGUgb2xkIGNvbW1lbnQgc3VnZ2VzdGVkIDIwNDgg Zm9yIHNvbWUgdW5zdXBwb3J0ZWQgaW50ZXJmYWNlLgoKV2lyZWxlc3MgVVNCLiAgQW5kIHRvIGJl IGZhaXIsIHRoZSBvbGQgY29tbWVudCBhbHNvIHNhaWQgdGhhdCAKc3VwcG9ydGluZyBzdWNoIGEg bGFyZ2UgdmFsdWUgd291bGQgcmVxdWlyZSBjaGFuZ2VzIG91dHNpZGUgdGhlIFVTQiAKc3RhY2su Cgo+ID4gQXJlIHlvdSBzdXJlIHRoYXQgeEhDSSBoYXMgdGhpcyByZXF1aXJlbWVudD8gIEkgaGF2 ZW4ndCBjaGVja2VkIHRoZQo+ID4gc3BlYy4gIEkga25vdyB0aGF0IFVIQ0ksIE9IQ0ksIGFuZCBF SENJIGRvIG5lZWQgdGhpcyBhbGlnbm1lbnQgKGFuZAo+ID4gT0hDSSBhbmQgRUhDSSBkbyBpbiBm YWN0IGhhdmUgaGFyZHdhcmUgc3VwcG9ydCBmb3Igc2NhdHRlci1nYXRoZXIpLgo+ID4gCj4gPiBN b3JlIHByZWNpc2VseSwgd2hhdCBtYXR0ZXJzIGlzIHdoZXRoZXIgdGhlIGNvbnRyb2xsZXIgaXMg YWJsZSB0byBtZXJnZQo+ID4gdHdvIGRpZmZlcmVudCBETUEgc2VnbWVudHMgaW50byBhIHNpbmds ZSBwYWNrZXQuICBVSENJIGNhbid0LiAgT0hDSSBhbmQKPiA+IEVIQ0kgY2FuLCBidXQgb25seSBp ZiB0aGUgZmlyc3Qgc2VnbWVudCBlbmRzIGF0IGEgcGFnZSBib3VuZGFyeSBhbmQgdGhlCj4gPiBz ZWNvbmQgYmVnaW5zIGF0IGEgcGFnZSBib3VuZGFyeSAtLSBpdCdzIGVhc2llciBqdXN0IHRvIHNh eSB0aGF0IHRoZQo+ID4gc2VnbWVudHMgaGF2ZSB0byBiZSBtYXhwYWNrZXQtYWxpZ25lZC4KPiAK PiBYSENJIChmb3IgVVNCMiBvciBVU0IzKSBjYW4gaGFuZGxlIGFsbW9zdCBhcmJpdHJhcnkgZnJh Z21lbnRzLgo+IFRoZXJlIGFyZSBhIGNvdXBsZSBvZiBhbm5veWluZyByZXN0cmljdGlvbnMgKElJ UkMpOgo+IC0gRnJhZ21lbnRzIGNhbm5vdCBjcm9zcyBhIDY0ayBib3VuZGFyeS4KPiAgIChIb3cg bXVjaCB3b3VsZCBpdCBjb3N0IHRoZSBoYXJkd2FyZSB0byBoYXZlIGEgMzJiaXQgKG9yIGV2ZW4g NjRiaXQpCj4gICBjb3VudGVyLikKPiAtIFRoZSAnTGluayBUUkInIGJldHdlZW4gcmluZyBzZWdt ZW50cyBtdXN0IGJlIGFsaWduZWQgdG8gYSBwYWNrZXQgYm91bmRhcnkuCj4gSSBiZWxpZXZlIHRo ZSBMaW51eCBYSENJIGRyaXZlciBub3cgaGFuZGxlcyBib3RoIHRoZXNlIGNhc2VzLgo+IChJdCBo YWRuJ3QgdXNlZCB0byAtIHdoaWNoIGlzIHdoeSBJIGtub3cgYW55dGhpbmcgYWJvdXQgVVNCMyBh bmQgWEhDSS4pCj4gCj4gSSBhbHNvIHJlY2FsbCBpc3N1ZXMgd2l0aCBub24gd29yZCBhbGlnbmVk IGJ1ZmZlcnMgZm9yIHNvbWUgY29udHJvbGxlcnMuCj4gCj4gPiA+IFRoZSBzZWNvbmQsIG9sZCwg Y29tbWVudCBpcyBhYm91dCBjb250cm9sbGVycy4KPiA+IAo+ID4gV2VsbCwgaWYgdGhlIGRyaXZl cnMgd291bGQgdXNlIGJvdW5jZSBidWZmZXJzIHRvIHdvcmsgYXJvdW5kIHRoZQo+ID4gY29udHJv bGxlcnMnIGlzc3VlcyB0aGVuIHRoZXkgd291bGRuJ3QgaGF2ZSB0aGlzIHNwZWNpYWwgcmVxdWly ZW1lbnQuCj4gPiBTbyBpdCByZWFsbHkgaXMgYSBjb21iaW5hdGlvbiBvZiB3aGF0IHRoZSBoYXJk d2FyZSBjYW4gZG8gYW5kIHdoYXQgdGhlCj4gPiBkcml2ZXIgY2FuIGRvLgo+IAo+IEluZGVlZC4K PiBTbyBhbnkgY29tbWVudCBzaG91bGQgcmVmZXIgdG8gd2hhdCB0aGUgbGludXggdXNiIGRyaXZl cnMgcmVxdWlyZXMKPiBvZiB0aGVpciAndXNlcicsIG5vdCB3aGF0IGhhcHBlbnMgb24gdGhlIFVT QiB3aXJlLgoKVGhlIGNvbW1lbnQgc2FpZCBub3RoaW5nIGFib3V0IHRoZSB3aXJlLiAgSXQgd2Fz IGFjdHVhbGx5IHJhdGhlciB2YWd1ZSwgCmp1c3Qgc2F5aW5nICJVU0IiIC0tIHRoaXMgY291bGQg bWVhbiB0aGUgaGFyZHdhcmUgcHJvdG9jb2wgb3IgdGhlIFVTQiAKaG9zdCBjb250cm9sbGVyIGRy aXZlci4KCj4gSXQgbWlnaHQgbWUgdGhhdCB5b3UgZG8gZW5kIHVwIGhhdmluZyB0byByZXF1ZXN0 IDFrIGFsaWduZWQKPiBidWZmZXJzIGZvciBYSENJLCBidXQgdGhlIGNvbW1lbnQgc2hvdWxkIHNh eSB0aGF0IHlvdSBhcmUKPiBhZGRpbmcgYSBmYXIgc3Ryb25nZXIgcmVzdHJpY3Rpb24gdGhhbiB0 aGF0IHJlcXVpcmVkIGJ5IHRoZQo+IGRyaXZlciBhbmQgY29udHJvbGxlci4KCkRlcGVuZHMgb24g dGhlIGRyaXZlci4gIEZvciB4SENJIHRoZSByZXN0cmljdGlvbiBpcyBzdHJvbmdlciB0aGFuIApy ZXF1aXJlZDsgZm9yIGFsbCB0aGUgb3RoZXJzIChhcyBmYXIgYXMgSSBrbm93KSBpdCBpcyBub3Qu Cgo+IEdpdmVuIHRoYXQgWEhDSSBpcyB0aGUgbW9zdCBjb21tb24gaW50ZXJmYWNlIChhdCBsZWFz dCBvbiB4ODYpCj4gaWYgdGhlIDFrIGFsaWdubWVudCBmb3JjZXMgZXh0cmEgYm91bmNlIGJ1ZmZl cnMgaW4gYW55IGNvZGUKPiBwYXRocyBpdCBjb3VsZCBiZSBhIHBlcmZvcm1hbmNlIGlzc3VlLgoK SWYgdGhhdCBoYXBwZW5zIHdlIGNhbiBhc2sgdGhlIGNvbnRyb2xsZXIgZHJpdmVyIHRvIGV4cG9y dCBhIAoic2NhdHRlci1nYXRoZXIgcmVxdWlyZWQgYWxpZ25tZW50IiBwYXJhbWV0ZXIuICBJdCBz ZWVtcyBwcmVtYXR1cmUgdG8gCndvcnJ5IGFib3V0IGl0IG5vdy4KCkFsYW4gU3Rlcm4K From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-0.9 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 51C85C43219 for ; Tue, 30 Apr 2019 14:39:27 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 294A521707 for ; Tue, 30 Apr 2019 14:39:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726202AbfD3Oj0 (ORCPT ); Tue, 30 Apr 2019 10:39:26 -0400 Received: from iolanthe.rowland.org ([192.131.102.54]:35578 "HELO iolanthe.rowland.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with SMTP id S1726073AbfD3Oj0 (ORCPT ); Tue, 30 Apr 2019 10:39:26 -0400 Received: (qmail 3134 invoked by uid 2102); 30 Apr 2019 10:39:25 -0400 Received: from localhost (sendmail-bs@127.0.0.1) by localhost with SMTP; 30 Apr 2019 10:39:25 -0400 Date: Tue, 30 Apr 2019 10:39:25 -0400 (EDT) From: Alan Stern X-X-Sender: stern@iolanthe.rowland.org To: David Laight cc: Oliver Neukum , "gregKH@linuxfoundation.org" , "linux-usb@vger.kernel.org" Subject: RE: [PATCH] UAS: fix alignment of scatter/gather segments In-Reply-To: <734e89837b454acea32b990fa2aff902@AcuMS.aculab.com> Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Sender: linux-usb-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-usb@vger.kernel.org Message-ID: <20190430143925.3xxuHqIQ7J-6KRGL2wvyYEj6KP2tu10fyhisbqlKgoY@z> On Tue, 30 Apr 2019, David Laight wrote: > From: Alan Stern > > Sent: 29 April 2019 18:55 > > On Mon, 29 Apr 2019, Oliver Neukum wrote: > > > > > On Mo, 2019-04-29 at 12:08 -0400, Alan Stern wrote: > > > > On Mon, 29 Apr 2019, Oliver Neukum wrote: > > > > > > > > > On Mo, 2019-04-29 at 15:06 +0000, David Laight wrote: > > > > > > > > > But the statement the old comment made are no longer correct. > > > > > > > > Perhaps David would be satisfied if the comment were changed to say > > > > that _some_ USB controller drivers have this unusual alignment > > > > requirement. > > > > > > It would seem to me that every controller that does not do > > > scatter/gather has this requirement. In other words, this is > > > the true requirement of USB. It does not come from the > > > controller. It comes from the protocol's need to not > > > send a short package. > > The hardware requirement is that packets that need to be joined > together to make a long request must be 'full'. > In many cases this means a zero length packet must be sent to > correctly terminate a request that is a multiple of the packet size. With mass storage this happens only when there's an error. In other cases the sender and receiver both know beforehand how long a bulk transfer will be, so there's no need to mark an early termination with a zero-length packet. > Since the software has to add the zero length packet there is > no real difference between a single scatter-gather transmit and > multiple single packet trnasmits. In fact, zero-length packets have nothing to do with it. The source of the data doesn't matter: It could be scatter-gather, a single contiguous buffer, or even a Ouija board. All that matters is that the data shows up at the receiver as a series of packets in which all (except possibly the last) are full size. The receiver doesn't care how those packets were assembled. > For USB2 bulk transfers the packet size is 512, and for USB3 1024. > The old comment suggested 2048 for some unsupported interface. Wireless USB. And to be fair, the old comment also said that supporting such a large value would require changes outside the USB stack. > > Are you sure that xHCI has this requirement? I haven't checked the > > spec. I know that UHCI, OHCI, and EHCI do need this alignment (and > > OHCI and EHCI do in fact have hardware support for scatter-gather). > > > > More precisely, what matters is whether the controller is able to merge > > two different DMA segments into a single packet. UHCI can't. OHCI and > > EHCI can, but only if the first segment ends at a page boundary and the > > second begins at a page boundary -- it's easier just to say that the > > segments have to be maxpacket-aligned. > > XHCI (for USB2 or USB3) can handle almost arbitrary fragments. > There are a couple of annoying restrictions (IIRC): > - Fragments cannot cross a 64k boundary. > (How much would it cost the hardware to have a 32bit (or even 64bit) > counter.) > - The 'Link TRB' between ring segments must be aligned to a packet boundary. > I believe the Linux XHCI driver now handles both these cases. > (It hadn't used to - which is why I know anything about USB3 and XHCI.) > > I also recall issues with non word aligned buffers for some controllers. > > > > The second, old, comment is about controllers. > > > > Well, if the drivers would use bounce buffers to work around the > > controllers' issues then they wouldn't have this special requirement. > > So it really is a combination of what the hardware can do and what the > > driver can do. > > Indeed. > So any comment should refer to what the linux usb drivers requires > of their 'user', not what happens on the USB wire. The comment said nothing about the wire. It was actually rather vague, just saying "USB" -- this could mean the hardware protocol or the USB host controller driver. > It might me that you do end up having to request 1k aligned > buffers for XHCI, but the comment should say that you are > adding a far stronger restriction than that required by the > driver and controller. Depends on the driver. For xHCI the restriction is stronger than required; for all the others (as far as I know) it is not. > Given that XHCI is the most common interface (at least on x86) > if the 1k alignment forces extra bounce buffers in any code > paths it could be a performance issue. If that happens we can ask the controller driver to export a "scatter-gather required alignment" parameter. It seems premature to worry about it now. Alan Stern