patch 8466489ef5ba48272ba4fa4ea9f8f403306de4c7 breaks Renesas USB3 controller functionality

patch 8466489ef5ba48272ba4fa4ea9f8f403306de4c7 breaks Renesas USB3 controller functionality

[Marc Zyngier]

On 03/05/18 09:42, Faiz Abbas wrote:
> Hi Marc,
> 
> On Thursday 03 May 2018 01:44 PM, Marc Zyngier wrote:
>> On 03/05/18 05:49, Faiz Abbas wrote:
>>> Hi Everyone,
>>>
>>> On Wednesday 11 April 2018 07:32 PM, Marc Zyngier wrote:
>>>> On Wed, 11 Apr 2018 14:11:52 +0100,
>>>> Bockholdt Arne wrote:
>>>>>
>>>>> Hi all,
>>>>>
>>>>> is there anything new, I've just tried the new stable 4.16.1 kernel
>>>>> without any change. The Renesys USB3 controller is still not
>>>>> functional. I'm willing to test any patch that is based on a stable
>>>>> kernel version because the machine is in production use.
>>>>
>>>> Have you tested the branch[1] I mentioned in my previous email?
>>>> Without your feedback, I cannot really make much progress on this.
>>>>
>>>> Thanks,
>>>>
>>>> 	M.
>>>> 	
>>>> [1] https://www.spinics.net/lists/linux-usb/msg167301.html
>>>>
>>>
>>> I was also having problems with a Renesas card (01:00.0 USB controller
>>> [0c03]: Renesas Technology Corp. uPD720201 USB 3.0 Host Controller
>>> [1912:0014]) on a dra72x-evm. The kernel would just hang because of the
>>> xhci reset.
>>>
>>> Log:https://pastebin.ubuntu.com/p/dYYV9DZMwx/
>>>
>>> The patches on Marc's branch have fixed this for me as well. Thanks for
>>> the fix.
>> OK. I'll rebase this on a more recent version of the kernel, make it
>> conditional on having an iommu (as it seems to be the only affected
>> configuration), and post that to a wider audience.
> 
> Just to be sure, you are talking about the original 32 bit DMA issue
> being conditional on iommu right? Because dra72x doesn't use iommu.

I'm talking about making the whole workaround dependent on the USB
controller being behind an iommu. No iommu, no workaround (because it is
likely that there is no problem in that case).

	M.

patch 8466489ef5ba48272ba4fa4ea9f8f403306de4c7 breaks Renesas USB3 controller functionality

[Marc Zyngier]

On Mon, 07 May 2018 06:00:56 +0100,
Bockholdt Arne wrote:
> 
> 
> 
> On Thu, 2018-05-03 at 10:41 +0100, Marc Zyngier wrote:
> > I'm talking about making the whole workaround dependent on the USB
> > controller being behind an iommu. No iommu, no workaround (because it
> > is
> > likely that there is no problem in that case).
> > 
> My server doesn't have an IOMMU at all and is affected by the bug. I'm
> the original reporter of the problem.

No, you are just reporting a regression caused by the workaround for
the original issue, which has to do with transitioning from 64bit
addresses to 32bit addresses.

Resting the controller solves that problem, but generates other issues
(yours and a few others). The current approach is to drop the
mandatory reset and to use a different sequence, only if the device is
behind an IOMMU.

So in your case, we won't do anything at all, which is what seem to
work for most people. Only a subset of the platforms that would be
affected will have another workaround. Machines that don't have an
IOMMU *and* perform an 64/32 address range transition may see memory
corruption or other side effects, but that's nothing new (the HW looks
terminally broken).

Hope this makes it clear.

	M.

patch 8466489ef5ba48272ba4fa4ea9f8f403306de4c7 breaks Renesas USB3 controller functionality

[Bockholdt Arne]

DQoNCk9uIFRodSwgMjAxOC0wNS0wMyBhdCAxMDo0MSArMDEwMCwgTWFyYyBaeW5naWVyIHdyb3Rl
Og0KPiBJJ20gdGFsa2luZyBhYm91dCBtYWtpbmcgdGhlIHdob2xlIHdvcmthcm91bmQgZGVwZW5k
ZW50IG9uIHRoZSBVU0INCj4gY29udHJvbGxlciBiZWluZyBiZWhpbmQgYW4gaW9tbXUuIE5vIGlv
bW11LCBubyB3b3JrYXJvdW5kIChiZWNhdXNlIGl0DQo+IGlzDQo+IGxpa2VseSB0aGF0IHRoZXJl
IGlzIG5vIHByb2JsZW0gaW4gdGhhdCBjYXNlKS4NCj4gDQpNeSBzZXJ2ZXIgZG9lc24ndCBoYXZl
IGFuIElPTU1VIGF0IGFsbCBhbmQgaXMgYWZmZWN0ZWQgYnkgdGhlIGJ1Zy4gSSdtDQp0aGUgb3Jp
Z2luYWwgcmVwb3J0ZXIgb2YgdGhlIHByb2JsZW0uDQoNCg0KUmVnYXJkcywNCg0KDQoJQXJuZQ0K
DQoNCg0K
---
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

patch 8466489ef5ba48272ba4fa4ea9f8f403306de4c7 breaks Renesas USB3 controller functionality

[Marc Zyngier]

On 03/05/18 11:41, Ard Biesheuvel wrote:
> On 3 May 2018 at 11:41, Marc Zyngier <marc.zyngier@arm.com> wrote:
>> On 03/05/18 09:42, Faiz Abbas wrote:
>>> Hi Marc,
>>>
>>> On Thursday 03 May 2018 01:44 PM, Marc Zyngier wrote:
>>>> On 03/05/18 05:49, Faiz Abbas wrote:
>>>>> Hi Everyone,
>>>>>
>>>>> On Wednesday 11 April 2018 07:32 PM, Marc Zyngier wrote:
>>>>>> On Wed, 11 Apr 2018 14:11:52 +0100,
>>>>>> Bockholdt Arne wrote:
>>>>>>>
>>>>>>> Hi all,
>>>>>>>
>>>>>>> is there anything new, I've just tried the new stable 4.16.1 kernel
>>>>>>> without any change. The Renesys USB3 controller is still not
>>>>>>> functional. I'm willing to test any patch that is based on a stable
>>>>>>> kernel version because the machine is in production use.
>>>>>>
>>>>>> Have you tested the branch[1] I mentioned in my previous email?
>>>>>> Without your feedback, I cannot really make much progress on this.
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>>    M.
>>>>>>
>>>>>> [1] https://www.spinics.net/lists/linux-usb/msg167301.html
>>>>>>
>>>>>
>>>>> I was also having problems with a Renesas card (01:00.0 USB controller
>>>>> [0c03]: Renesas Technology Corp. uPD720201 USB 3.0 Host Controller
>>>>> [1912:0014]) on a dra72x-evm. The kernel would just hang because of the
>>>>> xhci reset.
>>>>>
>>>>> Log:https://pastebin.ubuntu.com/p/dYYV9DZMwx/
>>>>>
>>>>> The patches on Marc's branch have fixed this for me as well. Thanks for
>>>>> the fix.
>>>> OK. I'll rebase this on a more recent version of the kernel, make it
>>>> conditional on having an iommu (as it seems to be the only affected
>>>> configuration), and post that to a wider audience.
>>>
>>> Just to be sure, you are talking about the original 32 bit DMA issue
>>> being conditional on iommu right? Because dra72x doesn't use iommu.
>>
>> I'm talking about making the whole workaround dependent on the USB
>> controller being behind an iommu. No iommu, no workaround (because it is
>> likely that there is no problem in that case).
>>
> 
> That is still only a partial solution, unfortunately.
> 
> On non-x86 systems with memory above and below the 4 GB mark, it is
> undefined which side the firmware will choose and which side the
> kernel will choose (although I suppose the kernel may attempt to
> preserve 32-bit addressable memory for peripherals that are only
> 32-bit capable)
> 
> This would be much easier to fix at the UEFI stub level (but still in
> kernel code, not firmware), by checking for PCI I/O protocol instances
> with the correct VID/PID and check whether the
> EFI_PCI_IO_ATTRIBUTE_DUAL_ADDRESS_CYCLE attribute is set, but that is
> also only a partial solution.

Unfortunately, I'm not sure there is a full solution that works for
everyone, and doesn't introduce regression. The reset method is proving
to be bad for a number of platform, so I'd rather have something that
narrowly matches the one broken case we know about.

Without knowing more about the internals of that controller, I'm not
convinced we can do much more...

	M.

patch 8466489ef5ba48272ba4fa4ea9f8f403306de4c7 breaks Renesas USB3 controller functionality

[Ard Biesheuvel]

On 3 May 2018 at 11:41, Marc Zyngier <marc.zyngier@arm.com> wrote:
> On 03/05/18 09:42, Faiz Abbas wrote:
>> Hi Marc,
>>
>> On Thursday 03 May 2018 01:44 PM, Marc Zyngier wrote:
>>> On 03/05/18 05:49, Faiz Abbas wrote:
>>>> Hi Everyone,
>>>>
>>>> On Wednesday 11 April 2018 07:32 PM, Marc Zyngier wrote:
>>>>> On Wed, 11 Apr 2018 14:11:52 +0100,
>>>>> Bockholdt Arne wrote:
>>>>>>
>>>>>> Hi all,
>>>>>>
>>>>>> is there anything new, I've just tried the new stable 4.16.1 kernel
>>>>>> without any change. The Renesys USB3 controller is still not
>>>>>> functional. I'm willing to test any patch that is based on a stable
>>>>>> kernel version because the machine is in production use.
>>>>>
>>>>> Have you tested the branch[1] I mentioned in my previous email?
>>>>> Without your feedback, I cannot really make much progress on this.
>>>>>
>>>>> Thanks,
>>>>>
>>>>>    M.
>>>>>
>>>>> [1] https://www.spinics.net/lists/linux-usb/msg167301.html
>>>>>
>>>>
>>>> I was also having problems with a Renesas card (01:00.0 USB controller
>>>> [0c03]: Renesas Technology Corp. uPD720201 USB 3.0 Host Controller
>>>> [1912:0014]) on a dra72x-evm. The kernel would just hang because of the
>>>> xhci reset.
>>>>
>>>> Log:https://pastebin.ubuntu.com/p/dYYV9DZMwx/
>>>>
>>>> The patches on Marc's branch have fixed this for me as well. Thanks for
>>>> the fix.
>>> OK. I'll rebase this on a more recent version of the kernel, make it
>>> conditional on having an iommu (as it seems to be the only affected
>>> configuration), and post that to a wider audience.
>>
>> Just to be sure, you are talking about the original 32 bit DMA issue
>> being conditional on iommu right? Because dra72x doesn't use iommu.
>
> I'm talking about making the whole workaround dependent on the USB
> controller being behind an iommu. No iommu, no workaround (because it is
> likely that there is no problem in that case).
>

That is still only a partial solution, unfortunately.

On non-x86 systems with memory above and below the 4 GB mark, it is
undefined which side the firmware will choose and which side the
kernel will choose (although I suppose the kernel may attempt to
preserve 32-bit addressable memory for peripherals that are only
32-bit capable)

This would be much easier to fix at the UEFI stub level (but still in
kernel code, not firmware), by checking for PCI I/O protocol instances
with the correct VID/PID and check whether the
EFI_PCI_IO_ATTRIBUTE_DUAL_ADDRESS_CYCLE attribute is set, but that is
also only a partial solution.
---
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

patch 8466489ef5ba48272ba4fa4ea9f8f403306de4c7 breaks Renesas USB3 controller functionality

[Faiz Abbas]

Hi Marc,

On Thursday 03 May 2018 01:44 PM, Marc Zyngier wrote:
> On 03/05/18 05:49, Faiz Abbas wrote:
>> Hi Everyone,
>>
>> On Wednesday 11 April 2018 07:32 PM, Marc Zyngier wrote:
>>> On Wed, 11 Apr 2018 14:11:52 +0100,
>>> Bockholdt Arne wrote:
>>>>
>>>> Hi all,
>>>>
>>>> is there anything new, I've just tried the new stable 4.16.1 kernel
>>>> without any change. The Renesys USB3 controller is still not
>>>> functional. I'm willing to test any patch that is based on a stable
>>>> kernel version because the machine is in production use.
>>>
>>> Have you tested the branch[1] I mentioned in my previous email?
>>> Without your feedback, I cannot really make much progress on this.
>>>
>>> Thanks,
>>>
>>> 	M.
>>> 	
>>> [1] https://www.spinics.net/lists/linux-usb/msg167301.html
>>>
>>
>> I was also having problems with a Renesas card (01:00.0 USB controller
>> [0c03]: Renesas Technology Corp. uPD720201 USB 3.0 Host Controller
>> [1912:0014]) on a dra72x-evm. The kernel would just hang because of the
>> xhci reset.
>>
>> Log:https://pastebin.ubuntu.com/p/dYYV9DZMwx/
>>
>> The patches on Marc's branch have fixed this for me as well. Thanks for
>> the fix.
> OK. I'll rebase this on a more recent version of the kernel, make it
> conditional on having an iommu (as it seems to be the only affected
> configuration), and post that to a wider audience.

Just to be sure, you are talking about the original 32 bit DMA issue
being conditional on iommu right? Because dra72x doesn't use iommu.

Thanks,
Faiz
---
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

patch 8466489ef5ba48272ba4fa4ea9f8f403306de4c7 breaks Renesas USB3 controller functionality

[Marc Zyngier]

On 03/05/18 05:49, Faiz Abbas wrote:
> Hi Everyone,
> 
> On Wednesday 11 April 2018 07:32 PM, Marc Zyngier wrote:
>> On Wed, 11 Apr 2018 14:11:52 +0100,
>> Bockholdt Arne wrote:
>>>
>>> Hi all,
>>>
>>> is there anything new, I've just tried the new stable 4.16.1 kernel
>>> without any change. The Renesys USB3 controller is still not
>>> functional. I'm willing to test any patch that is based on a stable
>>> kernel version because the machine is in production use.
>>
>> Have you tested the branch[1] I mentioned in my previous email?
>> Without your feedback, I cannot really make much progress on this.
>>
>> Thanks,
>>
>> 	M.
>> 	
>> [1] https://www.spinics.net/lists/linux-usb/msg167301.html
>>
> 
> I was also having problems with a Renesas card (01:00.0 USB controller
> [0c03]: Renesas Technology Corp. uPD720201 USB 3.0 Host Controller
> [1912:0014]) on a dra72x-evm. The kernel would just hang because of the
> xhci reset.
> 
> Log:https://pastebin.ubuntu.com/p/dYYV9DZMwx/
> 
> The patches on Marc's branch have fixed this for me as well. Thanks for
> the fix.
OK. I'll rebase this on a more recent version of the kernel, make it
conditional on having an iommu (as it seems to be the only affected
configuration), and post that to a wider audience.

I'll keep you guys on cc.

Thanks,

	M.

patch 8466489ef5ba48272ba4fa4ea9f8f403306de4c7 breaks Renesas USB3 controller functionality

[Faiz Abbas]

Hi Everyone,

On Wednesday 11 April 2018 07:32 PM, Marc Zyngier wrote:
> On Wed, 11 Apr 2018 14:11:52 +0100,
> Bockholdt Arne wrote:
>>
>> Hi all,
>>
>> is there anything new, I've just tried the new stable 4.16.1 kernel
>> without any change. The Renesys USB3 controller is still not
>> functional. I'm willing to test any patch that is based on a stable
>> kernel version because the machine is in production use.
> 
> Have you tested the branch[1] I mentioned in my previous email?
> Without your feedback, I cannot really make much progress on this.
> 
> Thanks,
> 
> 	M.
> 	
> [1] https://www.spinics.net/lists/linux-usb/msg167301.html
> 

I was also having problems with a Renesas card (01:00.0 USB controller
[0c03]: Renesas Technology Corp. uPD720201 USB 3.0 Host Controller
[1912:0014]) on a dra72x-evm. The kernel would just hang because of the
xhci reset.

Log:https://pastebin.ubuntu.com/p/dYYV9DZMwx/

The patches on Marc's branch have fixed this for me as well. Thanks for
the fix.


Regards,
Faiz
---
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

patch 8466489ef5ba48272ba4fa4ea9f8f403306de4c7 breaks Renesas USB3 controller functionality

[Bockholdt Arne]

DQoNCk9uIFRodSwgMjAxOC0wNC0xMiBhdCAwNzoyMCArMDIwMCwgQXJkIEJpZXNoZXV2ZWwgd3Jv
dGU6DQo+IE9uIDEyIEFwcmlsIDIwMTggYXQgMDc6MDUsIEJvY2tob2xkdCBBcm5lDQo+IDxhLmJv
Y2tob2xkdEBwcmVjaXRlYy1vcHRyb25pay5kZT4gd3JvdGU6DQo+ID4gDQo+ID4gT24gV2VkLCAy
MDE4LTA0LTExIGF0IDE1OjAyICswMTAwLCBNYXJjIFp5bmdpZXIgd3JvdGU6DQo+ID4gDQo+ID4g
T24gV2VkLCAxMSBBcHIgMjAxOCAxNDoxMTo1MiArMDEwMCwNCj4gPiBCb2NraG9sZHQgQXJuZSB3
cm90ZToNCj4gPiANCj4gPiANCj4gPiBIaSBhbGwsDQo+ID4gDQo+ID4gaXMgdGhlcmUgYW55dGhp
bmcgbmV3LCBJJ3ZlIGp1c3QgdHJpZWQgdGhlIG5ldyBzdGFibGUgNC4xNi4xIGtlcm5lbA0KPiA+
IHdpdGhvdXQgYW55IGNoYW5nZS4gVGhlIFJlbmVzeXMgVVNCMyBjb250cm9sbGVyIGlzIHN0aWxs
IG5vdA0KPiA+IGZ1bmN0aW9uYWwuIEknbSB3aWxsaW5nIHRvIHRlc3QgYW55IHBhdGNoIHRoYXQg
aXMgYmFzZWQgb24gYSBzdGFibGUNCj4gPiBrZXJuZWwgdmVyc2lvbiBiZWNhdXNlIHRoZSBtYWNo
aW5lIGlzIGluIHByb2R1Y3Rpb24gdXNlLg0KPiA+IA0KPiA+IA0KPiA+IEhhdmUgeW91IHRlc3Rl
ZCB0aGUgYnJhbmNoWzFdIEkgbWVudGlvbmVkIGluIG15IHByZXZpb3VzIGVtYWlsPw0KPiA+IFdp
dGhvdXQgeW91ciBmZWVkYmFjaywgSSBjYW5ub3QgcmVhbGx5IG1ha2UgbXVjaCBwcm9ncmVzcyBv
biB0aGlzLg0KPiA+IA0KPiA+IFRoYW5rcywNCj4gPiANCj4gPiBNLg0KPiA+IA0KPiA+IFsxXSBo
dHRwczovL3d3dy5zcGluaWNzLm5ldC9saXN0cy9saW51eC11c2IvbXNnMTY3MzAxLmh0bWwNCj4g
PiANCj4gPiANCj4gPiBJdCBzZWVtcyB0aGUgcmVwbyBpcyBiYXNlZCBvbiAzLjkga2VybmVsIGFu
ZCBub3Qgb24gYSBjdXJyZW50DQo+ID4gc3RhYmxlIGJyYW5jaCwNCj4gPiBpc24ndCBpdD8gSXQn
cyByYXRoZXIgb2xkLCBJJ20gbm90IHN1cmUgbXkgc2V0dXAgd2lsbCB3b3JrIHdpdGgNCj4gPiB0
aGlzIG9sZA0KPiA+IGtlcm5lbC4NCj4gPiANCj4gDQo+IEFyZSB5b3Ugc3VyZSB5b3UgcHVsbGVk
IHRoZSBjb3JyZWN0IGJyYW5jaD8gdXNiL3VQRDcyMDIwMi1yZXNldCBoYXMNCj4gdGhlIGZvbGxv
d2luZyBvbiB0b3Agb2YgKnY0LjE2LXJjNioNCj4gDQo+IFJldmVydCAieGhjaTogUmVzZXQgUmVu
ZXNhcyB1UEQ3MjAyMHggVVNCIGNvbnRyb2xsZXIgZm9yIDMyLWJpdCBETUENCj4gaXNzdWUiDQo+
IHhoY2k6IEFkZCBxdWlyayB0byB6ZXJvIDY0Yml0IHJlZ2lzdGVycyBvbiBSZW5lc2FzIFBDSWUg
Y29udHJvbGxlcnMNCj4gDQo+IChodHRwczovL2dpdC5rZXJuZWwub3JnL3B1Yi9zY20vbGludXgv
a2VybmVsL2dpdC9tYXovYXJtLQ0KPiBwbGF0Zm9ybXMuZ2l0L2xvZy8/aD11c2IvdVBENzIwMjAy
LXJlc2V0KQ0KDQoNCkkndmUgdGVzdGVkIHRoZSBrZXJuZWwgZnJvbSB5b3VyIGdpdCByZXBvIGFu
ZCBpdCB3b3Jrcy4gSSd2ZSBib290ZWQgdGhlDQprZXJuZWwsIGF0dGFjaGVkIGFuIFVTQjMuMCBk
aXNrIGFuZCBjb3BpZWQgc29tZSBHQnMgZnJvbSBpdCB3aXRob3V0IGFueQ0KcHJvYmxlbXMuIEhl
cmUncyB0aGUgZG1lc2cgWEhDSSBvdXRwdXQuDQoNCg0KWyAgICAxLjU2NzIyMF0geGhjaV9oY2Qg
MDAwMDowMzowMC4wOiB4SENJIEhvc3QgQ29udHJvbGxlcg0KWyAgICAxLjU2NzIzOF0geGhjaV9o
Y2QgMDAwMDowMzowMC4wOiBuZXcgVVNCIGJ1cyByZWdpc3RlcmVkLCBhc3NpZ25lZA0KYnVzIG51
bWJlciAyDQpbICAgIDEuNTY3MjY0XSB4aGNpX2hjZCAwMDAwOjAzOjAwLjA6IFplcm9pbmcgNjRi
aXQgYmFzZSByZWdpc3RlcnMsDQpleHBlY3RpbmcgZmF1bHQNClsgICAgMS42MDk0MjFdIHhoY2lf
aGNkIDAwMDA6MDM6MDAuMDogaGNjIHBhcmFtcyAweDAxNDA1MWNmIGhjaSB2ZXJzaW9uDQoweDEw
MCBxdWlya3MgMHg4MDAwMDQxMA0KWyAgICAxLjYwOTkzMV0gdXNiIHVzYjI6IE1hbnVmYWN0dXJl
cjogTGludXggNC4xNi4wLXJjNi1zZXJ2ZXJ2NCsgeGhjaS0NCmhjZA0KWyAgICAxLjYxMDQ2OV0g
eGhjaV9oY2QgMDAwMDowMzowMC4wOiB4SENJIEhvc3QgQ29udHJvbGxlcg0KWyAgICAxLjYxMDQ3
OV0geGhjaV9oY2QgMDAwMDowMzowMC4wOiBuZXcgVVNCIGJ1cyByZWdpc3RlcmVkLCBhc3NpZ25l
ZA0KYnVzIG51bWJlciAzDQpbICAgIDEuNjEzMDY4XSB1c2IgdXNiMzogTWFudWZhY3R1cmVyOiBM
aW51eCA0LjE2LjAtcmM2LXNlcnZlcnY0KyB4aGMNCmktaGNkDQoNClRoYW5rcywNCg0KCUFybmUN
Cg0K
---
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

patch 8466489ef5ba48272ba4fa4ea9f8f403306de4c7 breaks Renesas USB3 controller functionality

[Bockholdt Arne]

DQoNCk9uIFRodSwgMjAxOC0wNC0xMiBhdCAwNzoyMCArMDIwMCwgQXJkIEJpZXNoZXV2ZWwgd3Jv
dGU6DQo+IE9uIDEyIEFwcmlsIDIwMTggYXQgMDc6MDUsIEJvY2tob2xkdCBBcm5lDQo+IDxhLmJv
Y2tob2xkdEBwcmVjaXRlYy1vcHRyb25pay5kZT4gd3JvdGU6DQo+ID4gDQo+ID4gT24gV2VkLCAy
MDE4LTA0LTExIGF0IDE1OjAyICswMTAwLCBNYXJjIFp5bmdpZXIgd3JvdGU6DQo+ID4gDQo+ID4g
T24gV2VkLCAxMSBBcHIgMjAxOCAxNDoxMTo1MiArMDEwMCwNCj4gPiBCb2NraG9sZHQgQXJuZSB3
cm90ZToNCj4gPiANCj4gPiANCj4gPiBIaSBhbGwsDQo+ID4gDQo+ID4gaXMgdGhlcmUgYW55dGhp
bmcgbmV3LCBJJ3ZlIGp1c3QgdHJpZWQgdGhlIG5ldyBzdGFibGUgNC4xNi4xIGtlcm5lbA0KPiA+
IHdpdGhvdXQgYW55IGNoYW5nZS4gVGhlIFJlbmVzeXMgVVNCMyBjb250cm9sbGVyIGlzIHN0aWxs
IG5vdA0KPiA+IGZ1bmN0aW9uYWwuIEknbSB3aWxsaW5nIHRvIHRlc3QgYW55IHBhdGNoIHRoYXQg
aXMgYmFzZWQgb24gYSBzdGFibGUNCj4gPiBrZXJuZWwgdmVyc2lvbiBiZWNhdXNlIHRoZSBtYWNo
aW5lIGlzIGluIHByb2R1Y3Rpb24gdXNlLg0KPiA+IA0KPiA+IA0KPiA+IEhhdmUgeW91IHRlc3Rl
ZCB0aGUgYnJhbmNoWzFdIEkgbWVudGlvbmVkIGluIG15IHByZXZpb3VzIGVtYWlsPw0KPiA+IFdp
dGhvdXQgeW91ciBmZWVkYmFjaywgSSBjYW5ub3QgcmVhbGx5IG1ha2UgbXVjaCBwcm9ncmVzcyBv
biB0aGlzLg0KPiA+IA0KPiA+IFRoYW5rcywNCj4gPiANCj4gPiBNLg0KPiA+IA0KPiA+IFsxXSBo
dHRwczovL3d3dy5zcGluaWNzLm5ldC9saXN0cy9saW51eC11c2IvbXNnMTY3MzAxLmh0bWwNCj4g
PiANCj4gPiANCj4gPiBJdCBzZWVtcyB0aGUgcmVwbyBpcyBiYXNlZCBvbiAzLjkga2VybmVsIGFu
ZCBub3Qgb24gYSBjdXJyZW50DQo+ID4gc3RhYmxlIGJyYW5jaCwNCj4gPiBpc24ndCBpdD8gSXQn
cyByYXRoZXIgb2xkLCBJJ20gbm90IHN1cmUgbXkgc2V0dXAgd2lsbCB3b3JrIHdpdGgNCj4gPiB0
aGlzIG9sZA0KPiA+IGtlcm5lbC4NCj4gPiANCj4gDQo+IEFyZSB5b3Ugc3VyZSB5b3UgcHVsbGVk
IHRoZSBjb3JyZWN0IGJyYW5jaD8gdXNiL3VQRDcyMDIwMi1yZXNldCBoYXMNCj4gdGhlIGZvbGxv
d2luZyBvbiB0b3Agb2YgKnY0LjE2LXJjNioNCj4gDQo+IFJldmVydCAieGhjaTogUmVzZXQgUmVu
ZXNhcyB1UEQ3MjAyMHggVVNCIGNvbnRyb2xsZXIgZm9yIDMyLWJpdCBETUENCj4gaXNzdWUiDQo+
IHhoY2k6IEFkZCBxdWlyayB0byB6ZXJvIDY0Yml0IHJlZ2lzdGVycyBvbiBSZW5lc2FzIFBDSWUg
Y29udHJvbGxlcnMNCj4gDQo+IChodHRwczovL2dpdC5rZXJuZWwub3JnL3B1Yi9zY20vbGludXgv
a2VybmVsL2dpdC9tYXovYXJtLXBsYXRmb3Jtcy5naQ0KPiB0L2xvZy8/aD11c2IvdVBENzIwMjAy
LXJlc2V0KQ0KDQpPa2F5LCB0aGlzIHdhcyBteSBmYXVsdCwgc29ycnkuIEknbSBqdXN0IGNvbXBp
bGluZyB0aGUga2VybmVsIHdpdGggbXkNCmNvbmZpZyBhbmQgd2lsbCBnaXZlIGl0IGEgdHJ5IGF0
IHRoZSB3ZWVrZW5kLiANCg0KDQpUaGFua3MsDQoNCglBcm5l
---
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

patch 8466489ef5ba48272ba4fa4ea9f8f403306de4c7 breaks Renesas USB3 controller functionality

[Ard Biesheuvel]

On 12 April 2018 at 07:05, Bockholdt Arne
<a.bockholdt@precitec-optronik.de> wrote:
>
> On Wed, 2018-04-11 at 15:02 +0100, Marc Zyngier wrote:
>
> On Wed, 11 Apr 2018 14:11:52 +0100,
> Bockholdt Arne wrote:
>
>
> Hi all,
>
> is there anything new, I've just tried the new stable 4.16.1 kernel
> without any change. The Renesys USB3 controller is still not
> functional. I'm willing to test any patch that is based on a stable
> kernel version because the machine is in production use.
>
>
> Have you tested the branch[1] I mentioned in my previous email?
> Without your feedback, I cannot really make much progress on this.
>
> Thanks,
>
> M.
>
> [1] https://www.spinics.net/lists/linux-usb/msg167301.html
>
>
> It seems the repo is based on 3.9 kernel and not on a current stable branch,
> isn't it? It's rather old, I'm not sure my setup will work with this old
> kernel.
>

Are you sure you pulled the correct branch? usb/uPD720202-reset has
the following on top of *v4.16-rc6*

Revert "xhci: Reset Renesas uPD72020x USB controller for 32-bit DMA issue"
xhci: Add quirk to zero 64bit registers on Renesas PCIe controllers

(https://git.kernel.org/pub/scm/linux/kernel/git/maz/arm-platforms.git/log/?h=usb/uPD720202-reset)

> Do you have a patch for a less dated kernel?
>
> Thank you,
>
> Arne
---
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

patch 8466489ef5ba48272ba4fa4ea9f8f403306de4c7 breaks Renesas USB3 controller functionality

[Marc Zyngier]

On Wed, 11 Apr 2018 14:11:52 +0100,
Bockholdt Arne wrote:
> 
> Hi all,
> 
> is there anything new, I've just tried the new stable 4.16.1 kernel
> without any change. The Renesys USB3 controller is still not
> functional. I'm willing to test any patch that is based on a stable
> kernel version because the machine is in production use.

Have you tested the branch[1] I mentioned in my previous email?
Without your feedback, I cannot really make much progress on this.

Thanks,

	M.
	
[1] https://www.spinics.net/lists/linux-usb/msg167301.html

patch 8466489ef5ba48272ba4fa4ea9f8f403306de4c7 breaks Renesas USB3 controller functionality

[Marc Zyngier]

On 25/03/18 15:39, Ard Biesheuvel wrote:
> 
> 
>> On 25 Mar 2018, at 15:14, Marc Zyngier <marc.zyngier@arm.com> wrote:
>>
>> On Sun, 25 Mar 2018 14:26:58 +0100
>> Ard Biesheuvel <ard.biesheuvel@linaro.org> wrote:
>>
>>>> On 25 March 2018 at 13:52, Marc Zyngier <marc.zyngier@arm.com> wrote:
>>>> On Sun, 25 Mar 2018 13:38:19 +0100,
>>>> Ard Biesheuvel wrote:  
>>>>>
>>>>>> On 25 March 2018 at 13:31, Marc Zyngier <marc.zyngier@arm.com> wrote:  
>>>>>> On Sun, 25 Mar 2018 12:57:55 +0100,
>>>>>> Ard Biesheuvel wrote:  
>>>>>>>
>>>>>>>> On 25 March 2018 at 12:51, Marc Zyngier <marc.zyngier@arm.com> wrote:  
>>>>>>>> On Sun, 25 Mar 2018 11:48:35 +0100,
>>>>>>>> Ard Biesheuvel wrote:
>>>>>>>>
>>>>>>>> Hi Ard,
>>>>>>>>
>>>>>>>> [...]
>>>>>>>>
>>>>>>>>>> I finally found some time to work on this, and came up with an
>>>>>>>>>> alternative approach (it turns out that this chip is even more
>>>>>>>>>> braindead than I thought).
>>>>>>>>>>
>>>>>>>>>> It is slightly scary, in the sense that the USB controller seems to
>>>>>>>>>> perform memory accesses even when halted, and can generate faults,
>>>>>>>>>> but it works just fine on my system. And with this, we can drop the
>>>>>>>>>> hard reset at boot time. I'm still on the fence to limit it to systems
>>>>>>>>>> with an iommu though.
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Hi Marc,
>>>>>>>>>
>>>>>>>>> I take it you tested this on Cello?  
>>>>>>>>
>>>>>>>> Tested on Cello indeed (I should have mentioned that the first place).
>>>>>>>>
>>>>>>>>> There, it might make sense to
>>>>>>>>> limit this to systems with an IOMMU, but not in the general case, I
>>>>>>>>> think. The reason is that it is not guaranteed that the firmware will
>>>>>>>>> use 32-bit addressable allocations for these data structures, even if
>>>>>>>>> the kernel is able to without an IOMMU. (UEFI on arm64 will not prefer
>>>>>>>>> 32-bit addressable memory for PCI DMA if it is available, and usually
>>>>>>>>> serves heap allocations [such as the ones used for these data
>>>>>>>>> structures] starting at the top of DRAM)  
>>>>>>>>
>>>>>>>> My main worry is that this controller will happily try and DMA from
>>>>>>>> zero as we wipe the 64bit registers, even when halted. On Seattle (and
>>>>>>>> thus Cello), this is just fine as there is nothing there, and the
>>>>>>>> controller aborts with the HSE bit set.
>>>>>>>>
>>>>>>>> On other systems, where memory actually exists at 0, who knows what
>>>>>>>> this is going to do? On the other hand, this is not worse than the
>>>>>>>> current situation, where we could end-up with any odd address...
>>>>>>>>
>>>>>>>
>>>>>>> Is the PCI_COMMAND_MASTER bit enabled at this point? What happens if
>>>>>>> you clear it first?  
>>>>>>
>>>>>> Tried that. No difference whatsoever, as I still get a fault with the
>>>>>> device accessing address 0, and being caught by the iommu.
>>>>>>
>>>>>
>>>>> Wow so this device is more broken than I thought.  
>>>>
>>>> That's my impression as well. I came to the conclusion that the only
>>>> way to make it behave is to crash it first, and then to reset it.
>>>>
>>>
>>> OK, so what if it doesn't crash? Without an IOMMU, it is quite likely
>>> that putting zeroes in the lower half of a 64-bit memory address
>>> produces a physical address that is valid, and so the device will
>>> still misbehave in that case.
>>>
>>> If making it crash is what kicks it into submission, could we perhaps
>>> use U64_MAX instead?
>>
>> Just tried that. It gets into some even uglier state, and never
>> recovers. Even doing U64_MAX, fault, and then zero doesn't help. The
>> problem is that it dies with something in the top 32 bits, and that's
>> pretty final.
>>
>> If really feels that without an iommu in the path, this device is
>> completely unsafe, and should never be fed 64bit addresses.
>>
> 
> ... unless we do the pci reset in the exitbootservices handler in
> uefi, which is probably the most robust way of handling this (or wire
> up the iommu)
> 
> i have cello smmu patches if you’re interested

Could be worth a try.

	M.

patch 8466489ef5ba48272ba4fa4ea9f8f403306de4c7 breaks Renesas USB3 controller functionality

[Ard Biesheuvel]

> On 25 Mar 2018, at 15:14, Marc Zyngier <marc.zyngier@arm.com> wrote:
> 
> On Sun, 25 Mar 2018 14:26:58 +0100
> Ard Biesheuvel <ard.biesheuvel@linaro.org> wrote:
> 
>>> On 25 March 2018 at 13:52, Marc Zyngier <marc.zyngier@arm.com> wrote:
>>> On Sun, 25 Mar 2018 13:38:19 +0100,
>>> Ard Biesheuvel wrote:  
>>>> 
>>>>> On 25 March 2018 at 13:31, Marc Zyngier <marc.zyngier@arm.com> wrote:  
>>>>> On Sun, 25 Mar 2018 12:57:55 +0100,
>>>>> Ard Biesheuvel wrote:  
>>>>>> 
>>>>>>> On 25 March 2018 at 12:51, Marc Zyngier <marc.zyngier@arm.com> wrote:  
>>>>>>> On Sun, 25 Mar 2018 11:48:35 +0100,
>>>>>>> Ard Biesheuvel wrote:
>>>>>>> 
>>>>>>> Hi Ard,
>>>>>>> 
>>>>>>> [...]
>>>>>>> 
>>>>>>>>> I finally found some time to work on this, and came up with an
>>>>>>>>> alternative approach (it turns out that this chip is even more
>>>>>>>>> braindead than I thought).
>>>>>>>>> 
>>>>>>>>> It is slightly scary, in the sense that the USB controller seems to
>>>>>>>>> perform memory accesses even when halted, and can generate faults,
>>>>>>>>> but it works just fine on my system. And with this, we can drop the
>>>>>>>>> hard reset at boot time. I'm still on the fence to limit it to systems
>>>>>>>>> with an iommu though.
>>>>>>>>> 
>>>>>>>> 
>>>>>>>> Hi Marc,
>>>>>>>> 
>>>>>>>> I take it you tested this on Cello?  
>>>>>>> 
>>>>>>> Tested on Cello indeed (I should have mentioned that the first place).
>>>>>>> 
>>>>>>>> There, it might make sense to
>>>>>>>> limit this to systems with an IOMMU, but not in the general case, I
>>>>>>>> think. The reason is that it is not guaranteed that the firmware will
>>>>>>>> use 32-bit addressable allocations for these data structures, even if
>>>>>>>> the kernel is able to without an IOMMU. (UEFI on arm64 will not prefer
>>>>>>>> 32-bit addressable memory for PCI DMA if it is available, and usually
>>>>>>>> serves heap allocations [such as the ones used for these data
>>>>>>>> structures] starting at the top of DRAM)  
>>>>>>> 
>>>>>>> My main worry is that this controller will happily try and DMA from
>>>>>>> zero as we wipe the 64bit registers, even when halted. On Seattle (and
>>>>>>> thus Cello), this is just fine as there is nothing there, and the
>>>>>>> controller aborts with the HSE bit set.
>>>>>>> 
>>>>>>> On other systems, where memory actually exists at 0, who knows what
>>>>>>> this is going to do? On the other hand, this is not worse than the
>>>>>>> current situation, where we could end-up with any odd address...
>>>>>>> 
>>>>>> 
>>>>>> Is the PCI_COMMAND_MASTER bit enabled at this point? What happens if
>>>>>> you clear it first?  
>>>>> 
>>>>> Tried that. No difference whatsoever, as I still get a fault with the
>>>>> device accessing address 0, and being caught by the iommu.
>>>>> 
>>>> 
>>>> Wow so this device is more broken than I thought.  
>>> 
>>> That's my impression as well. I came to the conclusion that the only
>>> way to make it behave is to crash it first, and then to reset it.
>>> 
>> 
>> OK, so what if it doesn't crash? Without an IOMMU, it is quite likely
>> that putting zeroes in the lower half of a 64-bit memory address
>> produces a physical address that is valid, and so the device will
>> still misbehave in that case.
>> 
>> If making it crash is what kicks it into submission, could we perhaps
>> use U64_MAX instead?
> 
> Just tried that. It gets into some even uglier state, and never
> recovers. Even doing U64_MAX, fault, and then zero doesn't help. The
> problem is that it dies with something in the top 32 bits, and that's
> pretty final.
> 
> If really feels that without an iommu in the path, this device is
> completely unsafe, and should never be fed 64bit addresses.
> 

... unless we do the pci reset in the exitbootservices handler in uefi, which is probably the most robust way of handling this (or wire up the iommu)

i have cello smmu patches if you’re interested--
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

patch 8466489ef5ba48272ba4fa4ea9f8f403306de4c7 breaks Renesas USB3 controller functionality

[Marc Zyngier]

On Sun, 25 Mar 2018 14:26:58 +0100
Ard Biesheuvel <ard.biesheuvel@linaro.org> wrote:

> On 25 March 2018 at 13:52, Marc Zyngier <marc.zyngier@arm.com> wrote:
> > On Sun, 25 Mar 2018 13:38:19 +0100,
> > Ard Biesheuvel wrote:  
> >>
> >> On 25 March 2018 at 13:31, Marc Zyngier <marc.zyngier@arm.com> wrote:  
> >> > On Sun, 25 Mar 2018 12:57:55 +0100,
> >> > Ard Biesheuvel wrote:  
> >> >>
> >> >> On 25 March 2018 at 12:51, Marc Zyngier <marc.zyngier@arm.com> wrote:  
> >> >> > On Sun, 25 Mar 2018 11:48:35 +0100,
> >> >> > Ard Biesheuvel wrote:
> >> >> >
> >> >> > Hi Ard,
> >> >> >
> >> >> > [...]
> >> >> >  
> >> >> >> > I finally found some time to work on this, and came up with an
> >> >> >> > alternative approach (it turns out that this chip is even more
> >> >> >> > braindead than I thought).
> >> >> >> >
> >> >> >> > It is slightly scary, in the sense that the USB controller seems to
> >> >> >> > perform memory accesses even when halted, and can generate faults,
> >> >> >> > but it works just fine on my system. And with this, we can drop the
> >> >> >> > hard reset at boot time. I'm still on the fence to limit it to systems
> >> >> >> > with an iommu though.
> >> >> >> >  
> >> >> >>
> >> >> >> Hi Marc,
> >> >> >>
> >> >> >> I take it you tested this on Cello?  
> >> >> >
> >> >> > Tested on Cello indeed (I should have mentioned that the first place).
> >> >> >  
> >> >> >> There, it might make sense to
> >> >> >> limit this to systems with an IOMMU, but not in the general case, I
> >> >> >> think. The reason is that it is not guaranteed that the firmware will
> >> >> >> use 32-bit addressable allocations for these data structures, even if
> >> >> >> the kernel is able to without an IOMMU. (UEFI on arm64 will not prefer
> >> >> >> 32-bit addressable memory for PCI DMA if it is available, and usually
> >> >> >> serves heap allocations [such as the ones used for these data
> >> >> >> structures] starting at the top of DRAM)  
> >> >> >
> >> >> > My main worry is that this controller will happily try and DMA from
> >> >> > zero as we wipe the 64bit registers, even when halted. On Seattle (and
> >> >> > thus Cello), this is just fine as there is nothing there, and the
> >> >> > controller aborts with the HSE bit set.
> >> >> >
> >> >> > On other systems, where memory actually exists at 0, who knows what
> >> >> > this is going to do? On the other hand, this is not worse than the
> >> >> > current situation, where we could end-up with any odd address...
> >> >> >  
> >> >>
> >> >> Is the PCI_COMMAND_MASTER bit enabled at this point? What happens if
> >> >> you clear it first?  
> >> >
> >> > Tried that. No difference whatsoever, as I still get a fault with the
> >> > device accessing address 0, and being caught by the iommu.
> >> >  
> >>
> >> Wow so this device is more broken than I thought.  
> >
> > That's my impression as well. I came to the conclusion that the only
> > way to make it behave is to crash it first, and then to reset it.
> >  
> 
> OK, so what if it doesn't crash? Without an IOMMU, it is quite likely
> that putting zeroes in the lower half of a 64-bit memory address
> produces a physical address that is valid, and so the device will
> still misbehave in that case.
> 
> If making it crash is what kicks it into submission, could we perhaps
> use U64_MAX instead?

Just tried that. It gets into some even uglier state, and never
recovers. Even doing U64_MAX, fault, and then zero doesn't help. The
problem is that it dies with something in the top 32 bits, and that's
pretty final.

If really feels that without an iommu in the path, this device is
completely unsafe, and should never be fed 64bit addresses.

	M.

patch 8466489ef5ba48272ba4fa4ea9f8f403306de4c7 breaks Renesas USB3 controller functionality

[Ard Biesheuvel]

On 25 March 2018 at 13:52, Marc Zyngier <marc.zyngier@arm.com> wrote:
> On Sun, 25 Mar 2018 13:38:19 +0100,
> Ard Biesheuvel wrote:
>>
>> On 25 March 2018 at 13:31, Marc Zyngier <marc.zyngier@arm.com> wrote:
>> > On Sun, 25 Mar 2018 12:57:55 +0100,
>> > Ard Biesheuvel wrote:
>> >>
>> >> On 25 March 2018 at 12:51, Marc Zyngier <marc.zyngier@arm.com> wrote:
>> >> > On Sun, 25 Mar 2018 11:48:35 +0100,
>> >> > Ard Biesheuvel wrote:
>> >> >
>> >> > Hi Ard,
>> >> >
>> >> > [...]
>> >> >
>> >> >> > I finally found some time to work on this, and came up with an
>> >> >> > alternative approach (it turns out that this chip is even more
>> >> >> > braindead than I thought).
>> >> >> >
>> >> >> > It is slightly scary, in the sense that the USB controller seems to
>> >> >> > perform memory accesses even when halted, and can generate faults,
>> >> >> > but it works just fine on my system. And with this, we can drop the
>> >> >> > hard reset at boot time. I'm still on the fence to limit it to systems
>> >> >> > with an iommu though.
>> >> >> >
>> >> >>
>> >> >> Hi Marc,
>> >> >>
>> >> >> I take it you tested this on Cello?
>> >> >
>> >> > Tested on Cello indeed (I should have mentioned that the first place).
>> >> >
>> >> >> There, it might make sense to
>> >> >> limit this to systems with an IOMMU, but not in the general case, I
>> >> >> think. The reason is that it is not guaranteed that the firmware will
>> >> >> use 32-bit addressable allocations for these data structures, even if
>> >> >> the kernel is able to without an IOMMU. (UEFI on arm64 will not prefer
>> >> >> 32-bit addressable memory for PCI DMA if it is available, and usually
>> >> >> serves heap allocations [such as the ones used for these data
>> >> >> structures] starting at the top of DRAM)
>> >> >
>> >> > My main worry is that this controller will happily try and DMA from
>> >> > zero as we wipe the 64bit registers, even when halted. On Seattle (and
>> >> > thus Cello), this is just fine as there is nothing there, and the
>> >> > controller aborts with the HSE bit set.
>> >> >
>> >> > On other systems, where memory actually exists at 0, who knows what
>> >> > this is going to do? On the other hand, this is not worse than the
>> >> > current situation, where we could end-up with any odd address...
>> >> >
>> >>
>> >> Is the PCI_COMMAND_MASTER bit enabled at this point? What happens if
>> >> you clear it first?
>> >
>> > Tried that. No difference whatsoever, as I still get a fault with the
>> > device accessing address 0, and being caught by the iommu.
>> >
>>
>> Wow so this device is more broken than I thought.
>
> That's my impression as well. I came to the conclusion that the only
> way to make it behave is to crash it first, and then to reset it.
>

OK, so what if it doesn't crash? Without an IOMMU, it is quite likely
that putting zeroes in the lower half of a 64-bit memory address
produces a physical address that is valid, and so the device will
still misbehave in that case.

If making it crash is what kicks it into submission, could we perhaps
use U64_MAX instead?
---
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

patch 8466489ef5ba48272ba4fa4ea9f8f403306de4c7 breaks Renesas USB3 controller functionality

[Marc Zyngier]

On Sun, 25 Mar 2018 13:38:19 +0100,
Ard Biesheuvel wrote:
> 
> On 25 March 2018 at 13:31, Marc Zyngier <marc.zyngier@arm.com> wrote:
> > On Sun, 25 Mar 2018 12:57:55 +0100,
> > Ard Biesheuvel wrote:
> >>
> >> On 25 March 2018 at 12:51, Marc Zyngier <marc.zyngier@arm.com> wrote:
> >> > On Sun, 25 Mar 2018 11:48:35 +0100,
> >> > Ard Biesheuvel wrote:
> >> >
> >> > Hi Ard,
> >> >
> >> > [...]
> >> >
> >> >> > I finally found some time to work on this, and came up with an
> >> >> > alternative approach (it turns out that this chip is even more
> >> >> > braindead than I thought).
> >> >> >
> >> >> > It is slightly scary, in the sense that the USB controller seems to
> >> >> > perform memory accesses even when halted, and can generate faults,
> >> >> > but it works just fine on my system. And with this, we can drop the
> >> >> > hard reset at boot time. I'm still on the fence to limit it to systems
> >> >> > with an iommu though.
> >> >> >
> >> >>
> >> >> Hi Marc,
> >> >>
> >> >> I take it you tested this on Cello?
> >> >
> >> > Tested on Cello indeed (I should have mentioned that the first place).
> >> >
> >> >> There, it might make sense to
> >> >> limit this to systems with an IOMMU, but not in the general case, I
> >> >> think. The reason is that it is not guaranteed that the firmware will
> >> >> use 32-bit addressable allocations for these data structures, even if
> >> >> the kernel is able to without an IOMMU. (UEFI on arm64 will not prefer
> >> >> 32-bit addressable memory for PCI DMA if it is available, and usually
> >> >> serves heap allocations [such as the ones used for these data
> >> >> structures] starting at the top of DRAM)
> >> >
> >> > My main worry is that this controller will happily try and DMA from
> >> > zero as we wipe the 64bit registers, even when halted. On Seattle (and
> >> > thus Cello), this is just fine as there is nothing there, and the
> >> > controller aborts with the HSE bit set.
> >> >
> >> > On other systems, where memory actually exists at 0, who knows what
> >> > this is going to do? On the other hand, this is not worse than the
> >> > current situation, where we could end-up with any odd address...
> >> >
> >>
> >> Is the PCI_COMMAND_MASTER bit enabled at this point? What happens if
> >> you clear it first?
> >
> > Tried that. No difference whatsoever, as I still get a fault with the
> > device accessing address 0, and being caught by the iommu.
> >
> 
> Wow so this device is more broken than I thought.

That's my impression as well. I came to the conclusion that the only
way to make it behave is to crash it first, and then to reset it.

> In UEFI, we rely on clearing the bus master bit to ensure that all
> hardware stops doing DMA when ExitBootServices is called, but this is
> clearly not enough.

A sensible thing to do, but this device is pretty bonkers.

	M.

patch 8466489ef5ba48272ba4fa4ea9f8f403306de4c7 breaks Renesas USB3 controller functionality

[Ard Biesheuvel]

On 25 March 2018 at 13:31, Marc Zyngier <marc.zyngier@arm.com> wrote:
> On Sun, 25 Mar 2018 12:57:55 +0100,
> Ard Biesheuvel wrote:
>>
>> On 25 March 2018 at 12:51, Marc Zyngier <marc.zyngier@arm.com> wrote:
>> > On Sun, 25 Mar 2018 11:48:35 +0100,
>> > Ard Biesheuvel wrote:
>> >
>> > Hi Ard,
>> >
>> > [...]
>> >
>> >> > I finally found some time to work on this, and came up with an
>> >> > alternative approach (it turns out that this chip is even more
>> >> > braindead than I thought).
>> >> >
>> >> > It is slightly scary, in the sense that the USB controller seems to
>> >> > perform memory accesses even when halted, and can generate faults,
>> >> > but it works just fine on my system. And with this, we can drop the
>> >> > hard reset at boot time. I'm still on the fence to limit it to systems
>> >> > with an iommu though.
>> >> >
>> >>
>> >> Hi Marc,
>> >>
>> >> I take it you tested this on Cello?
>> >
>> > Tested on Cello indeed (I should have mentioned that the first place).
>> >
>> >> There, it might make sense to
>> >> limit this to systems with an IOMMU, but not in the general case, I
>> >> think. The reason is that it is not guaranteed that the firmware will
>> >> use 32-bit addressable allocations for these data structures, even if
>> >> the kernel is able to without an IOMMU. (UEFI on arm64 will not prefer
>> >> 32-bit addressable memory for PCI DMA if it is available, and usually
>> >> serves heap allocations [such as the ones used for these data
>> >> structures] starting at the top of DRAM)
>> >
>> > My main worry is that this controller will happily try and DMA from
>> > zero as we wipe the 64bit registers, even when halted. On Seattle (and
>> > thus Cello), this is just fine as there is nothing there, and the
>> > controller aborts with the HSE bit set.
>> >
>> > On other systems, where memory actually exists at 0, who knows what
>> > this is going to do? On the other hand, this is not worse than the
>> > current situation, where we could end-up with any odd address...
>> >
>>
>> Is the PCI_COMMAND_MASTER bit enabled at this point? What happens if
>> you clear it first?
>
> Tried that. No difference whatsoever, as I still get a fault with the
> device accessing address 0, and being caught by the iommu.
>

Wow so this device is more broken than I thought.

In UEFI, we rely on clearing the bus master bit to ensure that all
hardware stops doing DMA when ExitBootServices is called, but this is
clearly not enough.
---
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

patch 8466489ef5ba48272ba4fa4ea9f8f403306de4c7 breaks Renesas USB3 controller functionality

[Marc Zyngier]

On Sun, 25 Mar 2018 12:57:55 +0100,
Ard Biesheuvel wrote:
> 
> On 25 March 2018 at 12:51, Marc Zyngier <marc.zyngier@arm.com> wrote:
> > On Sun, 25 Mar 2018 11:48:35 +0100,
> > Ard Biesheuvel wrote:
> >
> > Hi Ard,
> >
> > [...]
> >
> >> > I finally found some time to work on this, and came up with an
> >> > alternative approach (it turns out that this chip is even more
> >> > braindead than I thought).
> >> >
> >> > It is slightly scary, in the sense that the USB controller seems to
> >> > perform memory accesses even when halted, and can generate faults,
> >> > but it works just fine on my system. And with this, we can drop the
> >> > hard reset at boot time. I'm still on the fence to limit it to systems
> >> > with an iommu though.
> >> >
> >>
> >> Hi Marc,
> >>
> >> I take it you tested this on Cello?
> >
> > Tested on Cello indeed (I should have mentioned that the first place).
> >
> >> There, it might make sense to
> >> limit this to systems with an IOMMU, but not in the general case, I
> >> think. The reason is that it is not guaranteed that the firmware will
> >> use 32-bit addressable allocations for these data structures, even if
> >> the kernel is able to without an IOMMU. (UEFI on arm64 will not prefer
> >> 32-bit addressable memory for PCI DMA if it is available, and usually
> >> serves heap allocations [such as the ones used for these data
> >> structures] starting at the top of DRAM)
> >
> > My main worry is that this controller will happily try and DMA from
> > zero as we wipe the 64bit registers, even when halted. On Seattle (and
> > thus Cello), this is just fine as there is nothing there, and the
> > controller aborts with the HSE bit set.
> >
> > On other systems, where memory actually exists at 0, who knows what
> > this is going to do? On the other hand, this is not worse than the
> > current situation, where we could end-up with any odd address...
> >
> 
> Is the PCI_COMMAND_MASTER bit enabled at this point? What happens if
> you clear it first?

Tried that. No difference whatsoever, as I still get a fault with the
device accessing address 0, and being caught by the iommu.

	M.

patch 8466489ef5ba48272ba4fa4ea9f8f403306de4c7 breaks Renesas USB3 controller functionality

[Ard Biesheuvel]

On 25 March 2018 at 12:51, Marc Zyngier <marc.zyngier@arm.com> wrote:
> On Sun, 25 Mar 2018 11:48:35 +0100,
> Ard Biesheuvel wrote:
>
> Hi Ard,
>
> [...]
>
>> > I finally found some time to work on this, and came up with an
>> > alternative approach (it turns out that this chip is even more
>> > braindead than I thought).
>> >
>> > It is slightly scary, in the sense that the USB controller seems to
>> > perform memory accesses even when halted, and can generate faults,
>> > but it works just fine on my system. And with this, we can drop the
>> > hard reset at boot time. I'm still on the fence to limit it to systems
>> > with an iommu though.
>> >
>>
>> Hi Marc,
>>
>> I take it you tested this on Cello?
>
> Tested on Cello indeed (I should have mentioned that the first place).
>
>> There, it might make sense to
>> limit this to systems with an IOMMU, but not in the general case, I
>> think. The reason is that it is not guaranteed that the firmware will
>> use 32-bit addressable allocations for these data structures, even if
>> the kernel is able to without an IOMMU. (UEFI on arm64 will not prefer
>> 32-bit addressable memory for PCI DMA if it is available, and usually
>> serves heap allocations [such as the ones used for these data
>> structures] starting at the top of DRAM)
>
> My main worry is that this controller will happily try and DMA from
> zero as we wipe the 64bit registers, even when halted. On Seattle (and
> thus Cello), this is just fine as there is nothing there, and the
> controller aborts with the HSE bit set.
>
> On other systems, where memory actually exists at 0, who knows what
> this is going to do? On the other hand, this is not worse than the
> current situation, where we could end-up with any odd address...
>

Is the PCI_COMMAND_MASTER bit enabled at this point? What happens if
you clear it first?
---
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

patch 8466489ef5ba48272ba4fa4ea9f8f403306de4c7 breaks Renesas USB3 controller functionality

[Marc Zyngier]

On Sun, 25 Mar 2018 11:48:35 +0100,
Ard Biesheuvel wrote:

Hi Ard,

[...]

> > I finally found some time to work on this, and came up with an
> > alternative approach (it turns out that this chip is even more
> > braindead than I thought).
> >
> > It is slightly scary, in the sense that the USB controller seems to
> > perform memory accesses even when halted, and can generate faults,
> > but it works just fine on my system. And with this, we can drop the
> > hard reset at boot time. I'm still on the fence to limit it to systems
> > with an iommu though.
> >
> 
> Hi Marc,
> 
> I take it you tested this on Cello?

Tested on Cello indeed (I should have mentioned that the first place).

> There, it might make sense to
> limit this to systems with an IOMMU, but not in the general case, I
> think. The reason is that it is not guaranteed that the firmware will
> use 32-bit addressable allocations for these data structures, even if
> the kernel is able to without an IOMMU. (UEFI on arm64 will not prefer
> 32-bit addressable memory for PCI DMA if it is available, and usually
> serves heap allocations [such as the ones used for these data
> structures] starting at the top of DRAM)

My main worry is that this controller will happily try and DMA from
zero as we wipe the 64bit registers, even when halted. On Seattle (and
thus Cello), this is just fine as there is nothing there, and the
controller aborts with the HSE bit set.

On other systems, where memory actually exists at 0, who knows what
this is going to do? On the other hand, this is not worse than the
current situation, where we could end-up with any odd address...

	M.

patch 8466489ef5ba48272ba4fa4ea9f8f403306de4c7 breaks Renesas USB3 controller functionality

[Ard Biesheuvel]

On 25 March 2018 at 11:37, Marc Zyngier <marc.zyngier@arm.com> wrote:
> On Fri, 02 Mar 2018 17:38:26 +0000,
> Bockholdt Arne wrote:
>
> Hi Arne,
>
>>
>> On Thu, 2018-03-01 at 17:37 +0000, Marc Zyngier wrote:
>> > On 01/03/18 08:01, Bockholdt Arne wrote:
>> > >
>> > > On Thu, 2018-02-15 at 19:29 +0000, Marc Zyngier wrote:
>> > > > [+ Ard, who helped me chasing the initial issue]
>> > > >
>> > > > On 15/02/18 06:43, Bockholdt Arne wrote:
>> > > > > Hi all,
>> > > > >
>> > > > > on our Intel Atom C2578 server with a SuperMicro A1SAi board
>> > > > > and a
>> > > > > Renesas uPD720201 USB 3.0 host controller the controller has
>> > > > > stopped
>> > > > > working since kernel 4.13.x. Before that kernel the dmesg
>> > > > > messages
>> > > > > from
>> > > > > XHCI were:
>> > > > >
>> > > > > dmesg-4.12.1-serverv4.log:xhci_hcd 0000:03:00.0: xHCI Host
>> > > > > Controller
>> > > > > dmesg-4.12.1-serverv4.log:xhci_hcd 0000:03:00.0: new USB bus
>> > > > > registered,
>> > > > > assigned bus number 2
>> > > > > dmesg-4.12.1-serverv4.log:xhci_hcd 0000:03:00.0: hcc params
>> > > > > 0x014051cf
>> > > > > hci version 0x100 quirks 0x00000010
>> > > > > dmesg-4.12.1-serverv4.log:usb usb2: Manufacturer: Linux 4.12.1-
>> > > > > serverv4
>> > > > > xhci-hcd
>> > > > > dmesg-4.12.1-serverv4.log:xhci_hcd 0000:03:00.0: xHCI Host
>> > > > > Controller
>> > > > > dmesg-4.12.1-serverv4.log:xhci_hcd 0000:03:00.0: new USB bus
>> > > > > registered,
>> > > > > assigned bus number 3
>> > > > > dmesg-4.12.1-serverv4.log:usb usb3: Manufacturer: Linux 4.12.1-
>> > > > > serverv4
>> > > > > xhci-hcd
>> > > > >
>> > > > > After that the message look like that:
>> > > > >
>> > > > > dmesg-4.13.1-serverv4.log:xhci_hcd 0000:03:00.0: Resetting
>> > > > > dmesg-4.13.1-serverv4.log:xhci_hcd 0000:03:00.0: Refused to
>> > > > > change
>> > > > > power
>> > > > > state, currently in D3
>> > > > > dmesg-4.13.1-serverv4.log:xhci_hcd 0000:03:00.0: xHCI Host
>> > > > > Controller
>> > > > > dmesg-4.13.1-serverv4.log:xhci_hcd 0000:03:00.0: new USB bus
>> > > > > registered,
>> > > > > assigned bus number 2
>> > > > > dmesg-4.13.1-serverv4.log:xhci_hcd 0000:03:00.0: Host halt
>> > > > > failed,
>> > > > > -19
>> > > > > dmesg-4.13.1-serverv4.log:xhci_hcd 0000:03:00.0: can't setup:
>> > > > > -19
>> > > > > dmesg-4.13.1-serverv4.log:xhci_hcd 0000:03:00.0: USB bus 2
>> > > > > deregistered
>> > > > > dmesg-4.13.1-serverv4.log:xhci_hcd 0000:03:00.0: init
>> > > > > 0000:03:00.0
>> > > > > fail, -19
>> > > > >
>> > > > > I've tested it with 4.15.3 too, it's still the same. I've
>> > > > > narrowed
>> > > > > it
>> > > > > down to the following patch:
>> > > > >
>> > > > > commit 8466489ef5ba48272ba4fa4ea9f8f403306de4c7
>> > > > > Author: Marc Zyngier <marc.zyngier@arm.com <mailto:marc.zyngier
>> > > > > @arm
>> > > > > .com>>
>> > > > > Date:   Tue Aug 1 20:11:08 2017 -0500
>> > > > >
>> > > > >     xhci: Reset Renesas uPD72020x USB controller for 32-bit DMA
>> > > > > issue
>> > > > >
>> > > > > Reverting the patch on top of 4.15.3 restores the USB3
>> > > > > functionality on
>> > > > > our server. Please let me know if there is anything I can do to
>> > > > > fix
>> > > > > the
>> > > > > problem. Thank you.
>> > > >
>> > > > Hi Arne,
>> > > >
>> > > > This looks pretty bad. I suspect that once reset, the firmware is
>> > > > lost.
>> > > > I'll try to write a patch dumping some information about it.
>> > > >
>> > > > Ard: Do you know if the Cello board has a SPI flash connected to
>> > > > the
>> > > > Renesas chip, from which it would load the firmware?
>> > > >
>> > > > Another possibility is that power management kicks in, and that
>> > > > the
>> > > > endpoint is stuck there. Could also be firmware related, but not
>> > > > only.
>> > > > I'd welcome any idea on the subject, as I cannot reproduce this
>> > > > issue
>> > > > on
>> > > > the HW I have.
>> > > >
>> > > > It we cannot work out what exactly is causing this, we may have
>> > > > to
>> > > > default to not resetting the part and rely on a command-line
>> > > > option
>> > > > to
>> > > > do it... I can't say I'm a fan.
>> > > >
>> > > > Thanks,
>> > > >
>> > > >         M.
>> > > >
>> > >
>> > > Hi Marc,
>> > >
>> > > I've tested it with 4.15.7 and it's still there. Is there anything
>> > > that
>> > > I can do to help fixing this problem?
>> >
>> > Would you mind trying the following patch and let me know if it
>> > helps?
>> > It is not exactly pretty, but we can polish it if that solves your
>> > issue.
>
> [...]
>
>> I've applied your patch on top of 4.15.7 and tried it on the server,
>> unfortunately the problem is still there. Here's the output from dmesg:
>>
>> [    1.570115] xhci_hcd 0000:03:00.0: Found a 64bit address in ERSTBA 4
>> [    1.570120] xhci_hcd 0000:03:00.0: Resetting
>> [    2.668066] xhci_hcd 0000:03:00.0: Refused to change power state,
>> currently in D3
>> [    2.668215] xhci_hcd 0000:03:00.0: xHCI Host Controller
>> [    2.668225] xhci_hcd 0000:03:00.0: new USB bus registered, assigned
>> bus number 2
>> [    2.668240] xhci_hcd 0000:03:00.0: Host halt failed, -19
>> [    2.668242] xhci_hcd 0000:03:00.0: can't setup: -19
>> [    2.668299] xhci_hcd 0000:03:00.0: USB bus 2 deregistered
>> [    2.668354] xhci_hcd 0000:03:00.0: init 0000:03:00.0 fail, -19
>>
>> If you need more informations to find the cause, I will gladly provide
>> it.
>
> I finally found some time to work on this, and came up with an
> alternative approach (it turns out that this chip is even more
> braindead than I thought).
>
> It is slightly scary, in the sense that the USB controller seems to
> perform memory accesses even when halted, and can generate faults,
> but it works just fine on my system. And with this, we can drop the
> hard reset at boot time. I'm still on the fence to limit it to systems
> with an iommu though.
>

Hi Marc,

I take it you tested this on Cello? There, it might make sense to
limit this to systems with an IOMMU, but not in the general case, I
think. The reason is that it is not guaranteed that the firmware will
use 32-bit addressable allocations for these data structures, even if
the kernel is able to without an IOMMU. (UEFI on arm64 will not prefer
32-bit addressable memory for PCI DMA if it is available, and usually
serves heap allocations [such as the ones used for these data
structures] starting at the top of DRAM)
---
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

patch 8466489ef5ba48272ba4fa4ea9f8f403306de4c7 breaks Renesas USB3 controller functionality

[Marc Zyngier]

On Fri, 02 Mar 2018 17:38:26 +0000,
Bockholdt Arne wrote:

Hi Arne,

> 
> On Thu, 2018-03-01 at 17:37 +0000, Marc Zyngier wrote:
> > On 01/03/18 08:01, Bockholdt Arne wrote:
> > > 
> > > On Thu, 2018-02-15 at 19:29 +0000, Marc Zyngier wrote:
> > > > [+ Ard, who helped me chasing the initial issue]
> > > > 
> > > > On 15/02/18 06:43, Bockholdt Arne wrote:
> > > > > Hi all,
> > > > > 
> > > > > on our Intel Atom C2578 server with a SuperMicro A1SAi board
> > > > > and a
> > > > > Renesas uPD720201 USB 3.0 host controller the controller has
> > > > > stopped
> > > > > working since kernel 4.13.x. Before that kernel the dmesg
> > > > > messages
> > > > > from
> > > > > XHCI were:
> > > > > 
> > > > > dmesg-4.12.1-serverv4.log:xhci_hcd 0000:03:00.0: xHCI Host
> > > > > Controller
> > > > > dmesg-4.12.1-serverv4.log:xhci_hcd 0000:03:00.0: new USB bus
> > > > > registered,
> > > > > assigned bus number 2
> > > > > dmesg-4.12.1-serverv4.log:xhci_hcd 0000:03:00.0: hcc params
> > > > > 0x014051cf
> > > > > hci version 0x100 quirks 0x00000010
> > > > > dmesg-4.12.1-serverv4.log:usb usb2: Manufacturer: Linux 4.12.1-
> > > > > serverv4
> > > > > xhci-hcd
> > > > > dmesg-4.12.1-serverv4.log:xhci_hcd 0000:03:00.0: xHCI Host
> > > > > Controller
> > > > > dmesg-4.12.1-serverv4.log:xhci_hcd 0000:03:00.0: new USB bus
> > > > > registered,
> > > > > assigned bus number 3
> > > > > dmesg-4.12.1-serverv4.log:usb usb3: Manufacturer: Linux 4.12.1-
> > > > > serverv4
> > > > > xhci-hcd
> > > > > 
> > > > > After that the message look like that:
> > > > > 
> > > > > dmesg-4.13.1-serverv4.log:xhci_hcd 0000:03:00.0: Resetting
> > > > > dmesg-4.13.1-serverv4.log:xhci_hcd 0000:03:00.0: Refused to
> > > > > change
> > > > > power
> > > > > state, currently in D3
> > > > > dmesg-4.13.1-serverv4.log:xhci_hcd 0000:03:00.0: xHCI Host
> > > > > Controller
> > > > > dmesg-4.13.1-serverv4.log:xhci_hcd 0000:03:00.0: new USB bus
> > > > > registered,
> > > > > assigned bus number 2
> > > > > dmesg-4.13.1-serverv4.log:xhci_hcd 0000:03:00.0: Host halt
> > > > > failed,
> > > > > -19
> > > > > dmesg-4.13.1-serverv4.log:xhci_hcd 0000:03:00.0: can't setup:
> > > > > -19
> > > > > dmesg-4.13.1-serverv4.log:xhci_hcd 0000:03:00.0: USB bus 2
> > > > > deregistered
> > > > > dmesg-4.13.1-serverv4.log:xhci_hcd 0000:03:00.0: init
> > > > > 0000:03:00.0
> > > > > fail, -19
> > > > > 
> > > > > I've tested it with 4.15.3 too, it's still the same. I've
> > > > > narrowed
> > > > > it
> > > > > down to the following patch:
> > > > > 
> > > > > commit 8466489ef5ba48272ba4fa4ea9f8f403306de4c7
> > > > > Author: Marc Zyngier <marc.zyngier@arm.com <mailto:marc.zyngier
> > > > > @arm
> > > > > .com>>
> > > > > Date:   Tue Aug 1 20:11:08 2017 -0500
> > > > > 
> > > > >     xhci: Reset Renesas uPD72020x USB controller for 32-bit DMA
> > > > > issue
> > > > > 
> > > > > Reverting the patch on top of 4.15.3 restores the USB3
> > > > > functionality on
> > > > > our server. Please let me know if there is anything I can do to
> > > > > fix
> > > > > the
> > > > > problem. Thank you.
> > > > 
> > > > Hi Arne,
> > > > 
> > > > This looks pretty bad. I suspect that once reset, the firmware is
> > > > lost.
> > > > I'll try to write a patch dumping some information about it.
> > > > 
> > > > Ard: Do you know if the Cello board has a SPI flash connected to
> > > > the
> > > > Renesas chip, from which it would load the firmware?
> > > > 
> > > > Another possibility is that power management kicks in, and that
> > > > the
> > > > endpoint is stuck there. Could also be firmware related, but not
> > > > only.
> > > > I'd welcome any idea on the subject, as I cannot reproduce this
> > > > issue
> > > > on
> > > > the HW I have.
> > > > 
> > > > It we cannot work out what exactly is causing this, we may have
> > > > to
> > > > default to not resetting the part and rely on a command-line
> > > > option
> > > > to
> > > > do it... I can't say I'm a fan.
> > > > 
> > > > Thanks,
> > > > 
> > > > 	M.
> > > > 
> > > 
> > > Hi Marc,
> > > 
> > > I've tested it with 4.15.7 and it's still there. Is there anything
> > > that
> > > I can do to help fixing this problem?
> > 
> > Would you mind trying the following patch and let me know if it
> > helps?
> > It is not exactly pretty, but we can polish it if that solves your
> > issue.

[...]

> I've applied your patch on top of 4.15.7 and tried it on the server,
> unfortunately the problem is still there. Here's the output from dmesg:
> 
> [    1.570115] xhci_hcd 0000:03:00.0: Found a 64bit address in ERSTBA 4
> [    1.570120] xhci_hcd 0000:03:00.0: Resetting
> [    2.668066] xhci_hcd 0000:03:00.0: Refused to change power state,
> currently in D3
> [    2.668215] xhci_hcd 0000:03:00.0: xHCI Host Controller
> [    2.668225] xhci_hcd 0000:03:00.0: new USB bus registered, assigned
> bus number 2
> [    2.668240] xhci_hcd 0000:03:00.0: Host halt failed, -19
> [    2.668242] xhci_hcd 0000:03:00.0: can't setup: -19
> [    2.668299] xhci_hcd 0000:03:00.0: USB bus 2 deregistered
> [    2.668354] xhci_hcd 0000:03:00.0: init 0000:03:00.0 fail, -19
> 
> If you need more informations to find the cause, I will gladly provide
> it.

I finally found some time to work on this, and came up with an
alternative approach (it turns out that this chip is even more
braindead than I thought).

It is slightly scary, in the sense that the USB controller seems to
perform memory accesses even when halted, and can generate faults,
but it works just fine on my system. And with this, we can drop the
hard reset at boot time. I'm still on the fence to limit it to systems
with an iommu though.

Could you please give this branch[1] a go?

Thanks,

	M.

[1] git://git.kernel.org/pub/scm/linux/kernel/git/maz/arm-platforms.git usb/uPD720202-reset

patch 8466489ef5ba48272ba4fa4ea9f8f403306de4c7 breaks Renesas USB3 controller functionality

[Bockholdt Arne]

On Thu, 2018-03-01 at 17:37 +0000, Marc Zyngier wrote:
> On 01/03/18 08:01, Bockholdt Arne wrote:
> > 
> > On Thu, 2018-02-15 at 19:29 +0000, Marc Zyngier wrote:
> > > [+ Ard, who helped me chasing the initial issue]
> > > 
> > > On 15/02/18 06:43, Bockholdt Arne wrote:
> > > > Hi all,
> > > > 
> > > > on our Intel Atom C2578 server with a SuperMicro A1SAi board
> > > > and a
> > > > Renesas uPD720201 USB 3.0 host controller the controller has
> > > > stopped
> > > > working since kernel 4.13.x. Before that kernel the dmesg
> > > > messages
> > > > from
> > > > XHCI were:
> > > > 
> > > > dmesg-4.12.1-serverv4.log:xhci_hcd 0000:03:00.0: xHCI Host
> > > > Controller
> > > > dmesg-4.12.1-serverv4.log:xhci_hcd 0000:03:00.0: new USB bus
> > > > registered,
> > > > assigned bus number 2
> > > > dmesg-4.12.1-serverv4.log:xhci_hcd 0000:03:00.0: hcc params
> > > > 0x014051cf
> > > > hci version 0x100 quirks 0x00000010
> > > > dmesg-4.12.1-serverv4.log:usb usb2: Manufacturer: Linux 4.12.1-
> > > > serverv4
> > > > xhci-hcd
> > > > dmesg-4.12.1-serverv4.log:xhci_hcd 0000:03:00.0: xHCI Host
> > > > Controller
> > > > dmesg-4.12.1-serverv4.log:xhci_hcd 0000:03:00.0: new USB bus
> > > > registered,
> > > > assigned bus number 3
> > > > dmesg-4.12.1-serverv4.log:usb usb3: Manufacturer: Linux 4.12.1-
> > > > serverv4
> > > > xhci-hcd
> > > > 
> > > > After that the message look like that:
> > > > 
> > > > dmesg-4.13.1-serverv4.log:xhci_hcd 0000:03:00.0: Resetting
> > > > dmesg-4.13.1-serverv4.log:xhci_hcd 0000:03:00.0: Refused to
> > > > change
> > > > power
> > > > state, currently in D3
> > > > dmesg-4.13.1-serverv4.log:xhci_hcd 0000:03:00.0: xHCI Host
> > > > Controller
> > > > dmesg-4.13.1-serverv4.log:xhci_hcd 0000:03:00.0: new USB bus
> > > > registered,
> > > > assigned bus number 2
> > > > dmesg-4.13.1-serverv4.log:xhci_hcd 0000:03:00.0: Host halt
> > > > failed,
> > > > -19
> > > > dmesg-4.13.1-serverv4.log:xhci_hcd 0000:03:00.0: can't setup:
> > > > -19
> > > > dmesg-4.13.1-serverv4.log:xhci_hcd 0000:03:00.0: USB bus 2
> > > > deregistered
> > > > dmesg-4.13.1-serverv4.log:xhci_hcd 0000:03:00.0: init
> > > > 0000:03:00.0
> > > > fail, -19
> > > > 
> > > > I've tested it with 4.15.3 too, it's still the same. I've
> > > > narrowed
> > > > it
> > > > down to the following patch:
> > > > 
> > > > commit 8466489ef5ba48272ba4fa4ea9f8f403306de4c7
> > > > Author: Marc Zyngier <marc.zyngier@arm.com <mailto:marc.zyngier
> > > > @arm
> > > > .com>>
> > > > Date:   Tue Aug 1 20:11:08 2017 -0500
> > > > 
> > > >     xhci: Reset Renesas uPD72020x USB controller for 32-bit DMA
> > > > issue
> > > > 
> > > > Reverting the patch on top of 4.15.3 restores the USB3
> > > > functionality on
> > > > our server. Please let me know if there is anything I can do to
> > > > fix
> > > > the
> > > > problem. Thank you.
> > > 
> > > Hi Arne,
> > > 
> > > This looks pretty bad. I suspect that once reset, the firmware is
> > > lost.
> > > I'll try to write a patch dumping some information about it.
> > > 
> > > Ard: Do you know if the Cello board has a SPI flash connected to
> > > the
> > > Renesas chip, from which it would load the firmware?
> > > 
> > > Another possibility is that power management kicks in, and that
> > > the
> > > endpoint is stuck there. Could also be firmware related, but not
> > > only.
> > > I'd welcome any idea on the subject, as I cannot reproduce this
> > > issue
> > > on
> > > the HW I have.
> > > 
> > > It we cannot work out what exactly is causing this, we may have
> > > to
> > > default to not resetting the part and rely on a command-line
> > > option
> > > to
> > > do it... I can't say I'm a fan.
> > > 
> > > Thanks,
> > > 
> > > 	M.
> > > 
> > 
> > Hi Marc,
> > 
> > I've tested it with 4.15.7 and it's still there. Is there anything
> > that
> > I can do to help fixing this problem?
> 
> Would you mind trying the following patch and let me know if it
> helps?
> It is not exactly pretty, but we can polish it if that solves your
> issue.
> 
> Thanks,
> 
> 	M.
> 
> From 9a253773b289f781b7114e887481939b3021bb30 Mon Sep 17 00:00:00
> 2001
> From: Marc Zyngier <marc.zyngier@arm.com>
> Date: Thu, 1 Mar 2018 17:27:42 +0000
> Subject: [PATCH] xhci: Only reset uPD72020x if programmed with 64bit
> DMA
>  addresses
> 
> The Renesas uPD72020x USB controller misbehaves when switching
> from 64bit DMA addresses to 32bit ones, and can only accept a
> new programming if hard reset first.
> 
> But it also misbehaves (in a different way) if reset for no
> good reason. So let's restrict the resetting to the cases where
> we detect the 64/32 issue, and leave the device alone in the
> other cases.
> 
> Fixes: 8466489ef5ba ("xhci: Reset Renesas uPD72020x USB controller
> for 32-bit DMA issue")
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> ---
>  drivers/usb/host/pci-quirks.c | 33 ++++++++++++++++++++++++++++++---
>  1 file changed, 30 insertions(+), 3 deletions(-)
> 
> diff --git a/drivers/usb/host/pci-quirks.c b/drivers/usb/host/pci-
> quirks.c
> index 67ad4bb6919a..b05a5c2ec08d 100644
> --- a/drivers/usb/host/pci-quirks.c
> +++ b/drivers/usb/host/pci-quirks.c
> @@ -16,6 +16,7 @@
>  #include <linux/export.h>
>  #include <linux/acpi.h>
>  #include <linux/dmi.h>
> +#include <linux/io-64-nonatomic-lo-hi.h>
>  #include "pci-quirks.h"
>  #include "xhci-ext-caps.h"
>  
> @@ -1277,13 +1278,39 @@ bool usb_xhci_needs_pci_reset(struct pci_dev
> *pdev)
>  	 * at the wrong addresses, as it keeps the top 32bit of some
>  	 * addresses from its previous programming under obscure
>  	 * circumstances.
> -	 * Give it a good wack at probe time. Unfortunately, this
> +	 * Give it a good whack at probe time if any of the ERST
> base
> +	 * addresses contains a 64bit address. Unfortunately, this
>  	 * needs to happen before we've had a chance to discover any
>  	 * quirk, or the system will be in a rather bad state.
>  	 */
>  	if (pdev->vendor == PCI_VENDOR_ID_RENESAS &&
> -	    (pdev->device == 0x0014 || pdev->device == 0x0015))
> -		return true;
> +	    (pdev->device == 0x0014 || pdev->device == 0x0015)) {
> +		int len = pci_resource_len(pdev, 0);
> +		void __iomem *base;
> +		bool sf = false;
> +		int i;
> +
> +		if (len < 0x720 || !mmio_resource_enabled(pdev, 0))
> +			return false;
> +
> +		base = ioremap_nocache(pci_resource_start(pdev, 0),
> len);
> +		if (!base)
> +			return false;
> +
> +		for (i = 0; i < 7; i++) {
> +			u64 val = lo_hi_readq(base + 0x600 + 0x30 +
> 20 * i);
> +			if (!upper_32_bits(val))
> +				continue;
> +
> +			dev_info(&pdev->dev,
> +				 "Found a 64bit address in ERSTBA
> %d\n", i);
> +			sf = true;
> +			break;
> +		}
> +
> +		iounmap(base);
> +		return sf;
> +	}
>  
>  	return false;
>  }
> -- 
> 2.14.2
> 


Hi Marc,

I've applied your patch on top of 4.15.7 and tried it on the server,
unfortunately the problem is still there. Here's the output from dmesg:

[    1.570115] xhci_hcd 0000:03:00.0: Found a 64bit address in ERSTBA 4
[    1.570120] xhci_hcd 0000:03:00.0: Resetting
[    2.668066] xhci_hcd 0000:03:00.0: Refused to change power state,
currently in D3
[    2.668215] xhci_hcd 0000:03:00.0: xHCI Host Controller
[    2.668225] xhci_hcd 0000:03:00.0: new USB bus registered, assigned
bus number 2
[    2.668240] xhci_hcd 0000:03:00.0: Host halt failed, -19
[    2.668242] xhci_hcd 0000:03:00.0: can't setup: -19
[    2.668299] xhci_hcd 0000:03:00.0: USB bus 2 deregistered
[    2.668354] xhci_hcd 0000:03:00.0: init 0000:03:00.0 fail, -19

If you need more informations to find the cause, I will gladly provide
it.

Thanks,

Arne

patch 8466489ef5ba48272ba4fa4ea9f8f403306de4c7 breaks Renesas USB3 controller functionality

[Marc Zyngier]

On 01/03/18 08:01, Bockholdt Arne wrote:
> 
> On Thu, 2018-02-15 at 19:29 +0000, Marc Zyngier wrote:
>> [+ Ard, who helped me chasing the initial issue]
>>
>> On 15/02/18 06:43, Bockholdt Arne wrote:
>>> Hi all,
>>>
>>> on our Intel Atom C2578 server with a SuperMicro A1SAi board and a
>>> Renesas uPD720201 USB 3.0 host controller the controller has
>>> stopped
>>> working since kernel 4.13.x. Before that kernel the dmesg messages
>>> from
>>> XHCI were:
>>>
>>> dmesg-4.12.1-serverv4.log:xhci_hcd 0000:03:00.0: xHCI Host
>>> Controller
>>> dmesg-4.12.1-serverv4.log:xhci_hcd 0000:03:00.0: new USB bus
>>> registered,
>>> assigned bus number 2
>>> dmesg-4.12.1-serverv4.log:xhci_hcd 0000:03:00.0: hcc params
>>> 0x014051cf
>>> hci version 0x100 quirks 0x00000010
>>> dmesg-4.12.1-serverv4.log:usb usb2: Manufacturer: Linux 4.12.1-
>>> serverv4
>>> xhci-hcd
>>> dmesg-4.12.1-serverv4.log:xhci_hcd 0000:03:00.0: xHCI Host
>>> Controller
>>> dmesg-4.12.1-serverv4.log:xhci_hcd 0000:03:00.0: new USB bus
>>> registered,
>>> assigned bus number 3
>>> dmesg-4.12.1-serverv4.log:usb usb3: Manufacturer: Linux 4.12.1-
>>> serverv4
>>> xhci-hcd
>>>
>>> After that the message look like that:
>>>
>>> dmesg-4.13.1-serverv4.log:xhci_hcd 0000:03:00.0: Resetting
>>> dmesg-4.13.1-serverv4.log:xhci_hcd 0000:03:00.0: Refused to change
>>> power
>>> state, currently in D3
>>> dmesg-4.13.1-serverv4.log:xhci_hcd 0000:03:00.0: xHCI Host
>>> Controller
>>> dmesg-4.13.1-serverv4.log:xhci_hcd 0000:03:00.0: new USB bus
>>> registered,
>>> assigned bus number 2
>>> dmesg-4.13.1-serverv4.log:xhci_hcd 0000:03:00.0: Host halt failed,
>>> -19
>>> dmesg-4.13.1-serverv4.log:xhci_hcd 0000:03:00.0: can't setup: -19
>>> dmesg-4.13.1-serverv4.log:xhci_hcd 0000:03:00.0: USB bus 2
>>> deregistered
>>> dmesg-4.13.1-serverv4.log:xhci_hcd 0000:03:00.0: init 0000:03:00.0
>>> fail, -19
>>>
>>> I've tested it with 4.15.3 too, it's still the same. I've narrowed
>>> it
>>> down to the following patch:
>>>
>>> commit 8466489ef5ba48272ba4fa4ea9f8f403306de4c7
>>> Author: Marc Zyngier <marc.zyngier@arm.com <mailto:marc.zyngier@arm
>>> .com>>
>>> Date:   Tue Aug 1 20:11:08 2017 -0500
>>>
>>>     xhci: Reset Renesas uPD72020x USB controller for 32-bit DMA
>>> issue
>>>
>>> Reverting the patch on top of 4.15.3 restores the USB3
>>> functionality on
>>> our server. Please let me know if there is anything I can do to fix
>>> the
>>> problem. Thank you.
>>
>> Hi Arne,
>>
>> This looks pretty bad. I suspect that once reset, the firmware is
>> lost.
>> I'll try to write a patch dumping some information about it.
>>
>> Ard: Do you know if the Cello board has a SPI flash connected to the
>> Renesas chip, from which it would load the firmware?
>>
>> Another possibility is that power management kicks in, and that the
>> endpoint is stuck there. Could also be firmware related, but not
>> only.
>> I'd welcome any idea on the subject, as I cannot reproduce this issue
>> on
>> the HW I have.
>>
>> It we cannot work out what exactly is causing this, we may have to
>> default to not resetting the part and rely on a command-line option
>> to
>> do it... I can't say I'm a fan.
>>
>> Thanks,
>>
>> 	M.
>>
> 
> Hi Marc,
> 
> I've tested it with 4.15.7 and it's still there. Is there anything that
> I can do to help fixing this problem?

Would you mind trying the following patch and let me know if it helps?
It is not exactly pretty, but we can polish it if that solves your issue.

Thanks,

	M.

From 9a253773b289f781b7114e887481939b3021bb30 Mon Sep 17 00:00:00 2001
From: Marc Zyngier <marc.zyngier@arm.com>
Date: Thu, 1 Mar 2018 17:27:42 +0000
Subject: [PATCH] xhci: Only reset uPD72020x if programmed with 64bit DMA
 addresses

The Renesas uPD72020x USB controller misbehaves when switching
from 64bit DMA addresses to 32bit ones, and can only accept a
new programming if hard reset first.

But it also misbehaves (in a different way) if reset for no
good reason. So let's restrict the resetting to the cases where
we detect the 64/32 issue, and leave the device alone in the
other cases.

Fixes: 8466489ef5ba ("xhci: Reset Renesas uPD72020x USB controller for 32-bit DMA issue")
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 drivers/usb/host/pci-quirks.c | 33 ++++++++++++++++++++++++++++++---
 1 file changed, 30 insertions(+), 3 deletions(-)

diff --git a/drivers/usb/host/pci-quirks.c b/drivers/usb/host/pci-quirks.c
index 67ad4bb6919a..b05a5c2ec08d 100644
--- a/drivers/usb/host/pci-quirks.c
+++ b/drivers/usb/host/pci-quirks.c
@@ -16,6 +16,7 @@
 #include <linux/export.h>
 #include <linux/acpi.h>
 #include <linux/dmi.h>
+#include <linux/io-64-nonatomic-lo-hi.h>
 #include "pci-quirks.h"
 #include "xhci-ext-caps.h"
 
@@ -1277,13 +1278,39 @@ bool usb_xhci_needs_pci_reset(struct pci_dev *pdev)
 	 * at the wrong addresses, as it keeps the top 32bit of some
 	 * addresses from its previous programming under obscure
 	 * circumstances.
-	 * Give it a good wack at probe time. Unfortunately, this
+	 * Give it a good whack at probe time if any of the ERST base
+	 * addresses contains a 64bit address. Unfortunately, this
 	 * needs to happen before we've had a chance to discover any
 	 * quirk, or the system will be in a rather bad state.
 	 */
 	if (pdev->vendor == PCI_VENDOR_ID_RENESAS &&
-	    (pdev->device == 0x0014 || pdev->device == 0x0015))
-		return true;
+	    (pdev->device == 0x0014 || pdev->device == 0x0015)) {
+		int len = pci_resource_len(pdev, 0);
+		void __iomem *base;
+		bool sf = false;
+		int i;
+
+		if (len < 0x720 || !mmio_resource_enabled(pdev, 0))
+			return false;
+
+		base = ioremap_nocache(pci_resource_start(pdev, 0), len);
+		if (!base)
+			return false;
+
+		for (i = 0; i < 7; i++) {
+			u64 val = lo_hi_readq(base + 0x600 + 0x30 + 20 * i);
+			if (!upper_32_bits(val))
+				continue;
+
+			dev_info(&pdev->dev,
+				 "Found a 64bit address in ERSTBA %d\n", i);
+			sf = true;
+			break;
+		}
+
+		iounmap(base);
+		return sf;
+	}
 
 	return false;
 }