Re: USB devices on Dell TB16 dock stop working after resuming

[Paul Menzel <pmenzel@molgen.mpg.de>]

[-- Attachment #1: Type: text/plain, Size: 8489 bytes --]

Dear Mathias,


On 2019-11-26 13:44, Mathias Nyman wrote:
> On 26.11.2019 13.33, Paul Menzel wrote:

>> On 2019-11-25 10:20, Mathias Nyman wrote:
>>> On 22.11.2019 13.41, Mika Westerberg wrote:
>>>> On Fri, Nov 22, 2019 at 12:33:44PM +0100, Paul Menzel wrote:
>>
>>>>> On 2019-11-22 12:29, Mika Westerberg wrote:
>>>>>> On Fri, Nov 22, 2019 at 12:05:13PM +0100, Paul Menzel wrote:
>>>>>
>>>>>>> On 2019-11-22 11:50, Mika Westerberg wrote:
>>>>>>>> On Wed, Nov 20, 2019 at 12:50:53PM +0200, Mika Westerberg wrote:
>>>>>>>>> On Tue, Nov 19, 2019 at 05:55:43PM +0100, Paul Menzel wrote:
>>>>>>>
>>>>>>>>>> On 2019-11-04 17:21, Mika Westerberg wrote:
>>>>>>>>>>> On Mon, Nov 04, 2019 at 05:11:10PM +0100, Paul Menzel wrote:
>>>>>>>>>>
>>>>>>>>>>>> On 2019-11-04 16:49, Mario.Limonciello@dell.com wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>>> From: Mika Westerberg <mika.westerberg@linux.intel.com>
>>>>>>>>>>>>>> Sent: Monday, November 4, 2019 9:45 AM
>>>>>>>>>>>>
>>>>>>>>>>>>>> On Mon, Nov 04, 2019 at 04:44:40PM +0200, Mika Westerberg wrote:
>>>>>>>>>>>>>>> On Mon, Nov 04, 2019 at 04:25:03PM +0200, Mika Westerberg wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Mon, Nov 04, 2019 at 02:13:13PM +0100, Paul Menzel wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On the Dell XPS 13 9380 with Debian Sid/unstable with Linux 5.3.7
>>>>>>>>>>>>>>>>> suspending the system, and resuming with Dell’s Thunderbolt TB16
>>>>>>>>>>>>>>>>> dock connected, the USB input devices, keyboard and mouse,
>>>>>>>>>>>>>>>>> connected to the TB16 stop working. They work for a few seconds
>>>>>>>>>>>>>>>>> (mouse cursor can be moved), but then stop working. The laptop
>>>>>>>>>>>>>>>>> keyboard and touchpad still works fine. All firmware is up-to-date
>>>>>>>>>>>>>>>>> according to `fwupdmgr`.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> What are the exact steps to reproduce? Just "echo mem >
>>>>>>>>>>>>>>>> /sys/power/state" and then resume by pressing power button?
>>>>>>>>>>>>
>>>>>>>>>>>> GNOME Shell 3.34.1+git20191024-1 is used, and the user just closes the
>>>>>>>>>>>> display. So more than `echo mem > /sys/power/state` is done. What
>>>>>>>>>>>> distribution do you use?
>>>>>>>>>>>
>>>>>>>>>>> I have buildroot based "distro" so there is no UI running.
>>>>>>>>>>
>>>>>>>>>> Hmm, this is quite different from the “normal” use-case of the these devices.
>>>>>>>>>> That way you won’t hit the bugs of the normal users. ;-)
>>>>>>>>>
>>>>>>>>> Well, I can install some distro to that thing also :) I suppose Debian
>>>>>>>>> 10.2 does have this issue, no?
>>>>>>>>>
>>>>>>>>>>>>>>> I tried v5.4-rc6 on my 9380 with TB16 dock connected and did a couple of
>>>>>>>>>>>>>>> suspend/resume cycles (to s2idle) but I don't see any issues.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I may have older/different firmware than you, though.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Upgraded BIOS to 1.8.0 and TBT NVM to v44 but still can't reproduce this
>>>>>>>>>>>>>> on my system :/
>>>>>>>>>>>>
>>>>>>>>>>>> The user reported the issue with the previous firmwares 1.x and TBT NVM v40.
>>>>>>>>>>>> Updating to the recent version (I got the logs with) did not fix the issue.
>>>>>>>>>>>
>>>>>>>>>>> I also tried v40 (that was originally on that system) but I was not able
>>>>>>>>>>> to reproduce it.
>>>>>>>>>>>
>>>>>>>>>>> Do you know if the user changed any BIOS settings?
>>>>>>>>>>
>>>>>>>>>> We had to disable the Thunderbolt security settings as otherwise the USB
>>>>>>>>>> devices wouldn’t work at cold boot either.
>>>>>>>>>
>>>>>>>>> That does not sound right at all. There is the preboot ACL that allows
>>>>>>>>> you to use TBT dock aready on boot. Bolt takes care of this.
>>>>>>>>>
>>>>>>>>> Are you talking about USB devices connected to the TB16 dock?
>>>>>>>>>
>>>>>>>>> Also are you connecting the TB16 dock to the Thunderbolt ports (left
>>>>>>>>> side of the system marked with small lightning logo) or to the normal
>>>>>>>>> Type-C ports (right side)?
>>>>>>>>>
>>>>>>>>>> So, I built Linux 5.4-rc8 (`make bindeb-pkg -j8`), but unfortunately the
>>>>>>>>>> error is still there. Sometimes, re-plugging the dock helped, and sometimes
>>>>>>>>>> it did not.
>>>>>>>>>>
>>>>>>>>>> Please find the logs attached. The strange thing is, the Linux kernel detects
>>>>>>>>>> the devices and I do not see any disconnect events. But, `lsusb` does not list
>>>>>>>>>> the keyboard and the mouse. Is that expected.
>>>>>>>>>
>>>>>>>>> I'm bit confused. Can you describe the exact steps what you do (so I can
>>>>>>>>> replicate them).
>>>>>>>>
>>>>>>>> I managed to reproduce following scenario.
>>>>>>>>
>>>>>>>> 1. Boot the system up to UI
>>>>>>>> 2. Connect TB16 dock (and see that it gets authorized by bolt)
>>>>>>>> 3. Connect keyboard and mouse to the TB16 dock
>>>>>>>> 4. Both mouse and keyboard are functional
>>>>>>>> 5. Enter s2idle by closing laptop lid
>>>>>>>> 6. Exit s2idle by opening the laptop lid
>>>>>>>> 7. After ~10 seconds or so the mouse or keyboard or both do not work
>>>>>>>>      anymore. They do not respond but they are still "present".
>>>>>>>>
>>>>>>>> The above does not happen always but from time to time.
>>>>>>>>
>>>>>>>> Is this the scenario you see as well?
>>>>>>>
>>>>>>> Yes, it is. Though I’d say it’s only five seconds or so.
>>>>>>>
>>>>>>>> This is on Ubuntu 19.10 with the 5.3 stock kernel.
>>>>>>>
>>>>>>> “stock” in upstream’s or Ubuntu’s?
>>>>>>
>>>>>> It is Ubuntu's.
>>>>>>
>>>>>>>> I can get them work again by unplugging them and plugging back (leaving
>>>>>>>> the TBT16 dock connected). Also if you run lspci when the problem
>>>>>>>> occurs it still shows the dock so PCIe link stays up.
>>>>>>>
>>>>>>> Re-connecting the USB devices does not help here, but I still suspect it’s
>>>>>>> the same issue.
>>>>>>
>>>>>> Yeah, sounds like so. Did you try to connect the device (mouse,
>>>>>> keyboard) to another USB port?
>>>>>
>>>>> I do not think I did, but I can’t remember. Next week would be the next chance
>>>>> to test this.
>>>>>
>>>>>>> Yesterday, I had my hand on a Dell XPS 13 7390 (10th Intel generation) and
>>>>>>> tried it with the shipped Ubuntu 18.04 LTS. There, the problem was not
>>>>>>> always reproducible, but it still happened. Sometimes, only one of the USB
>>>>>>> device (either keyboard or mouse) stopped working.
>>>>>>
>>>>>> I suppose this is also with the TB16 dock connected, correct?
>>>>>
>>>>> Correct.
>>>>>
>>>>> Can I ask again, how the USB devices connected to the dock can be listed on
>>>>> the command line? lsusb needs to be adapted for that or is a different
>>>>> mechanism needed?
>>>>
>>>> The TB16 dock has ASMEDIA xHCI controller, which is PCIe device so you
>>>> can see it by running lsusb and looking at the devices under that
>>>> controller. I think maybe 'lsusb -t' is helpful.
>>>>
>>>> The xHCI controller itself you can see by running lspci.
>>>
>>> I got traces from the ASMedia xHC controller in the TB16 dock.
>>
>> Nice. Thank you for looking into that. How can these traces be captured?
> 
> The Linux tracepoints added to the xhci driver can be enabled by:
> 
> mount -t debugfs none /sys/kernel/debug
> echo 81920 > /sys/kernel/debug/tracing/buffer_size_kb
> echo 1 > /sys/kernel/debug/tracing/events/xhci-hcd/enable
> < Trigger the issue >
> 
> Copy traces found in /sys/kernel/debug/tracing/trace
> 
> Trace file grows fast.

>>> There are issues with split transactions between the ASMedia host and the 7 port
>>> High speed hub built in to the dock.
>>>
>>> host reports a split transaction error for mouse or keyboard full-speed/low-speed
>>> interrupt transactions. Endpoint doesn't recover after resetting it.
>>>
>>> Split transaction allows full- and low-speed devices to be attached to high-speed
>>> hubs, and are used only between the host and the HS hub. A transaction translator (TT)
>>> in the HS hub will translate the high-speed split transactions on its upstream port to
>>> low/full speed transactions on the downstream port.
>>>
>>> I'll see if there are any xHC parameters driver is setting that trigger these
>>> split transaction errors to trigger more easy.
>>
>> I always wonder how Microsoft Windows driver do it.
>>
>> Mario, should I contact the Dell support regarding this issue?
Sorry for bothering, but were you able to find some workaround for this issue?


Kind regards,

Paul


[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 5174 bytes --]