Re: USB device disconnects on resume

[Martin Kepplinger <martin.kepplinger@puri.sm>]

[-- Attachment #1: Type: text/plain, Size: 9021 bytes --]

Alan, thanks so much for asking these questions and thinking this
through! It helps.

Am Donnerstag, dem 21.04.2022 um 10:24 -0400 schrieb Alan Stern:
> On Thu, Apr 21, 2022 at 12:38:56PM +0200, Martin Kepplinger wrote:
> > Am Mittwoch, dem 20.04.2022 um 10:40 -0400 schrieb Alan Stern:
> > > On Wed, Apr 20, 2022 at 12:37:36PM +0200, Martin Kepplinger
> > > wrote:
> > > > Resetting itself doesn't usually fail in the sense that a
> > > > device
> > > > would
> > > > not work anymore after resetting. The problem is that the
> > > > resets
> > > > happen
> > > > in the first place. 90+% of runtime-resumes are fine - auto-
> > > > and
> > > > wakeup-resume. Resetting is a major problem though, imagine a
> > > > modem
> > > > device being re-enumerated during a phone call or "realtime"
> > > > data
> > > > connection. I see that a lot.
> > > 
> > > Okay, I see.
> 
> By the way, I assume that while resetting the modem is a major
> problem 
> for your potential use cases, having it crash with no hope of
> recovery 
> is even worse.  But maybe I'm wrong...

Sure that's true (and I have other usb-related problems too that result
in the xhci HC dying even, when I see "xhci-hcd.4.auto: Port resume
timed out, port 1-1: 0xfe3", so the Hub where the modem is connected)
but these resets are a seperate issue I want to tackle now and see
whether it help with other usb-related problems, so let's focus on this
disconnect/reset situation. So let's not get confused :)

> 
> > > > Let me record what hub.c says when leading up to the reset of
> > > > 1-1.2
> > > > (the modem), with logs of a normal runtime resume/suspend cycle
> > > > included before that, as reference:
> > > > 
> > > > 1650447001.174798 pureos kernel: usb 1-1: usb auto-resume
> > > > 1650447001.242810 pureos kernel: usb 1-1: Waited 0ms for
> > > > CONNECT
> > > > 1650447001.247853 pureos kernel: usb 1-1: finish resume
> > > > 1650447001.249697 pureos kernel: hub 1-1:1.0: hub_resume
> > > > 1650447001.251409 pureos kernel: usb 1-1-port1: status 0507
> > > > change
> > > > 0000
> > > > 1650447001.251624 pureos kernel: usb 1-1-port2: status 0507
> > > > change
> > > > 0000
> > > > 1650447001.251793 pureos kernel: hub 1-1:1.0: state 7 ports 3
> > > > chg
> > > > 0000
> > > > evt 0000
> > > > 1650447001.253052 pureos kernel: usb 1-1.2: usb auto-resume
> > > 
> > > What is the cause of this runtime resume?  According to the port
> > > status 
> > > above, the 1-1.2 device did not send a wakeup request.
> > 
> > How would I find out? Recording via usbmon is next on my todo list.
> 
> That's a good thing to try in any case.
> 
> I asked because it seemed possible that you had manually caused the 
> resume somehow.

I use the modem "a lot" in order to wake it up often, that's all.

> 
> > > > 1650447001.318845 pureos kernel: usb 1-1.2: Waited 0ms for
> > > > CONNECT
> > > > 1650447001.324925 pureos kernel: usb 1-1.2: finish resume
> > > > 1650447003.831095 pureos kernel: usb 1-1.2: usb auto-suspend,
> > > > wakeup 1
> > > > 1650447003.854701 pureos kernel: hub 1-1:1.0: hub_suspend
> > > > 1650447003.874773 pureos kernel: usb 1-1: usb auto-suspend,
> > > > wakeup
> > > > 1
> > > > 1650447003.922054 pureos kernel: usb 1-1: usb wakeup-resume
> > > 
> > > This wakeup occurred only 48 ms after the hub was runtime
> > > suspended. 
> > > But here at least the cause is evident: The hub sent a wakeup
> > > request
> > > because its child (the 1-1.2 modem) disconnected.
> > 
> > fwiw, that wakeup-resume *always* comes about 50 ms after the last
> > runtime suspend.
> 
> Maybe the modem's firmware has some 50-ms timeout that expires and
> then 
> causes the crash.

I now recorded usbmon (1u) traffic of a similar sequence I sent here,
with a few usbmon-timestamps in the logs:

https://source.puri.sm/Librem5/linux/-/issues/303#note_197913

the usbmon log textfile is attached in that comment link and in this
email.

> 
> > > > 1650447003.942066 pureos kernel: usb 1-1: Waited 0ms for
> > > > CONNECT
> > > > 1650447003.945755 pureos kernel: usb 1-1: finish resume
> > > > 1650447003.947589 pureos kernel: hub 1-1:1.0: hub_resume
> > > > 1650447003.949226 pureos kernel: usb 1-1-port1: status 0507
> > > > change
> > > > 0000
> > > > 1650447003.949430 pureos kernel: usb 1-1-port2: status 0101
> > > > change
> > > > 0005
> > > > 1650447004.058779 pureos kernel: hub 1-1:1.0: state 7 ports 3
> > > > chg
> > > > 0004
> > > > evt 0000
> > > > 1650447004.074089 pureos kernel: usb 1-1.2: usb wakeup-resume
> > > > 1650447004.094056 pureos kernel: usb 1-1.2: Waited 0ms for
> > > > CONNECT
> > > > 1650447004.097255 pureos kernel: usb 1-1.2: finish reset-resume
> > > > 1650447004.182333 pureos kernel: usb 1-1.2: reset high-speed
> > > > USB
> > > > device
> > > > number 5 using xhci-hcd
> > > > 1650447004.314425 pureos kernel: usb 1-1-port2: resume, status
> > > > 0
> > > > 1650447004.317628 pureos kernel: usb 1-1-port2: status 0101,
> > > > change
> > > > 0004, 12 Mb/s
> > > > 1650447004.318673 pureos kernel: usb 1-1.2: USB disconnect,
> > > > device
> > > > number 5
> > > > 1650447004.323374 pureos kernel: usb 1-1.2: unregistering
> > > > device
> > > 
> > > And it looks like in this case, the reset-resume failed.
> > 
> > Well, at least reset_resume has been set, which I want to avoid.
> 
> Do you mean you would prefer to have the modem disconnect permanently
> (or at least until the next reboot)?

no. I mean I'd prefer the kernel keep the device usable when such a
disconnect happens - after a short pause until the modem is up again -
and not re-enumerate.

An audio stream is not being interrupted by such a usb disconnect and
if I'd keep the ttyUSB device for userspace, I'd keep an opened ttyUSB
controllable - even if possibly with a large latency spike. That would
stay a bad workaround of course. Do you know what I'm thinking about?

> 
> > > > So before resetting, the hub reads
> > > > "usb 1-1-port2: status 0101 change 0005" instead of normally
> > > > "usb 1-1-port2: status 0507 change 0000"
> > > > 
> > > > but I don't know why. That portstatus/portchange doesn't change
> > > > over
> > > > time when I just keep reading portstatus/portchange in
> > > > hub_activate()
> > > > in a loop.
> > > 
> > > You mean that if the port status and change values are originally
> > > 0101 and 0005 in hub_activate(), they remain equal to those
> > > values? 
> > > And 
> > > similarly if they are originally 0507 and 0000?
> > > 
> > > That is to be expected.  Nothing happens to make those values
> > > change 
> > > until hub_activate() sends some commands to the hub.
> > 
> > I see.
> > 
> > Of course this doesn't make much sense, but just so you know: if I
> > just
> > don't let hub_activate() set udev->reset_resume to 1, then
> > check_port_resume_type() will do so, and thus again
> > finish_port_resume() will reset the device by calling
> > usb_reset_and_verify_device().
> 
> Yeah, there's some redundancy in there.  The core really tries pretty
> hard to make sure that devices don't get lost just because they have 
> trouble handling a suspend + resume.
> 
> > > What it means is that the modem was electronically disconnected
> > > from
> > > the 
> > > USB bus.  In theory this could be the result of a mixup in the
> > > resume
> > > signals, but it's more likely that the modem did this
> > > deliberately 
> > > because its firmware crashed.  (Why it should crash while it is 
> > > suspended is a good question, though...)
> > 
> > ok. assuming such a firmware bug, if I set a new quirk for the
> > device,
> > do you think I can work around that (in hub.c?) in a way that
> > userspace
> > doesn't really notice?
> 
> One thing you can do pretty easily, without changing the kernel, is 
> prevent the modem from going into runtime suspend in the first
> place.  
> For example, if you have a program like powertop overseeing your
> runtime 
> power management, you could tell it not to let the modem suspend.  Or
> if 
> you want to set it up by hand, the command is:
> 
>         echo on >/sys/bus/usb/devices/.../power/control
> 
> (where "..." is the appropriate path for the modem device, such as 
> "1-1.2").  You can even write a udev script to do this automatically 
> whenever the modem is detected.

I know that this "solves" my problems but that also prevents the usb2
hub (1-1) from suspending and that's not a practical solution. The hub
uses a *lot* of power.

> 
> > In theory, if I know this behaviour in advance, I think I should be
> > able to somehow wait until the device is ready again instead of
> > resetting.
> 
> What if the modem never becomes ready again (or not until you
> reboot)?  
> I think that sort of behavior is not at all unlikely.  You can test
> this 
> by disabling the code in finish_port_resume() that does reset-
> resumes.

if I just do that, I get "gone after usb resume? status -5" and thus a
reset is triggered after all (also, when I do msleep(500) instead of
reset_and_verify_device() there).

> 
> Alan Stern


[-- Attachment #2: 1_usbmon.txt --]
[-- Type: text/plain, Size: 17988 bytes --]

ffff0000bf334a00 3411803859 C Bi:1:006:8 -104 0
ffff0000440c2a00 3411804162 C Bi:1:006:8 -104 0
ffff0000440c2500 3411804170 C Bi:1:006:8 -104 0
ffff0000440c2d00 3411804175 C Bi:1:006:8 -104 0
ffff0000440c2900 3411804183 C Bi:1:006:8 -104 0
ffff0000440c2c00 3411804187 C Bi:1:006:8 -104 0
ffff0000440c2f00 3411804193 C Bi:1:006:8 -104 0
ffff00002472de00 3411804201 C Bi:1:006:8 -104 0
ffff0000440c2400 3411804538 C Bi:1:006:8 -104 0
ffff0000440c2600 3411804547 C Bi:1:006:8 -104 0
ffff0000440c2e00 3411804808 C Bi:1:006:8 -104 0
ffff0000440c2b00 3411804817 C Bi:1:006:8 -104 0
ffff0000bf271800 3411804825 C Bi:1:006:8 -104 0
ffff0000440c2200 3411804831 C Bi:1:006:8 -104 0
ffff0000009e9500 3411804837 C Bi:1:006:8 -104 0
ffff0000bf14ba00 3411804845 C Bi:1:006:8 -104 0
ffff0000440c2700 3411804852 C Bi:1:006:8 -104 0
ffff000055d1b600 3411805205 C Bi:1:006:8 -104 0
ffff000055d1b100 3411805212 C Bi:1:006:8 -104 0
ffff000055d1b400 3411805221 C Bi:1:006:8 -104 0
ffff0000520e5a00 3411805227 C Bi:1:006:8 -104 0
ffff0000520e5800 3411805235 C Bi:1:006:8 -104 0
ffff0000520e5700 3411805241 C Bi:1:006:8 -104 0
ffff0000520e5c00 3411805245 C Bi:1:006:8 -104 0
ffff000055d1b800 3411805256 C Bi:1:006:8 -104 0
ffff0000520e5d00 3411805607 C Bi:1:006:8 -104 0
ffff0000520e5500 3411805615 C Bi:1:006:8 -104 0
ffff000047036a00 3411805623 C Bi:1:006:8 -104 0
ffff0000290ee800 3411805630 C Bi:1:006:8 -104 0
ffff00002cc74600 3411805641 C Bi:1:006:8 -104 0
ffff000026ddb900 3411805651 C Bi:1:006:8 -104 0
ffff000000997000 3411805657 C Bi:1:006:8 -104 0
ffff0000520e5600 3411805663 C Bi:1:006:8 -104 0
ffff00000d5fcf00 3411805968 C Bi:1:006:8 -104 0
ffff0000bbf44800 3411805976 C Bi:1:006:8 -104 0
ffff00001fd2d200 3411805986 C Bi:1:006:8 -104 0
ffff00001fd2d300 3411805994 C Bi:1:006:8 -104 0
ffff00000d5fcd00 3411806001 C Bi:1:006:8 -104 0
ffff0000043d0700 3411806369 C Bi:1:006:8 -104 0
ffff0000043d0300 3411806378 C Bi:1:006:8 -104 0
ffff0000043d0f00 3411806392 C Bi:1:006:8 -104 0
ffff0000043d0500 3411806398 C Bi:1:006:8 -104 0
ffff000052e0b800 3411806404 C Bi:1:006:8 -104 0
ffff000052e0b500 3411806412 C Bi:1:006:8 -104 0
ffff000046e8a200 3411806419 C Bi:1:006:8 -104 0
ffff000052e0bb00 3411806709 C Bi:1:006:8 -104 0
ffff000052e0bc00 3411806726 C Bi:1:006:8 -104 0
ffff000052e0b000 3411806734 C Bi:1:006:8 -104 0
ffff000052e0be00 3411806744 C Bi:1:006:8 -104 0
ffff000052e0b200 3411806752 C Bi:1:006:8 -104 0
ffff000046f03c00 3411807093 C Bi:1:006:8 -104 0
ffff000055bfe900 3411807102 C Bi:1:006:8 -104 0
ffff0000bc7ef300 3411807110 C Bi:1:006:8 -104 0
ffff0000bc7ef200 3411807150 C Bi:1:006:8 -104 0
ffff000046f53700 3411807162 C Bi:1:006:8 -104 0
ffff000047399100 3411807167 C Bi:1:006:8 -104 0
ffff000052e0bd00 3411807178 C Bi:1:006:8 -104 0
ffff000047399000 3411807439 C Bi:1:006:8 -104 0
ffff00002472d300 3411807449 C Bi:1:006:8 -104 0
ffff000047399200 3411807455 C Bi:1:006:8 -104 0
ffff0000bbe51d00 3411807962 C Ii:1:006:7 -2:16 0
ffff00004834fc00 3411810266 C Bi:1:006:4 -2 0
ffff0000bc5e0200 3411810461 C Bi:1:006:4 -2 0
ffff0000bc5e0300 3411811398 C Bi:1:006:4 -2 0
ffff0000bc5e0400 3411811595 C Bi:1:006:4 -2 0
ffff00004834f500 3411811793 C Ii:1:006:3 -2:16 0
ffff000031511f00 3411812759 C Bi:1:006:2 -2 0
ffff000031511200 3411813650 C Bi:1:006:2 -2 0
ffff000031511b00 3411814522 C Bi:1:006:2 -2 0
ffff000031511d00 3411814754 C Bi:1:006:2 -2 0
ffff00001fd2df00 3411815721 C Bi:1:006:1 -2 0
ffff00001fd2d000 3411815895 C Bi:1:006:1 -2 0
ffff00000e904100 3411816824 C Bi:1:006:1 -2 0
ffff000014838000 3411817704 C Bi:1:006:1 -2 0
ffff000046f53700 3411817862 S Co:1:006:0 s 00 03 0001 0000 0000 0
ffff000046f53700 3411818894 C Co:1:006:0 0 0
ffff000046f53700 3411818959 S Co:1:002:0 s 23 03 0002 0002 0000 0
ffff000046f53700 3411819071 C Co:1:002:0 0 0
ffff0000befcda00 3411841641 C Ii:1:002:1 -2:2048 0
ffff000046f53700 3411841759 S Co:1:002:0 s 00 03 0001 0000 0000 0
ffff000046f53700 3411841840 C Co:1:002:0 0 0
ffff000046f53700 3411841865 S Co:1:001:0 s 23 03 0002 0001 0000 0
ffff000046f53700 3411864838 C Co:1:001:0 0 0
ffff0000bf035e00 3411888967 C Ii:1:001:1 -2:2048 0
ffff000046f53700 3412240229 S Ci:1:001:0 s a3 00 0000 0001 0004 4 <
ffff000046f53700 3412240507 C Ci:1:001:0 0 4 = 07050000
ffff0000bf035e00 3412240641 S Ii:1:001:1 -115:2048 4 <
ffff0000bf035e00 3412280877 C Ii:1:001:1 0:2048 1 = 02
ffff0000bf035e00 3412281031 S Ii:1:001:1 -115:2048 4 <
ffff000046f53700 3412281339 S Ci:1:001:0 s a3 00 0000 0001 0004 4 <
ffff000046f53700 3412281659 C Ci:1:001:0 0 4 = 03050400
ffff000046f53700 3412281701 S Co:1:001:0 s 23 01 0012 0001 0000 0
ffff000046f53700 3412281729 C Co:1:001:0 0 0
ffff000046f53700 3412300931 S Ci:1:001:0 s a3 00 0000 0001 0004 4 <
ffff000046f53700 3412301029 C Ci:1:001:0 0 4 = 03050000
ffff000046f53700 3412320851 S Ci:1:002:0 s 80 00 0000 0000 0002 2 <
ffff000046f53700 3412321210 C Ci:1:002:0 0 2 = 0300
ffff000046f53700 3412321239 S Co:1:002:0 s 00 01 0001 0000 0000 0
ffff000046f53700 3412321312 C Co:1:002:0 0 0
ffff000046f53700 3412321397 S Ci:1:002:0 s a3 00 0000 0001 0004 4 <
ffff000046f53700 3412321476 C Ci:1:002:0 0 4 = 07050000
ffff000046f53700 3412321536 S Ci:1:002:0 s a3 00 0000 0002 0004 4 <
ffff000046f53700 3412321615 C Ci:1:002:0 0 4 = 03050400
ffff000046f53700 3412321657 S Ci:1:002:0 s a3 00 0000 0003 0004 4 <
ffff000046f53700 3412321731 C Ci:1:002:0 0 4 = 00010000
ffff0000befcda00 3412321756 S Ii:1:002:1 -115:2048 1 <
ffff000046f53700 3412322018 S Ci:1:002:0 s a3 00 0000 0002 0004 4 <
ffff000046f53700 3412322091 C Ci:1:002:0 0 4 = 03050400
ffff000046f53700 3412322115 S Co:1:002:0 s 23 01 0012 0002 0000 0
ffff000046f53700 3412322171 C Co:1:002:0 0 0
ffff000046f53700 3412341019 S Ci:1:002:0 s a3 00 0000 0002 0004 4 <
ffff000046f53700 3412341303 C Ci:1:002:0 0 4 = 03050000
ffff000046f53700 3412360988 S Ci:1:006:0 s 80 00 0000 0000 0002 2 <
ffff000046f53700 3412361457 C Ci:1:006:0 0 2 = 0200
ffff000046f53700 3412361501 S Co:1:006:0 s 00 01 0001 0000 0000 0
ffff000046f53700 3412361679 C Co:1:006:0 0 0
ffff00001fd2df00 3412361790 S Bi:1:006:1 -115 4096 <
ffff00001fd2d000 3412361848 S Bi:1:006:1 -115 4096 <
ffff00000e904100 3412361873 S Bi:1:006:1 -115 4096 <
ffff000014838000 3412361897 S Bi:1:006:1 -115 4096 <
ffff000031511f00 3412361962 S Bi:1:006:2 -115 4096 <
ffff000031511200 3412361997 S Bi:1:006:2 -115 4096 <
ffff000031511b00 3412362024 S Bi:1:006:2 -115 4096 <
ffff000031511d00 3412362051 S Bi:1:006:2 -115 4096 <
ffff00004834f500 3412362117 S Ii:1:006:3 -115:16 64 <
ffff00004834fc00 3412362157 S Bi:1:006:4 -115 4096 <
ffff0000bc5e0200 3412362193 S Bi:1:006:4 -115 4096 <
ffff0000bc5e0300 3412362216 S Bi:1:006:4 -115 4096 <
ffff0000bc5e0400 3412362239 S Bi:1:006:4 -115 4096 <
ffff0000bbe51d00 3412362329 S Ii:1:006:7 -115:16 64 <
ffff000046f53700 3412362447 S Bi:1:006:8 -115 1500 <
ffff000046f53100 3412362504 S Bi:1:006:8 -115 1500 <
ffff000046f53700 3412362575 C Bi:1:006:8 0 588 = 4500024c 5b9e4000 34068f24 55d0612a 0a91985e 2328e8d6 918eb931 83bdb959
ffff000046f53700 3412362634 S Bi:1:006:8 -115 1500 <
ffff000046f53c00 3412362754 S Bi:1:006:8 -115 1500 <
ffff000046f53d00 3412362780 S Bi:1:006:8 -115 1500 <
ffff000046f53a00 3412362806 S Bi:1:006:8 -115 1500 <
ffff000047399200 3412362833 S Bi:1:006:8 -115 1500 <
ffff000047399000 3412362856 S Bi:1:006:8 -115 1500 <
ffff000047399100 3412362883 S Bi:1:006:8 -115 1500 <
ffff0000bc7ef200 3412362911 S Bi:1:006:8 -115 1500 <
ffff0000bc7ef300 3412362941 S Bi:1:006:8 -115 1500 <
ffff000055bfe900 3412362973 S Bi:1:006:8 -115 1500 <
ffff000046e8a200 3412363008 S Bi:1:006:8 -115 1500 <
ffff000052e0bd00 3412363775 S Bo:1:006:5 -115 52 = 45000034 54514000 40068c89 0a91985e 55d0612a e8d62328 83bdb959 918ebb49
ffff000052e0be00 3412364173 S Bi:1:006:8 -115 1500 <
ffff000052e0b200 3412364194 S Bi:1:006:8 -115 1500 <
ffff000052e0b000 3412364231 S Bi:1:006:8 -115 1500 <
ffff000052e0bc00 3412364256 S Bi:1:006:8 -115 1500 <
ffff000052e0bb00 3412364282 S Bi:1:006:8 -115 1500 <
ffff000052e0b500 3412364308 S Bi:1:006:8 -115 1500 <
ffff000052e0b800 3412364335 S Bi:1:006:8 -115 1500 <
ffff00007a7f2d00 3412364390 S Bi:1:006:8 -115 1500 <
ffff00007a7f2100 3412364419 S Bi:1:006:8 -115 1500 <
ffff00007a7f2300 3412364448 S Bi:1:006:8 -115 1500 <
ffff000052e0bd00 3412364477 C Bo:1:006:5 0 52 >
ffff00007a7f2400 3412364623 S Bi:1:006:8 -115 1500 <
ffff00007a7f2c00 3412364654 S Bi:1:006:8 -115 1500 <
ffff00007a7f2a00 3412364711 S Bi:1:006:8 -115 1500 <
ffff00007a7f2200 3412364740 S Bi:1:006:8 -115 1500 <
ffff00007a7f2000 3412364771 S Bi:1:006:8 -115 1500 <
ffff00007a7f2e00 3412364798 S Bi:1:006:8 -115 1500 <
ffff00007a7f2f00 3412364823 S Bi:1:006:8 -115 1500 <
ffff00007a7f2900 3412364911 S Bi:1:006:8 -115 1500 <
ffff000052e0bd00 3412364940 S Bi:1:006:8 -115 1500 <
ffff0000483bb700 3412364980 S Bi:1:006:8 -115 1500 <
ffff0000287f4e00 3412365020 S Bi:1:006:8 -115 1500 <
ffff0000287f4700 3412365056 S Bi:1:006:8 -115 1500 <
ffff0000bc79ed00 3412365090 S Bi:1:006:8 -115 1500 <
ffff00000ecb3d00 3412365117 S Bi:1:006:8 -115 1500 <
ffff00000ecb3600 3412365143 S Bi:1:006:8 -115 1500 <
ffff00000ecb3800 3412365169 S Bi:1:006:8 -115 1500 <
ffff00000e9dda00 3412365210 S Bi:1:006:8 -115 1500 <
ffff000052667c00 3412365238 S Bi:1:006:8 -115 1500 <
ffff000052495e00 3412365273 S Bi:1:006:8 -115 1500 <
ffff000052495800 3412365300 S Bi:1:006:8 -115 1500 <
ffff000052495700 3412365333 S Bi:1:006:8 -115 1500 <
ffff000052495300 3412365361 S Bi:1:006:8 -115 1500 <
ffff000052495400 3412365388 S Bi:1:006:8 -115 1500 <
ffff0000bc889f00 3412365419 S Bi:1:006:8 -115 1500 <
ffff00006c53e800 3412365462 S Bi:1:006:8 -115 1500 <
ffff0000bbb00a00 3412365518 S Bi:1:006:8 -115 1500 <
ffff0000bee65b00 3412365553 S Bi:1:006:8 -115 1500 <
ffff000044107b00 3412365587 S Bi:1:006:8 -115 1500 <
ffff000030dff800 3412365620 S Bi:1:006:8 -115 1500 <
ffff000022a72400 3412365659 S Bi:1:006:8 -115 1500 <
ffff000046f6a200 3412365702 S Bi:1:006:8 -115 1500 <
ffff000046f6a500 3412365744 S Bi:1:006:8 -115 1500 <
ffff000046f6a300 3412365773 S Bi:1:006:8 -115 1500 <
ffff000046f6a400 3412365802 S Bi:1:006:8 -115 1500 <
ffff000046f6ab00 3412365829 S Bi:1:006:8 -115 1500 <
ffff000046f6ad00 3412365863 S Bi:1:006:8 -115 1500 <
ffff00002472d300 3412365900 S Bi:1:006:8 -115 1500 <
ffff00002472de00 3412365926 S Bi:1:006:8 -115 1500 <
ffff000046f53700 3414449727 C Bi:1:006:8 -104 0
ffff000046f53c00 3414449896 C Bi:1:006:8 -104 0
ffff000046f53d00 3414449902 C Bi:1:006:8 -104 0
ffff000046f53a00 3414449909 C Bi:1:006:8 -104 0
ffff000047399200 3414449923 C Bi:1:006:8 -104 0
ffff000047399000 3414449927 C Bi:1:006:8 -104 0
ffff000047399100 3414449934 C Bi:1:006:8 -104 0
ffff000046f53100 3414449939 C Bi:1:006:8 -104 0
ffff0000bc7ef300 3414450253 C Bi:1:006:8 -104 0
ffff000055bfe900 3414450261 C Bi:1:006:8 -104 0
ffff000046e8a200 3414450267 C Bi:1:006:8 -104 0
ffff0000bc7ef200 3414450273 C Bi:1:006:8 -104 0
ffff000052e0b200 3414450499 C Bi:1:006:8 -104 0
ffff000052e0b000 3414450511 C Bi:1:006:8 -104 0
ffff000052e0bc00 3414450524 C Bi:1:006:8 -104 0
ffff000052e0bb00 3414450530 C Bi:1:006:8 -104 0
ffff000052e0be00 3414450538 C Bi:1:006:8 -104 0
ffff000052e0b800 3414450745 C Bi:1:006:8 -104 0
ffff00007a7f2d00 3414450755 C Bi:1:006:8 -104 0
ffff000052e0b500 3414450761 C Bi:1:006:8 -104 0
ffff00007a7f2300 3414451022 C Bi:1:006:8 -104 0
ffff00007a7f2400 3414451027 C Bi:1:006:8 -104 0
ffff00007a7f2c00 3414451033 C Bi:1:006:8 -104 0
ffff00007a7f2a00 3414451036 C Bi:1:006:8 -104 0
ffff00007a7f2200 3414451050 C Bi:1:006:8 -104 0
ffff00007a7f2000 3414451058 C Bi:1:006:8 -104 0
ffff00007a7f2100 3414451064 C Bi:1:006:8 -104 0
ffff00007a7f2f00 3414451262 C Bi:1:006:8 -104 0
ffff00007a7f2e00 3414451269 C Bi:1:006:8 -104 0
ffff000052e0bd00 3414451519 C Bi:1:006:8 -104 0
ffff0000483bb700 3414451526 C Bi:1:006:8 -104 0
ffff0000287f4e00 3414451531 C Bi:1:006:8 -104 0
ffff0000287f4700 3414451537 C Bi:1:006:8 -104 0
ffff0000bc79ed00 3414451545 C Bi:1:006:8 -104 0
ffff00007a7f2900 3414451552 C Bi:1:006:8 -104 0
ffff00000ecb3600 3414451742 C Bi:1:006:8 -104 0
ffff00000ecb3d00 3414451753 C Bi:1:006:8 -104 0
ffff00000e9dda00 3414452052 C Bi:1:006:8 -104 0
ffff000052667c00 3414452065 C Bi:1:006:8 -104 0
ffff000052495e00 3414452072 C Bi:1:006:8 -104 0
ffff000052495800 3414452078 C Bi:1:006:8 -104 0
ffff000052495700 3414452086 C Bi:1:006:8 -104 0
ffff000052495300 3414452093 C Bi:1:006:8 -104 0
ffff000052495400 3414452100 C Bi:1:006:8 -104 0
ffff00000ecb3800 3414452105 C Bi:1:006:8 -104 0
ffff00006c53e800 3414452402 C Bi:1:006:8 -104 0
ffff0000bc889f00 3414452418 C Bi:1:006:8 -104 0
ffff0000bee65b00 3414452631 C Bi:1:006:8 -104 0
ffff000044107b00 3414452640 C Bi:1:006:8 -104 0
ffff0000bbb00a00 3414452648 C Bi:1:006:8 -104 0
ffff000022a72400 3414452902 C Bi:1:006:8 -104 0
ffff000046f6a200 3414452911 C Bi:1:006:8 -104 0
ffff000046f6a500 3414452918 C Bi:1:006:8 -104 0
ffff000030dff800 3414452925 C Bi:1:006:8 -104 0
ffff000046f6a400 3414453134 C Bi:1:006:8 -104 0
ffff000046f6ab00 3414453148 C Bi:1:006:8 -104 0
ffff000046f6a300 3414453154 C Bi:1:006:8 -104 0
ffff00002472d300 3414453436 C Bi:1:006:8 -104 0
ffff00002472de00 3414453443 C Bi:1:006:8 -104 0
ffff000046f6ad00 3414453452 C Bi:1:006:8 -104 0
ffff0000bbe51d00 3414454606 C Ii:1:006:7 -2:16 0
ffff00004834fc00 3414455737 C Bi:1:006:4 -2 0
ffff0000bc5e0200 3414456688 C Bi:1:006:4 -2 0
ffff0000bc5e0300 3414456896 C Bi:1:006:4 -2 0
ffff0000bc5e0400 3414457772 C Bi:1:006:4 -2 0
ffff00004834f500 3414458677 C Ii:1:006:3 -2:16 0
ffff000031511f00 3414459626 C Bi:1:006:2 -2 0
ffff000031511200 3414459845 C Bi:1:006:2 -2 0
ffff000031511b00 3414460031 C Bi:1:006:2 -2 0
ffff000031511d00 3414460924 C Bi:1:006:2 -2 0
ffff00001fd2df00 3414461909 C Bi:1:006:1 -2 0
ffff00001fd2d000 3414462085 C Bi:1:006:1 -2 0
ffff00000e904100 3414463049 C Bi:1:006:1 -2 0
ffff000014838000 3414463216 C Bi:1:006:1 -2 0
ffff00000bc3c300 3414463353 S Co:1:006:0 s 00 03 0001 0000 0000 0
ffff00000bc3c300 3414463654 C Co:1:006:0 0 0
ffff00000bc3c300 3414463709 S Co:1:002:0 s 23 03 0002 0002 0000 0
ffff00000bc3c300 3414463795 C Co:1:002:0 0 0
ffff0000befcda00 3414485333 C Ii:1:002:1 -2:2048 0
ffff00000bc3c300 3414485443 S Co:1:002:0 s 00 03 0001 0000 0000 0
ffff00000bc3c300 3414485535 C Co:1:002:0 0 0
ffff00000bc3c300 3414485563 S Co:1:001:0 s 23 03 0002 0001 0000 0
ffff00000bc3c300 3414504874 C Co:1:001:0 0 0
ffff0000bf035e00 3414548807 C Ii:1:001:1 0:2048 1 = 02
ffff0000bf035e00 3414548937 S Ii:1:001:1 -115:2048 4 <
ffff0000bc5a0600 3414549188 S Ci:1:001:0 s a3 00 0000 0001 0004 4 <
ffff0000bc5a0600 3414550125 C Ci:1:001:0 0 4 = 03050400
ffff0000bc5a0600 3414550224 S Co:1:001:0 s 23 01 0012 0001 0000 0
ffff0000bc5a0600 3414550253 C Co:1:001:0 0 0
ffff0000bc5a0600 3414572257 S Ci:1:001:0 s a3 00 0000 0001 0004 4 <
ffff0000bc5a0600 3414572363 C Ci:1:001:0 0 4 = 03050000
ffff0000bc5a0600 3414592209 S Ci:1:002:0 s 80 00 0000 0000 0002 2 <
ffff0000bc5a0600 3414593195 C Ci:1:002:0 0 2 = 0300
ffff0000bc5a0600 3414593249 S Co:1:002:0 s 00 01 0001 0000 0000 0
ffff0000bc5a0600 3414594022 C Co:1:002:0 0 0
ffff0000bc5a0600 3414594134 S Ci:1:002:0 s a3 00 0000 0001 0004 4 <
ffff0000bc5a0600 3414594854 C Ci:1:002:0 0 4 = 07050000
ffff0000bc5a0600 3414594917 S Ci:1:002:0 s a3 00 0000 0002 0004 4 <
ffff0000bc5a0600 3414595674 C Ci:1:002:0 0 4 = 01010500
ffff0000bc5a0600 3414595723 S Co:1:002:0 s 23 01 0010 0002 0000 0
ffff0000bc5a0600 3414596471 C Co:1:002:0 0 0
ffff0000bc5a0600 3414596885 S Ci:1:002:0 s a3 00 0000 0003 0004 4 <
ffff0000bc5a0600 3414597661 C Ci:1:002:0 0 4 = 00010000
ffff0000befcda00 3414704815 S Ii:1:002:1 -115:2048 1 <
ffff0000bc5a0600 3414705456 S Ci:1:002:0 s a3 00 0000 0002 0004 4 <
ffff0000bc5a0600 3414705668 C Ci:1:002:0 0 4 = 01010500
ffff0000bc5a0600 3414705706 S Co:1:002:0 s 23 01 0010 0002 0000 0
ffff0000bc5a0600 3414705781 C Co:1:002:0 0 0
ffff0000bc5a0600 3414705810 S Co:1:002:0 s 23 01 0012 0002 0000 0
ffff0000bc5a0600 3414705876 C Co:1:002:0 0 0
ffff0000bc5a0600 3414728273 S Ci:1:002:0 s a3 00 0000 0002 0004 4 <
ffff0000bc5a0600 3414729045 C Ci:1:002:0 0 4 = 01010000
ffff0000bc5a0600 3414749623 S Co:1:002:0 s 23 03 0004 0002 0000 0
ffff0000bc5a0600 3414750388 C Co:1:002:0 0 0
ffff0000bc5a0600 3414772782 S Ci:1:002:0 s a3 00 0000 0002 0004 4 <
ffff0000bc5a0600 3414773561 C Ci:1:002:0 0 4 = 03051000
ffff0000bc5a0600 3414773606 S Co:1:002:0 s 23 01 0014 0002 0000 0
ffff0000bc5a0600 3414773677 C Co:1:002:0 0 0
ffff0000bc5a0600 3414857343 S Ci:1:000:0 s 80 06 0100 0000 0040 64 <
ffff0000bc5a0600 3414857813 C Ci:1:000:0 0 18 = 12010002 00000040 20206020 00000302 0401
ffff0000bc5a0600 3414857922 S Co:1:002:0 s 23 03 0004 0002 0000 0
ffff0000bc5a0600 3414858015 C Co:1:002:0 0 0
ffff0000bc5a0600 3414876395 S Ci:1:002:0 s a3 00 0000 0002 0004 4 <
ffff0000bc5a0600 3414877371 C Ci:1:002:0 0 4 = 03051000
ffff0000bc5a0600 3414878332 S Co:1:002:0 s 23 01 0014 0002 0000 0
ffff0000bc5a0600 3414878450 C Co:1:002:0 0 0
ffff0000bc5a0600 3414956790 S Ci:1:006:0 s 80 06 0100 0000 0012 18 <
ffff0000bc5a0600 3414957867 C Ci:1:006:0 0 18 = 12010002 00000040 20206020 00000302 0401
ffff0000bc5a0300 3414958120 S Ci:1:006:0 s 80 06 0200 0000 0091 145 <
ffff0000bc5a0300 3414959137 C Ci:1:006:0 0 145 = 09029100 050101e0 fa090400 0002ffff ff000705 81020002 00070501 02000200
ffff0000bc5a0600 3414983088 S Co:1:006:0 s 00 09 0001 0000 0000 0
ffff0000bc5a0600 3414984395 C Co:1:006:0 0 0
ffff0000bc5a0600 3414988497 S Ci:1:006:0 s 80 00 0000 0000 0002 2 <
ffff0000bc5a0600 3414988847 C Ci:1:006:0 0 2 = 0000
ffff0000bbe51d00 3414989618 S Ii:1:006:7 -115:16 64 <
ffff0000bc5a0600 3414989746 S Bi:1:006:8 -115 1500 <
ffff0000bc5a0300 3414989778 S Bi:1:006:8 -115 1500 <
ffff0000bc5a0200 3414989807 S Bi:1:006:8 -115 1500 <
ffff0000bc5a0400 3414989835 S Bi:1:006:8 -115 1500 <
ffff0000481b8c00 3414989867 S Bi:1:006:8 -115 1500 <
ffff0000481b8a00 3414989891 S Bi:1:006:8 -115 1500 <
ffff0000481b8400 3414989915 S Bi:1:006:8 -115 1500 <
ffff0000481b8300 3414989935 S Bi:1:006:8 -115 1500 <
ffff0000481b8d00 3414989959 S Bi:1:006:8 -115 1500 <
ffff0000481b8900 3414989984 S Bi:1:006:8 -115 1500 <