Memory barrier needed with wake_up_process()?

Memory barrier needed with wake_up_process()?

[Alan Stern]

Paul, Peter, and Ingo:

This must have come up before, but I don't know what was decided.

Isn't it often true that a memory barrier is needed before a call to 
wake_up_process()?  A typical scenario might look like this:

	CPU 0
	-----
	for (;;) {
		set_current_state(TASK_INTERRUPTIBLE);
		if (signal_pending(current))
			break;
		if (wakeup_flag)
			break;
		schedule();
	}
	__set_current_state(TASK_RUNNING);
	wakeup_flag = 0;


	CPU 1
	-----
	wakeup_flag = 1;
	wake_up_process(my_task);

The underlying pattern is:

	CPU 0				CPU 1
	-----				-----
	write current->state		write wakeup_flag
	smp_mb();
	read wakeup_flag		read my_task->state

where set_current_state() does the write to current->state and 
automatically adds the smp_mb(), and wake_up_process() reads 
my_task->state to see whether the task needs to be woken up.

The kerneldoc for wake_up_process() says that it has no implied memory
barrier if it doesn't actually wake anything up.  And even when it
does, the implied barrier is only smp_wmb, not smp_mb.

This is the so-called SB (Store Buffer) pattern, which is well known to
require a full smp_mb on both sides.  Since wake_up_process() doesn't
include smp_mb(), isn't it correct that the caller must add it
explicitly?

In other words, shouldn't the code for CPU 1 really be:

	wakeup_flag = 1;
	smp_mb();
	wake_up_process(task);

If my reasoning is correct, then why doesn't wake_up_process() include 
this memory barrier automatically, the way set_current_state() does?  
There could be an alternate version (__wake_up_process()) which omits 
the barrier, just like __set_current_state().

Alan Stern

Re: Memory barrier needed with wake_up_process()?

[Paul E. McKenney]

On Fri, Sep 02, 2016 at 02:10:13PM -0400, Alan Stern wrote:
> Paul, Peter, and Ingo:
> 
> This must have come up before, but I don't know what was decided.
> 
> Isn't it often true that a memory barrier is needed before a call to 
> wake_up_process()?  A typical scenario might look like this:
> 
> 	CPU 0
> 	-----
> 	for (;;) {
> 		set_current_state(TASK_INTERRUPTIBLE);
> 		if (signal_pending(current))
> 			break;
> 		if (wakeup_flag)
> 			break;
> 		schedule();
> 	}
> 	__set_current_state(TASK_RUNNING);
> 	wakeup_flag = 0;
> 
> 
> 	CPU 1
> 	-----
> 	wakeup_flag = 1;
> 	wake_up_process(my_task);
> 
> The underlying pattern is:
> 
> 	CPU 0				CPU 1
> 	-----				-----
> 	write current->state		write wakeup_flag
> 	smp_mb();
> 	read wakeup_flag		read my_task->state
> 
> where set_current_state() does the write to current->state and 
> automatically adds the smp_mb(), and wake_up_process() reads 
> my_task->state to see whether the task needs to be woken up.
> 
> The kerneldoc for wake_up_process() says that it has no implied memory
> barrier if it doesn't actually wake anything up.  And even when it
> does, the implied barrier is only smp_wmb, not smp_mb.
> 
> This is the so-called SB (Store Buffer) pattern, which is well known to
> require a full smp_mb on both sides.  Since wake_up_process() doesn't
> include smp_mb(), isn't it correct that the caller must add it
> explicitly?
> 
> In other words, shouldn't the code for CPU 1 really be:
> 
> 	wakeup_flag = 1;
> 	smp_mb();
> 	wake_up_process(task);
> 
> If my reasoning is correct, then why doesn't wake_up_process() include 
> this memory barrier automatically, the way set_current_state() does?  
> There could be an alternate version (__wake_up_process()) which omits 
> the barrier, just like __set_current_state().

A common case uses locking, in which case additional memory barriers
inside of the wait/wakeup functions are not needed.  Any accesses made
while holding the lock before invoking the wakeup function (e.g.,
wake_up()) are guaranteed to be seen after acquiring that same
lock following return from the wait function (e.g., wait_event()).
In this case, adding barriers to the wait and wakeup functions would
just add overhead.

But yes, this decision does mean that people using the wait/wakeup
functions without locking need to be more careful.  Something like
this:

	/* prior accesses. */
	smp_mb();
	wakeup_flag = 1;
	wake_up(...);

And on the other task:

	wait_event(... wakeup_flag == 1 ...);
	smp_mb();
	/* The waker's prior accesses will be visible here. */

Or am I missing your point?

						Thanx, Paul

Re: Memory barrier needed with wake_up_process()?

[Alan Stern]

On Fri, 2 Sep 2016, Paul E. McKenney wrote:

> On Fri, Sep 02, 2016 at 02:10:13PM -0400, Alan Stern wrote:
> > Paul, Peter, and Ingo:
> > 
> > This must have come up before, but I don't know what was decided.
> > 
> > Isn't it often true that a memory barrier is needed before a call to 
> > wake_up_process()?  A typical scenario might look like this:
> > 
> > 	CPU 0
> > 	-----
> > 	for (;;) {
> > 		set_current_state(TASK_INTERRUPTIBLE);
> > 		if (signal_pending(current))
> > 			break;
> > 		if (wakeup_flag)
> > 			break;
> > 		schedule();
> > 	}
> > 	__set_current_state(TASK_RUNNING);
> > 	wakeup_flag = 0;
> > 
> > 
> > 	CPU 1
> > 	-----
> > 	wakeup_flag = 1;
> > 	wake_up_process(my_task);
> > 
> > The underlying pattern is:
> > 
> > 	CPU 0				CPU 1
> > 	-----				-----
> > 	write current->state		write wakeup_flag
> > 	smp_mb();
> > 	read wakeup_flag		read my_task->state
> > 
> > where set_current_state() does the write to current->state and 
> > automatically adds the smp_mb(), and wake_up_process() reads 
> > my_task->state to see whether the task needs to be woken up.
> > 
> > The kerneldoc for wake_up_process() says that it has no implied memory
> > barrier if it doesn't actually wake anything up.  And even when it
> > does, the implied barrier is only smp_wmb, not smp_mb.
> > 
> > This is the so-called SB (Store Buffer) pattern, which is well known to
> > require a full smp_mb on both sides.  Since wake_up_process() doesn't
> > include smp_mb(), isn't it correct that the caller must add it
> > explicitly?
> > 
> > In other words, shouldn't the code for CPU 1 really be:
> > 
> > 	wakeup_flag = 1;
> > 	smp_mb();
> > 	wake_up_process(task);
> > 
> > If my reasoning is correct, then why doesn't wake_up_process() include 
> > this memory barrier automatically, the way set_current_state() does?  
> > There could be an alternate version (__wake_up_process()) which omits 
> > the barrier, just like __set_current_state().
> 
> A common case uses locking, in which case additional memory barriers
> inside of the wait/wakeup functions are not needed.  Any accesses made
> while holding the lock before invoking the wakeup function (e.g.,
> wake_up()) are guaranteed to be seen after acquiring that same
> lock following return from the wait function (e.g., wait_event()).
> In this case, adding barriers to the wait and wakeup functions would
> just add overhead.
> 
> But yes, this decision does mean that people using the wait/wakeup
> functions without locking need to be more careful.  Something like
> this:
> 
> 	/* prior accesses. */
> 	smp_mb();
> 	wakeup_flag = 1;
> 	wake_up(...);
> 
> And on the other task:
> 
> 	wait_event(... wakeup_flag == 1 ...);
> 	smp_mb();
> 	/* The waker's prior accesses will be visible here. */
> 
> Or am I missing your point?

I'm afraid so.  The code doesn't use wait_event(), in part because
there's no wait_queue (since only one task is involved).

But maybe there's another barrier which needs to be fixed.  Felipe, can
you check to see if received_cbw() is getting called in
get_next_command(), and if so, what value it returns?  Or is the
preceding sleep_thread() the one that never wakes up?

It could be that the smp_wmb() in wakeup_thread() needs to be smp_mb().  
The reason being that get_next_command() runs outside the protection of 
the spinlock.

Alan Stern

Re: Memory barrier needed with wake_up_process()?

[Paul E. McKenney]

On Fri, Sep 02, 2016 at 04:29:19PM -0400, Alan Stern wrote:
> On Fri, 2 Sep 2016, Paul E. McKenney wrote:
> 
> > On Fri, Sep 02, 2016 at 02:10:13PM -0400, Alan Stern wrote:
> > > Paul, Peter, and Ingo:
> > > 
> > > This must have come up before, but I don't know what was decided.
> > > 
> > > Isn't it often true that a memory barrier is needed before a call to 
> > > wake_up_process()?  A typical scenario might look like this:
> > > 
> > > 	CPU 0
> > > 	-----
> > > 	for (;;) {
> > > 		set_current_state(TASK_INTERRUPTIBLE);
> > > 		if (signal_pending(current))
> > > 			break;
> > > 		if (wakeup_flag)
> > > 			break;
> > > 		schedule();
> > > 	}
> > > 	__set_current_state(TASK_RUNNING);
> > > 	wakeup_flag = 0;
> > > 
> > > 
> > > 	CPU 1
> > > 	-----
> > > 	wakeup_flag = 1;
> > > 	wake_up_process(my_task);
> > > 
> > > The underlying pattern is:
> > > 
> > > 	CPU 0				CPU 1
> > > 	-----				-----
> > > 	write current->state		write wakeup_flag
> > > 	smp_mb();
> > > 	read wakeup_flag		read my_task->state
> > > 
> > > where set_current_state() does the write to current->state and 
> > > automatically adds the smp_mb(), and wake_up_process() reads 
> > > my_task->state to see whether the task needs to be woken up.
> > > 
> > > The kerneldoc for wake_up_process() says that it has no implied memory
> > > barrier if it doesn't actually wake anything up.  And even when it
> > > does, the implied barrier is only smp_wmb, not smp_mb.
> > > 
> > > This is the so-called SB (Store Buffer) pattern, which is well known to
> > > require a full smp_mb on both sides.  Since wake_up_process() doesn't
> > > include smp_mb(), isn't it correct that the caller must add it
> > > explicitly?
> > > 
> > > In other words, shouldn't the code for CPU 1 really be:
> > > 
> > > 	wakeup_flag = 1;
> > > 	smp_mb();
> > > 	wake_up_process(task);
> > > 
> > > If my reasoning is correct, then why doesn't wake_up_process() include 
> > > this memory barrier automatically, the way set_current_state() does?  
> > > There could be an alternate version (__wake_up_process()) which omits 
> > > the barrier, just like __set_current_state().
> > 
> > A common case uses locking, in which case additional memory barriers
> > inside of the wait/wakeup functions are not needed.  Any accesses made
> > while holding the lock before invoking the wakeup function (e.g.,
> > wake_up()) are guaranteed to be seen after acquiring that same
> > lock following return from the wait function (e.g., wait_event()).
> > In this case, adding barriers to the wait and wakeup functions would
> > just add overhead.
> > 
> > But yes, this decision does mean that people using the wait/wakeup
> > functions without locking need to be more careful.  Something like
> > this:
> > 
> > 	/* prior accesses. */
> > 	smp_mb();
> > 	wakeup_flag = 1;
> > 	wake_up(...);
> > 
> > And on the other task:
> > 
> > 	wait_event(... wakeup_flag == 1 ...);
> > 	smp_mb();
> > 	/* The waker's prior accesses will be visible here. */
> > 
> > Or am I missing your point?
> 
> I'm afraid so.  The code doesn't use wait_event(), in part because
> there's no wait_queue (since only one task is involved).

Ah, got it.

The required pattern should be very similar, however.

> But maybe there's another barrier which needs to be fixed.  Felipe, can
> you check to see if received_cbw() is getting called in
> get_next_command(), and if so, what value it returns?  Or is the
> preceding sleep_thread() the one that never wakes up?
> 
> It could be that the smp_wmb() in wakeup_thread() needs to be smp_mb().  
> The reason being that get_next_command() runs outside the protection of 
> the spinlock.

This sounds very likely to me.

							Thanx, Paul

Re: Memory barrier needed with wake_up_process()?

[Peter Zijlstra]

On Fri, Sep 02, 2016 at 04:29:19PM -0400, Alan Stern wrote:
> I'm afraid so.  The code doesn't use wait_event(), in part because
> there's no wait_queue (since only one task is involved).

You can use wait_queue fine with just one task, and it would clean up
the code tremendously.

You can replace things like the earlier mentioned:

	while (bh->state != BUF_STATE_EMPTY) {
		rc = sleep_thread(common, false);
		if (rc)
			return rc;
	}

with:

	rc = wait_event_interruptible(&common->wq, bh->state == BUF_STATE_EMPTY);
	if (rc)
		return rc;

> But maybe there's another barrier which needs to be fixed.  Felipe, can
> you check to see if received_cbw() is getting called in
> get_next_command(), and if so, what value it returns?  Or is the
> preceding sleep_thread() the one that never wakes up?
> 
> It could be that the smp_wmb() in wakeup_thread() needs to be smp_mb().  
> The reason being that get_next_command() runs outside the protection of 
> the spinlock.

Being somewhat confused by the code, I fail to follow that argument.
wakeup_thread() is always called under that spinlock(), but since the
critical section is 2 stores, I fail to see how a smp_mb() can make any
difference over the smp_wmb() already there.

Re: Memory barrier needed with wake_up_process()?

[Alan Stern]

On Sat, 3 Sep 2016, Peter Zijlstra wrote:

> On Fri, Sep 02, 2016 at 04:29:19PM -0400, Alan Stern wrote:
> > I'm afraid so.  The code doesn't use wait_event(), in part because
> > there's no wait_queue (since only one task is involved).
> 
> You can use wait_queue fine with just one task, and it would clean up
> the code tremendously.
> 
> You can replace things like the earlier mentioned:
> 
> 	while (bh->state != BUF_STATE_EMPTY) {
> 		rc = sleep_thread(common, false);
> 		if (rc)
> 			return rc;
> 	}
> 
> with:
> 
> 	rc = wait_event_interruptible(&common->wq, bh->state == BUF_STATE_EMPTY);
> 	if (rc)
> 		return rc;

If someone wants to devote time and effort to cleaning up the driver, 
that would be a good start.

> > But maybe there's another barrier which needs to be fixed.  Felipe, can
> > you check to see if received_cbw() is getting called in
> > get_next_command(), and if so, what value it returns?  Or is the
> > preceding sleep_thread() the one that never wakes up?
> > 
> > It could be that the smp_wmb() in wakeup_thread() needs to be smp_mb().  
> > The reason being that get_next_command() runs outside the protection of 
> > the spinlock.
> 
> Being somewhat confused by the code, I fail to follow that argument.
> wakeup_thread() is always called under that spinlock(), but since the
> critical section is 2 stores, I fail to see how a smp_mb() can make any
> difference over the smp_wmb() already there.

But sleep_thread() and the code that follows it are _not_ called under
the spinlock.  And the following code examines values that were written
by DMA, not by the CPU calling wakeup_thread().  (Although that CPU
_is_ the one that receives the DMA-completion notice.)

In other words, we have:

	CPU 0				CPU 1
	-----				-----
	Start DMA			Handle DMA-complete irq
	Sleep until bh->state		Set bh->state
					smp_wmb()
					Wake up CPU 0
	smp_rmb()
	Compute rc based on contents
		of the DMA buffer

This was written many years ago, at a time when I did not fully
understand all the details of memory ordering.  Do you agree that both
of those barriers should really be smp_mb()?  That's what Felipe has
been testing.

Alan Stern

Re: Memory barrier needed with wake_up_process()?

[Alan Stern]

On Sat, 3 Sep 2016, Alan Stern wrote:

> In other words, we have:
> 
> 	CPU 0				CPU 1
> 	-----				-----
> 	Start DMA			Handle DMA-complete irq
> 	Sleep until bh->state		Set bh->state
> 					smp_wmb()
> 					Wake up CPU 0
> 	smp_rmb()
> 	Compute rc based on contents
> 		of the DMA buffer
> 
> This was written many years ago, at a time when I did not fully
> understand all the details of memory ordering.  Do you agree that both
> of those barriers should really be smp_mb()?  That's what Felipe has
> been testing.

Actually, seeing it written out like this, one realizes that it really 
ought to be:

	CPU 0				CPU 1
        -----				-----
        Start DMA			Handle DMA-complete irq
        Sleep until bh->state		smp_mb()
					set bh->state
					Wake up CPU 0
	smp_mb()
	Compute rc based on contents of the DMA buffer

(Bear in mind also that on some platforms, the I/O operation is carried 
out by PIO rather than DMA.)

Also, the smp_wmb() in bulk_out_complete() looks unnecessary.  I can't 
remember why I put it there originally.

Alan Stern

Re: Memory barrier needed with wake_up_process()?

[Peter Zijlstra]

On Sat, Sep 03, 2016 at 10:49:39AM -0400, Alan Stern wrote:
> On Sat, 3 Sep 2016, Alan Stern wrote:
> 
> > In other words, we have:
> > 
> > 	CPU 0				CPU 1
> > 	-----				-----
> > 	Start DMA			Handle DMA-complete irq
> > 	Sleep until bh->state		Set bh->state
> > 					smp_wmb()
> > 					Wake up CPU 0
> > 	smp_rmb()
> > 	Compute rc based on contents
> > 		of the DMA buffer
> > 
> > This was written many years ago, at a time when I did not fully
> > understand all the details of memory ordering.  Do you agree that both
> > of those barriers should really be smp_mb()?  That's what Felipe has
> > been testing.
> 
> Actually, seeing it written out like this, one realizes that it really 
> ought to be:
> 
> 	CPU 0				CPU 1
>         -----				-----
>         Start DMA			Handle DMA-complete irq
>         Sleep until bh->state		smp_mb()
> 					set bh->state
> 					Wake up CPU 0
> 	smp_mb()
> 	Compute rc based on contents of the DMA buffer
> 
> (Bear in mind also that on some platforms, the I/O operation is carried 
> out by PIO rather than DMA.)

I'm sorry, but I still don't follow. This could be because I seldom
interact with DMA agents and therefore am not familiar with that stuff.

Also, smp_mb() doesn't necessarily interact with MMIO/DMA at all IIRC.
Its only defined to do CPU/CPU interactions.

I would very much expect the device IO stuff to order things for us in
this case. "Start DMA" should very much include sufficient fences to
ensure the data under DMA is visible to the DMA engine, this would very
much include things like flushing store buffers and maybe even writeback
caches, depending on platform needs.

At the same time, I would expect "Handle DMA-complete irq", even if it
were done with a PIO polling loop, to guarantee ordering against later
operations such that 'complete' really means that.

Re: Memory barrier needed with wake_up_process()?

[Alan Stern]

On Mon, 5 Sep 2016, Peter Zijlstra wrote:

> > Actually, seeing it written out like this, one realizes that it really 
> > ought to be:
> > 
> > 	CPU 0				CPU 1
> >         -----				-----
> >         Start DMA			Handle DMA-complete irq
> >         Sleep until bh->state		smp_mb()
> > 					set bh->state
> > 					Wake up CPU 0
> > 	smp_mb()
> > 	Compute rc based on contents of the DMA buffer
> > 
> > (Bear in mind also that on some platforms, the I/O operation is carried 
> > out by PIO rather than DMA.)
> 
> I'm sorry, but I still don't follow. This could be because I seldom
> interact with DMA agents and therefore am not familiar with that stuff.

I haven't seen the details of memory ordering requirements in the
presence of DMA spelled out anywhere.  The documents I have read merely
state that you have to careful to flush caches before doing DMA OUT and
to avoid filling caches before DMA IN is complete.  Neither of those is
an issue here, apparently.

> Also, smp_mb() doesn't necessarily interact with MMIO/DMA at all IIRC.
> Its only defined to do CPU/CPU interactions.

Suppose the DMA master finishes filling up an input buffer and issues a
completion irq to CPU1.  Presumably the data would then be visible to
CPU1 if the interrupt handler looked at it.  So if CPU1 executes smp_mb
before setting bh->state and waking up CPU0, and if CPU0 executes
smp_mb after testing bh->state and before reading the data buffer,
wouldn't CPU0 then see the correct data in the buffer?  Even if CPU0 
never did go to sleep?

Or would something more be needed?

> I would very much expect the device IO stuff to order things for us in
> this case. "Start DMA" should very much include sufficient fences to
> ensure the data under DMA is visible to the DMA engine, this would very
> much include things like flushing store buffers and maybe even writeback
> caches, depending on platform needs.
> 
> At the same time, I would expect "Handle DMA-complete irq", even if it
> were done with a PIO polling loop, to guarantee ordering against later
> operations such that 'complete' really means that.

That's what I would expect too.

Back in the original email thread where the problem was first reported, 
Felipe says that the problem appears to be something else.  Here's what 
it looks like now, in schematic form:

	CPU0
	----
	get_next_command():
	  while (bh->state != BUF_STATE_EMPTY)
		sleep_thread();
	  start an input request for bh
	  while (bh->state != BUF_STATE_FULL)
		sleep_thread();

As mentioned above, the input involves DMA and is terminated by an irq.
The request's completion handler is bulk_out_complete():

	CPU1
	----
	bulk_out_complete():
	  bh->state = BUF_STATE_FULL;
	  wakeup_thread();

According to Felipe, when CPU0 wakes up and checks bh->state, it sees a 
value different from BUF_STATE_FULL.  So it goes back to sleep again 
and doesn't make any forward progress.

It's possible that something else is changing bh->state when it 
shouldn't.  But if this were the explanation, why would Felipe see that 
the problem goes away when he changes the memory barriers in 
sleep_thread() and wakeup_thread() to smp_mb()?

Alan Stern

Re: Memory barrier needed with wake_up_process()?

[Peter Zijlstra]

On Mon, Sep 05, 2016 at 11:29:26AM -0400, Alan Stern wrote:
> On Mon, 5 Sep 2016, Peter Zijlstra wrote:
> 
> > > Actually, seeing it written out like this, one realizes that it really 
> > > ought to be:
> > > 
> > > 	CPU 0				CPU 1
> > >         -----				-----
> > >         Start DMA			Handle DMA-complete irq
> > >         Sleep until bh->state		smp_mb()
> > > 					set bh->state
> > > 					Wake up CPU 0
> > > 	smp_mb()
> > > 	Compute rc based on contents of the DMA buffer
> > > 
> > > (Bear in mind also that on some platforms, the I/O operation is carried 
> > > out by PIO rather than DMA.)

> > Also, smp_mb() doesn't necessarily interact with MMIO/DMA at all IIRC.
> > Its only defined to do CPU/CPU interactions.
> 
> Suppose the DMA master finishes filling up an input buffer and issues a
> completion irq to CPU1.  Presumably the data would then be visible to
> CPU1 if the interrupt handler looked at it.  So if CPU1 executes smp_mb
> before setting bh->state and waking up CPU0, and if CPU0 executes
> smp_mb after testing bh->state and before reading the data buffer,
> wouldn't CPU0 then see the correct data in the buffer?  Even if CPU0 
> never did go to sleep?

Couple of notes here; I would expect the DMA master to make its stores
_globally_ visible on 'completion'. Because I'm not sure our smp_mb()
would help make it globally visible, since its only defined on CPU/CPU
interactions, not external.

Note that for example ARM has the distinction where smp_mb() uses
DMB-ISH barrier, dma_[rw]mb() uses DMB-OSH{LD,ST} and mb() uses DSB-SY.

The ISH domain is the Inner-SHarable (IIRC) and only includes CPUs. The
OSH is Outer-SHarable and adds external agents like DMA (also includes
CPUs). The DSB-SY thing is even heavier and syncs world or something; I
always forget these details.

> Or would something more be needed?

The thing is, this is x86 (TSO). Most everything is globally
ordered/visible and full barriers.

The only reorder allowed by TSO is a later read can happen before a
prior store. That is, only:

	X = 1;
	smp_mb();
	r = Y;

actually needs a full barrier. All the other variants are no-ops.

Note that x86's dma_[rw]mb() are no-ops (like the regular smp_[rw]mb()).
Which seems to imply DMA is coherent and globally visible.

> > I would very much expect the device IO stuff to order things for us in
> > this case. "Start DMA" should very much include sufficient fences to
> > ensure the data under DMA is visible to the DMA engine, this would very
> > much include things like flushing store buffers and maybe even writeback
> > caches, depending on platform needs.
> > 
> > At the same time, I would expect "Handle DMA-complete irq", even if it
> > were done with a PIO polling loop, to guarantee ordering against later
> > operations such that 'complete' really means that.
> 
> That's what I would expect too.
> 
> Back in the original email thread where the problem was first reported, 
> Felipe says that the problem appears to be something else.  Here's what 
> it looks like now, in schematic form:
> 
> 	CPU0
> 	----
> 	get_next_command():
> 	  while (bh->state != BUF_STATE_EMPTY)
> 		sleep_thread();
> 	  start an input request for bh
> 	  while (bh->state != BUF_STATE_FULL)
> 		sleep_thread();
> 
> As mentioned above, the input involves DMA and is terminated by an irq.
> The request's completion handler is bulk_out_complete():
> 
> 	CPU1
> 	----
> 	bulk_out_complete():
> 	  bh->state = BUF_STATE_FULL;
> 	  wakeup_thread();
> 
> According to Felipe, when CPU0 wakes up and checks bh->state, it sees a 
> value different from BUF_STATE_FULL.  So it goes back to sleep again 
> and doesn't make any forward progress.
> 
> It's possible that something else is changing bh->state when it 
> shouldn't.  But if this were the explanation, why would Felipe see that 
> the problem goes away when he changes the memory barriers in 
> sleep_thread() and wakeup_thread() to smp_mb()?

Good question, let me stare at the code more.

Could you confirm that bulk_{in,out}_complete() work on different
usb_request structures, and they can not, at any time, get called on the
_same_ request?

Re: Memory barrier needed with wake_up_process()?

[Felipe Balbi]

[-- Attachment #1: Type: text/plain, Size: 4811 bytes --]


Hi,

Peter Zijlstra <peterz@infradead.org> writes:
> On Mon, Sep 05, 2016 at 11:29:26AM -0400, Alan Stern wrote:
>> On Mon, 5 Sep 2016, Peter Zijlstra wrote:
>> 
>> > > Actually, seeing it written out like this, one realizes that it really 
>> > > ought to be:
>> > > 
>> > > 	CPU 0				CPU 1
>> > >         -----				-----
>> > >         Start DMA			Handle DMA-complete irq
>> > >         Sleep until bh->state		smp_mb()
>> > > 					set bh->state
>> > > 					Wake up CPU 0
>> > > 	smp_mb()
>> > > 	Compute rc based on contents of the DMA buffer
>> > > 
>> > > (Bear in mind also that on some platforms, the I/O operation is carried 
>> > > out by PIO rather than DMA.)
>
>> > Also, smp_mb() doesn't necessarily interact with MMIO/DMA at all IIRC.
>> > Its only defined to do CPU/CPU interactions.
>> 
>> Suppose the DMA master finishes filling up an input buffer and issues a
>> completion irq to CPU1.  Presumably the data would then be visible to
>> CPU1 if the interrupt handler looked at it.  So if CPU1 executes smp_mb
>> before setting bh->state and waking up CPU0, and if CPU0 executes
>> smp_mb after testing bh->state and before reading the data buffer,
>> wouldn't CPU0 then see the correct data in the buffer?  Even if CPU0 
>> never did go to sleep?
>
> Couple of notes here; I would expect the DMA master to make its stores
> _globally_ visible on 'completion'. Because I'm not sure our smp_mb()
> would help make it globally visible, since its only defined on CPU/CPU
> interactions, not external.
>
> Note that for example ARM has the distinction where smp_mb() uses
> DMB-ISH barrier, dma_[rw]mb() uses DMB-OSH{LD,ST} and mb() uses DSB-SY.
>
> The ISH domain is the Inner-SHarable (IIRC) and only includes CPUs. The
> OSH is Outer-SHarable and adds external agents like DMA (also includes
> CPUs). The DSB-SY thing is even heavier and syncs world or something; I
> always forget these details.
>
>> Or would something more be needed?
>
> The thing is, this is x86 (TSO). Most everything is globally
> ordered/visible and full barriers.
>
> The only reorder allowed by TSO is a later read can happen before a
> prior store. That is, only:
>
> 	X = 1;
> 	smp_mb();
> 	r = Y;
>
> actually needs a full barrier. All the other variants are no-ops.
>
> Note that x86's dma_[rw]mb() are no-ops (like the regular smp_[rw]mb()).
> Which seems to imply DMA is coherent and globally visible.
>
>> > I would very much expect the device IO stuff to order things for us in
>> > this case. "Start DMA" should very much include sufficient fences to
>> > ensure the data under DMA is visible to the DMA engine, this would very
>> > much include things like flushing store buffers and maybe even writeback
>> > caches, depending on platform needs.
>> > 
>> > At the same time, I would expect "Handle DMA-complete irq", even if it
>> > were done with a PIO polling loop, to guarantee ordering against later
>> > operations such that 'complete' really means that.
>> 
>> That's what I would expect too.
>> 
>> Back in the original email thread where the problem was first reported, 
>> Felipe says that the problem appears to be something else.  Here's what 
>> it looks like now, in schematic form:
>> 
>> 	CPU0
>> 	----
>> 	get_next_command():
>> 	  while (bh->state != BUF_STATE_EMPTY)
>> 		sleep_thread();
>> 	  start an input request for bh
>> 	  while (bh->state != BUF_STATE_FULL)
>> 		sleep_thread();
>> 
>> As mentioned above, the input involves DMA and is terminated by an irq.
>> The request's completion handler is bulk_out_complete():
>> 
>> 	CPU1
>> 	----
>> 	bulk_out_complete():
>> 	  bh->state = BUF_STATE_FULL;
>> 	  wakeup_thread();
>> 
>> According to Felipe, when CPU0 wakes up and checks bh->state, it sees a 
>> value different from BUF_STATE_FULL.  So it goes back to sleep again 
>> and doesn't make any forward progress.
>> 
>> It's possible that something else is changing bh->state when it 
>> shouldn't.  But if this were the explanation, why would Felipe see that 
>> the problem goes away when he changes the memory barriers in 
>> sleep_thread() and wakeup_thread() to smp_mb()?
>
> Good question, let me stare at the code more.
>
> Could you confirm that bulk_{in,out}_complete() work on different
> usb_request structures, and they can not, at any time, get called on the
> _same_ request?

usb_requests are allocated for a specific endpoint and USB Device
Controller (UDC) drivers refuse to queue requests allocated for epX to
epY, so this really can never happen.

My fear now, however, is that changing smp_[rw]mb() to smp_mb() just
adds extra overhead which makes the problem much, much less likely to
happen. Does that sound plausible to you?

-- 
balbi

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 800 bytes --]

Re: Memory barrier needed with wake_up_process()?

[Peter Zijlstra]

On Tue, Sep 06, 2016 at 02:43:39PM +0300, Felipe Balbi wrote:

> > Could you confirm that bulk_{in,out}_complete() work on different
> > usb_request structures, and they can not, at any time, get called on the
> > _same_ request?
> 
> usb_requests are allocated for a specific endpoint and USB Device
> Controller (UDC) drivers refuse to queue requests allocated for epX to
> epY, so this really can never happen.

Good, thanks!

> My fear now, however, is that changing smp_[rw]mb() to smp_mb() just
> adds extra overhead which makes the problem much, much less likely to
> happen. Does that sound plausible to you?

I did consider that, but I've not sufficiently grokked the code to rule
out actual fail. So let me stare at this a bit more.

Re: Memory barrier needed with wake_up_process()?

[Peter Zijlstra]

On Tue, Sep 06, 2016 at 01:49:37PM +0200, Peter Zijlstra wrote:
> On Tue, Sep 06, 2016 at 02:43:39PM +0300, Felipe Balbi wrote:

> > My fear now, however, is that changing smp_[rw]mb() to smp_mb() just
> > adds extra overhead which makes the problem much, much less likely to
> > happen. Does that sound plausible to you?
> 
> I did consider that, but I've not sufficiently grokked the code to rule
> out actual fail. So let me stare at this a bit more.

OK, so I'm really not seeing it, we've got:

while (bh->state != FULL) {
        for (;;) {
                set_current_state(INTERRUPTIBLE); /* MB after */
                if (signal_pending(current))
                        return -EINTR;
                if (common->thread_wakeup_needed)
                        break;
                schedule(); /* MB */
        }
        __set_current_state(RUNNING);
        common->thread_wakeup_needed = 0;
        smp_rmb(); /* NOP */
}


VS.


spin_lock(&common->lock); /* MB */
bh->state = FULL;
smp_wmb(); /* NOP */
common->thread_wakeup_needed = 1;
wake_up_process(common->thread_task); /* MB before */
spin_unlock(&common->lock);



(the MB annotations specific to x86, not true in general)


If we observe thread_wakeup_needed, we must also observe bh->state.

And the sleep/wakeup ordering is also correct, we either see
thread_wakeup_needed and continue, or we see task->state == RUNNING
(from the wakeup) and NO-OP schedule(). The MB from set_current_statE()
then matches with the MB from wake_up_process() to ensure we must see
thead_wakeup_needed.

Or, we go sleep, and get woken up, at which point the same happens.
Since the waking CPU gets the task back on its RQ the happens-before
chain includes the waking CPUs state along with the state of the task
itself before it went to sleep.

At which point we're back where we started, once we see
thread_wakeup_needed we must then also see bh->state (and all state
prior to that on the waking CPU).



There's enough cruft in the while-sleep loop to force reload bh->state.

Load/store tearing cannot be a problem because all values are single
bytes (the variables are multi bytes, but all values used only affect
the LSB).

Colour me puzzled.

Re: Memory barrier needed with wake_up_process()?

[Alan Stern]

On Tue, 6 Sep 2016, Peter Zijlstra wrote:

> On Tue, Sep 06, 2016 at 01:49:37PM +0200, Peter Zijlstra wrote:
> > On Tue, Sep 06, 2016 at 02:43:39PM +0300, Felipe Balbi wrote:
> 
> > > My fear now, however, is that changing smp_[rw]mb() to smp_mb() just
> > > adds extra overhead which makes the problem much, much less likely to
> > > happen. Does that sound plausible to you?
> > 
> > I did consider that, but I've not sufficiently grokked the code to rule
> > out actual fail. So let me stare at this a bit more.
> 
> OK, so I'm really not seeing it, we've got:
> 
> while (bh->state != FULL) {
>         for (;;) {
>                 set_current_state(INTERRUPTIBLE); /* MB after */
>                 if (signal_pending(current))
>                         return -EINTR;
>                 if (common->thread_wakeup_needed)
>                         break;
>                 schedule(); /* MB */
>         }
>         __set_current_state(RUNNING);
>         common->thread_wakeup_needed = 0;
>         smp_rmb(); /* NOP */
> }
> 
> 
> VS.
> 
> 
> spin_lock(&common->lock); /* MB */
> bh->state = FULL;
> smp_wmb(); /* NOP */
> common->thread_wakeup_needed = 1;
> wake_up_process(common->thread_task); /* MB before */
> spin_unlock(&common->lock);
> 
> 
> 
> (the MB annotations specific to x86, not true in general)
> 
> 
> If we observe thread_wakeup_needed, we must also observe bh->state.
> 
> And the sleep/wakeup ordering is also correct, we either see
> thread_wakeup_needed and continue, or we see task->state == RUNNING
> (from the wakeup) and NO-OP schedule(). The MB from set_current_statE()
> then matches with the MB from wake_up_process() to ensure we must see
> thead_wakeup_needed.
> 
> Or, we go sleep, and get woken up, at which point the same happens.
> Since the waking CPU gets the task back on its RQ the happens-before
> chain includes the waking CPUs state along with the state of the task
> itself before it went to sleep.
> 
> At which point we're back where we started, once we see
> thread_wakeup_needed we must then also see bh->state (and all state
> prior to that on the waking CPU).
> 
> 
> 
> There's enough cruft in the while-sleep loop to force reload bh->state.
> 
> Load/store tearing cannot be a problem because all values are single
> bytes (the variables are multi bytes, but all values used only affect
> the LSB).
> 
> Colour me puzzled.

Felipe, can you please try this patch on an unmodified tree?  If the 
problem still occurs, what shows up in the kernel log?

Alan Stern



Index: usb-4.x/drivers/usb/gadget/function/f_mass_storage.c
===================================================================
--- usb-4.x.orig/drivers/usb/gadget/function/f_mass_storage.c
+++ usb-4.x/drivers/usb/gadget/function/f_mass_storage.c
@@ -485,6 +485,8 @@ static void bulk_out_complete(struct usb
 	spin_lock(&common->lock);
 	bh->outreq_busy = 0;
 	bh->state = BUF_STATE_FULL;
+	if (bh->bulk_out_intended_length == US_BULK_CB_WRAP_LEN)
+		INFO(common, "compl: bh %p state %d\n", bh, bh->state);
 	wakeup_thread(common);
 	spin_unlock(&common->lock);
 }
@@ -2207,6 +2209,7 @@ static int get_next_command(struct fsg_c
 		rc = sleep_thread(common, true);
 		if (rc)
 			return rc;
+		INFO(common, "next: bh %p state %d\n", bh, bh->state);
 	}
 	smp_rmb();
 	rc = fsg_is_set(common) ? received_cbw(common->fsg, bh) : -EIO;

Re: Memory barrier needed with wake_up_process()?

[Peter Zijlstra]

On Tue, Sep 06, 2016 at 10:46:55AM -0400, Alan Stern wrote:

Not knowing where INFO() goes, you should use trace_printk() not
printk(), as the former is strictly per cpu, while the latter is
globally serialized and can hide all these problems.

> Index: usb-4.x/drivers/usb/gadget/function/f_mass_storage.c
> ===================================================================
> --- usb-4.x.orig/drivers/usb/gadget/function/f_mass_storage.c
> +++ usb-4.x/drivers/usb/gadget/function/f_mass_storage.c
> @@ -485,6 +485,8 @@ static void bulk_out_complete(struct usb
>  	spin_lock(&common->lock);
>  	bh->outreq_busy = 0;
>  	bh->state = BUF_STATE_FULL;
> +	if (bh->bulk_out_intended_length == US_BULK_CB_WRAP_LEN)
> +		INFO(common, "compl: bh %p state %d\n", bh, bh->state);
>  	wakeup_thread(common);
>  	spin_unlock(&common->lock);
>  }
> @@ -2207,6 +2209,7 @@ static int get_next_command(struct fsg_c
>  		rc = sleep_thread(common, true);
>  		if (rc)
>  			return rc;
> +		INFO(common, "next: bh %p state %d\n", bh, bh->state);
>  	}
>  	smp_rmb();
>  	rc = fsg_is_set(common) ? received_cbw(common->fsg, bh) : -EIO;
> 

Re: Memory barrier needed with wake_up_process()?

[Felipe Balbi]

[-- Attachment #1: Type: text/plain, Size: 4809 bytes --]


Hi,

Alan Stern <stern@rowland.harvard.edu> writes:
> On Tue, 6 Sep 2016, Peter Zijlstra wrote:
>
>> On Tue, Sep 06, 2016 at 01:49:37PM +0200, Peter Zijlstra wrote:
>> > On Tue, Sep 06, 2016 at 02:43:39PM +0300, Felipe Balbi wrote:
>> 
>> > > My fear now, however, is that changing smp_[rw]mb() to smp_mb() just
>> > > adds extra overhead which makes the problem much, much less likely to
>> > > happen. Does that sound plausible to you?
>> > 
>> > I did consider that, but I've not sufficiently grokked the code to rule
>> > out actual fail. So let me stare at this a bit more.
>> 
>> OK, so I'm really not seeing it, we've got:
>> 
>> while (bh->state != FULL) {
>>         for (;;) {
>>                 set_current_state(INTERRUPTIBLE); /* MB after */
>>                 if (signal_pending(current))
>>                         return -EINTR;
>>                 if (common->thread_wakeup_needed)
>>                         break;
>>                 schedule(); /* MB */
>>         }
>>         __set_current_state(RUNNING);
>>         common->thread_wakeup_needed = 0;
>>         smp_rmb(); /* NOP */
>> }
>> 
>> 
>> VS.
>> 
>> 
>> spin_lock(&common->lock); /* MB */
>> bh->state = FULL;
>> smp_wmb(); /* NOP */
>> common->thread_wakeup_needed = 1;
>> wake_up_process(common->thread_task); /* MB before */
>> spin_unlock(&common->lock);
>> 
>> 
>> 
>> (the MB annotations specific to x86, not true in general)
>> 
>> 
>> If we observe thread_wakeup_needed, we must also observe bh->state.
>> 
>> And the sleep/wakeup ordering is also correct, we either see
>> thread_wakeup_needed and continue, or we see task->state == RUNNING
>> (from the wakeup) and NO-OP schedule(). The MB from set_current_statE()
>> then matches with the MB from wake_up_process() to ensure we must see
>> thead_wakeup_needed.
>> 
>> Or, we go sleep, and get woken up, at which point the same happens.
>> Since the waking CPU gets the task back on its RQ the happens-before
>> chain includes the waking CPUs state along with the state of the task
>> itself before it went to sleep.
>> 
>> At which point we're back where we started, once we see
>> thread_wakeup_needed we must then also see bh->state (and all state
>> prior to that on the waking CPU).
>> 
>> 
>> 
>> There's enough cruft in the while-sleep loop to force reload bh->state.
>> 
>> Load/store tearing cannot be a problem because all values are single
>> bytes (the variables are multi bytes, but all values used only affect
>> the LSB).
>> 
>> Colour me puzzled.
>
> Felipe, can you please try this patch on an unmodified tree?  If the 
> problem still occurs, what shows up in the kernel log?
>
> Alan Stern
>
>
>
> Index: usb-4.x/drivers/usb/gadget/function/f_mass_storage.c
> ===================================================================
> --- usb-4.x.orig/drivers/usb/gadget/function/f_mass_storage.c
> +++ usb-4.x/drivers/usb/gadget/function/f_mass_storage.c
> @@ -485,6 +485,8 @@ static void bulk_out_complete(struct usb
>  	spin_lock(&common->lock);
>  	bh->outreq_busy = 0;
>  	bh->state = BUF_STATE_FULL;
> +	if (bh->bulk_out_intended_length == US_BULK_CB_WRAP_LEN)
> +		INFO(common, "compl: bh %p state %d\n", bh, bh->state);
>  	wakeup_thread(common);
>  	spin_unlock(&common->lock);
>  }
> @@ -2207,6 +2209,7 @@ static int get_next_command(struct fsg_c
>  		rc = sleep_thread(common, true);
>  		if (rc)
>  			return rc;
> +		INFO(common, "next: bh %p state %d\n", bh, bh->state);
>  	}
>  	smp_rmb();
>  	rc = fsg_is_set(common) ? received_cbw(common->fsg, bh) : -EIO;

I've replace INFO() with trace_printk() (which is what I have been using
anyway):

diff --git a/drivers/usb/gadget/function/f_mass_storage.c b/drivers/usb/gadget/function/f_mass_storage.c
index 2505117e88e8..dbc6a380b38b 100644
--- a/drivers/usb/gadget/function/f_mass_storage.c
+++ b/drivers/usb/gadget/function/f_mass_storage.c
@@ -485,6 +485,8 @@ static void bulk_out_complete(struct usb_ep *ep, struct usb_request *req)
 	spin_lock(&common->lock);
 	bh->outreq_busy = 0;
 	bh->state = BUF_STATE_FULL;
+	if (bh->bulk_out_intended_length == US_BULK_CB_WRAP_LEN)
+		trace_printk("compl: bh %p state %d\n", bh, bh->state);
 	wakeup_thread(common);
 	spin_unlock(&common->lock);
 }
@@ -2207,6 +2209,7 @@ static int get_next_command(struct fsg_common *common)
 		rc = sleep_thread(common, true);
 		if (rc)
 			return rc;
+		trace_printk("next: bh %p state %d\n", bh, bh->state);
 	}
 	smp_rmb();
 	rc = fsg_is_set(common) ? received_cbw(common->fsg, bh) : -EIO;

But I can't reproduce as reliably as before. I'll keep the thing running
an infinite loop which will stop only when interrupts in UDC (dwc3 in
this case) stop increasing.

-- 
balbi

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 800 bytes --]

Re: Memory barrier needed with wake_up_process()?

[Felipe Balbi]

[-- Attachment #1: Type: text/plain, Size: 107727 bytes --]


Hi,

Felipe Balbi <felipe.balbi@linux.intel.com> writes:
> Alan Stern <stern@rowland.harvard.edu> writes:
>> On Tue, 6 Sep 2016, Peter Zijlstra wrote:
>>
>>> On Tue, Sep 06, 2016 at 01:49:37PM +0200, Peter Zijlstra wrote:
>>> > On Tue, Sep 06, 2016 at 02:43:39PM +0300, Felipe Balbi wrote:
>>> 
>>> > > My fear now, however, is that changing smp_[rw]mb() to smp_mb() just
>>> > > adds extra overhead which makes the problem much, much less likely to
>>> > > happen. Does that sound plausible to you?
>>> > 
>>> > I did consider that, but I've not sufficiently grokked the code to rule
>>> > out actual fail. So let me stare at this a bit more.
>>> 
>>> OK, so I'm really not seeing it, we've got:
>>> 
>>> while (bh->state != FULL) {
>>>         for (;;) {
>>>                 set_current_state(INTERRUPTIBLE); /* MB after */
>>>                 if (signal_pending(current))
>>>                         return -EINTR;
>>>                 if (common->thread_wakeup_needed)
>>>                         break;
>>>                 schedule(); /* MB */
>>>         }
>>>         __set_current_state(RUNNING);
>>>         common->thread_wakeup_needed = 0;
>>>         smp_rmb(); /* NOP */
>>> }
>>> 
>>> 
>>> VS.
>>> 
>>> 
>>> spin_lock(&common->lock); /* MB */
>>> bh->state = FULL;
>>> smp_wmb(); /* NOP */
>>> common->thread_wakeup_needed = 1;
>>> wake_up_process(common->thread_task); /* MB before */
>>> spin_unlock(&common->lock);
>>> 
>>> 
>>> 
>>> (the MB annotations specific to x86, not true in general)
>>> 
>>> 
>>> If we observe thread_wakeup_needed, we must also observe bh->state.
>>> 
>>> And the sleep/wakeup ordering is also correct, we either see
>>> thread_wakeup_needed and continue, or we see task->state == RUNNING
>>> (from the wakeup) and NO-OP schedule(). The MB from set_current_statE()
>>> then matches with the MB from wake_up_process() to ensure we must see
>>> thead_wakeup_needed.
>>> 
>>> Or, we go sleep, and get woken up, at which point the same happens.
>>> Since the waking CPU gets the task back on its RQ the happens-before
>>> chain includes the waking CPUs state along with the state of the task
>>> itself before it went to sleep.
>>> 
>>> At which point we're back where we started, once we see
>>> thread_wakeup_needed we must then also see bh->state (and all state
>>> prior to that on the waking CPU).
>>> 
>>> 
>>> 
>>> There's enough cruft in the while-sleep loop to force reload bh->state.
>>> 
>>> Load/store tearing cannot be a problem because all values are single
>>> bytes (the variables are multi bytes, but all values used only affect
>>> the LSB).
>>> 
>>> Colour me puzzled.
>>
>> Felipe, can you please try this patch on an unmodified tree?  If the 
>> problem still occurs, what shows up in the kernel log?
>>
>> Alan Stern
>>
>>
>>
>> Index: usb-4.x/drivers/usb/gadget/function/f_mass_storage.c
>> ===================================================================
>> --- usb-4.x.orig/drivers/usb/gadget/function/f_mass_storage.c
>> +++ usb-4.x/drivers/usb/gadget/function/f_mass_storage.c
>> @@ -485,6 +485,8 @@ static void bulk_out_complete(struct usb
>>  	spin_lock(&common->lock);
>>  	bh->outreq_busy = 0;
>>  	bh->state = BUF_STATE_FULL;
>> +	if (bh->bulk_out_intended_length == US_BULK_CB_WRAP_LEN)
>> +		INFO(common, "compl: bh %p state %d\n", bh, bh->state);
>>  	wakeup_thread(common);
>>  	spin_unlock(&common->lock);
>>  }
>> @@ -2207,6 +2209,7 @@ static int get_next_command(struct fsg_c
>>  		rc = sleep_thread(common, true);
>>  		if (rc)
>>  			return rc;
>> +		INFO(common, "next: bh %p state %d\n", bh, bh->state);
>>  	}
>>  	smp_rmb();
>>  	rc = fsg_is_set(common) ? received_cbw(common->fsg, bh) : -EIO;
>
> I've replace INFO() with trace_printk() (which is what I have been using
> anyway):
>
> diff --git a/drivers/usb/gadget/function/f_mass_storage.c b/drivers/usb/gadget/function/f_mass_storage.c
> index 2505117e88e8..dbc6a380b38b 100644
> --- a/drivers/usb/gadget/function/f_mass_storage.c
> +++ b/drivers/usb/gadget/function/f_mass_storage.c
> @@ -485,6 +485,8 @@ static void bulk_out_complete(struct usb_ep *ep, struct usb_request *req)
>  	spin_lock(&common->lock);
>  	bh->outreq_busy = 0;
>  	bh->state = BUF_STATE_FULL;
> +	if (bh->bulk_out_intended_length == US_BULK_CB_WRAP_LEN)
> +		trace_printk("compl: bh %p state %d\n", bh, bh->state);
>  	wakeup_thread(common);
>  	spin_unlock(&common->lock);
>  }
> @@ -2207,6 +2209,7 @@ static int get_next_command(struct fsg_common *common)
>  		rc = sleep_thread(common, true);
>  		if (rc)
>  			return rc;
> +		trace_printk("next: bh %p state %d\n", bh, bh->state);
>  	}
>  	smp_rmb();
>  	rc = fsg_is_set(common) ? received_cbw(common->fsg, bh) : -EIO;
>
> But I can't reproduce as reliably as before. I'll keep the thing running
> an infinite loop which will stop only when interrupts in UDC (dwc3 in
> this case) stop increasing.

Finally :-) Here's the diff I used:

diff --git a/drivers/usb/gadget/function/f_mass_storage.c b/drivers/usb/gadget/function/f_mass_storage.c
index 8f3659b65f53..0716024f6b65 100644
--- a/drivers/usb/gadget/function/f_mass_storage.c
+++ b/drivers/usb/gadget/function/f_mass_storage.c
@@ -481,6 +481,8 @@ static void bulk_out_complete(struct usb_ep *ep, struct usb_request *req)
        spin_lock(&common->lock);
        bh->outreq_busy = 0;
        bh->state = BUF_STATE_FULL;
+       if (bh->bulk_out_intended_length == US_BULK_CB_WRAP_LEN)
+               trace_printk("compl: bh %p state %d\n", bh, bh->state);
        wakeup_thread(common);
        spin_unlock(&common->lock);
 }
@@ -2208,6 +2210,7 @@ static int get_next_command(struct fsg_common *common)
                rc = sleep_thread(common, true);
                if (rc)
                        return rc;
+               trace_printk("next: bh %p state %d\n", bh, bh->state);
        }
        smp_rmb();
        rc = fsg_is_set(common) ? received_cbw(common->fsg, bh) : -EIO;


And here's trace output:

# tracer: nop
#
# entries-in-buffer/entries-written: 1002/1002   #P:4
#
#                              _-----=> irqs-off
#                             / _----=> need-resched
#                            | / _---=> hardirq/softirq
#                            || / _--=> preempt-depth
#                            ||| /     delay
#           TASK-PID   CPU#  ||||    TIMESTAMP  FUNCTION
#              | |       |   ||||       |         |
    file-storage-3578  [000] .... 21166.789127: fsg_main_thread: next: bh ffff880111e69a00 state 2
    file-storage-3578  [000] .... 21166.789312: fsg_main_thread: next: bh ffff880111e69a00 state 2
     irq/17-dwc3-3579  [003] d..1 21166.789395: bulk_out_complete: compl: bh ffff880111e69a00 state 1
    file-storage-3578  [000] .... 21166.789445: fsg_main_thread: next: bh ffff880111e69a00 state 1
    file-storage-3578  [000] .... 21166.893131: fsg_main_thread: next: bh ffff880111e69a80 state 2
    file-storage-3578  [000] .... 21166.893523: fsg_main_thread: next: bh ffff880111e69a80 state 2
     irq/17-dwc3-3579  [003] d..1 21166.893681: bulk_out_complete: compl: bh ffff880111e69a80 state 1
    file-storage-3578  [000] .... 21166.893733: fsg_main_thread: next: bh ffff880111e69a80 state 1
    file-storage-3578  [000] .... 21166.893937: fsg_main_thread: next: bh ffff880111e69b00 state 2
    file-storage-3578  [000] .... 21166.893967: fsg_main_thread: next: bh ffff880111e69b00 state 2
     irq/17-dwc3-3579  [003] d..1 21166.894026: bulk_out_complete: compl: bh ffff880111e69b00 state 1
    file-storage-3578  [000] .... 21166.894035: fsg_main_thread: next: bh ffff880111e69b00 state 1
    file-storage-3578  [000] .... 21166.894166: fsg_main_thread: next: bh ffff880111e69b80 state 2
    file-storage-3578  [000] .... 21166.894194: fsg_main_thread: next: bh ffff880111e69b80 state 2
     irq/17-dwc3-3579  [003] d..1 21166.896360: bulk_out_complete: compl: bh ffff880111e69b80 state 1
    file-storage-3578  [000] .... 21166.896366: fsg_main_thread: next: bh ffff880111e69b80 state 1
    file-storage-3578  [000] .... 21166.896445: fsg_main_thread: next: bh ffff880111e69c00 state 2
    file-storage-3578  [000] .... 21166.896455: fsg_main_thread: next: bh ffff880111e69c00 state 2
     irq/17-dwc3-3579  [003] d..1 21166.896532: bulk_out_complete: compl: bh ffff880111e69c00 state 1
    file-storage-3578  [000] .... 21166.896535: fsg_main_thread: next: bh ffff880111e69c00 state 1
    file-storage-3578  [002] .... 21167.005103: fsg_main_thread: next: bh ffff880111e69c80 state 2
    file-storage-3578  [002] .... 21167.005283: fsg_main_thread: next: bh ffff880111e69c80 state 2
     irq/17-dwc3-3579  [003] d..1 21167.005342: bulk_out_complete: compl: bh ffff880111e69c80 state 1
    file-storage-3578  [002] .... 21167.005346: fsg_main_thread: next: bh ffff880111e69c80 state 1
    file-storage-3578  [002] .... 21167.005422: fsg_main_thread: next: bh ffff880111e69d00 state 2
     irq/17-dwc3-3579  [003] d..1 21167.005547: bulk_out_complete: compl: bh ffff880111e69d00 state 1
    file-storage-3578  [003] .... 21167.006087: fsg_main_thread: next: bh ffff880111e69d00 state 1
    file-storage-3578  [003] .... 21167.010189: fsg_main_thread: next: bh ffff880111e69d40 state 2
     irq/17-dwc3-3579  [003] d..1 21167.010253: bulk_out_complete: compl: bh ffff880111e69d40 state 1
    file-storage-3578  [003] .... 21167.010262: fsg_main_thread: next: bh ffff880111e69d40 state 1
    file-storage-3578  [003] .... 21167.012669: fsg_main_thread: next: bh ffff880111e6ad80 state 2
     irq/17-dwc3-3579  [003] d..1 21167.012942: bulk_out_complete: compl: bh ffff880111e6ad80 state 1
    file-storage-3578  [003] .... 21167.012951: fsg_main_thread: next: bh ffff880111e6ad80 state 1
    file-storage-3578  [003] .... 21167.015329: fsg_main_thread: next: bh ffff880111e6bdc0 state 2
     irq/17-dwc3-3579  [003] d..1 21167.015606: bulk_out_complete: compl: bh ffff880111e6bdc0 state 1
    file-storage-3578  [003] .... 21167.015616: fsg_main_thread: next: bh ffff880111e6bdc0 state 1
    file-storage-3578  [003] .... 21167.015716: fsg_main_thread: next: bh ffff880111e6be40 state 2
     irq/17-dwc3-3579  [003] d..1 21167.015807: bulk_out_complete: compl: bh ffff880111e6be40 state 1
    file-storage-3578  [003] .... 21167.015810: fsg_main_thread: next: bh ffff880111e6be40 state 1
    file-storage-3578  [003] .... 21167.015905: fsg_main_thread: next: bh ffff880111e6bec0 state 2
     irq/17-dwc3-3579  [003] d..1 21167.015986: bulk_out_complete: compl: bh ffff880111e6bec0 state 1
    file-storage-3578  [003] .... 21167.015990: fsg_main_thread: next: bh ffff880111e6bec0 state 1
    file-storage-3578  [003] .... 21167.016078: fsg_main_thread: next: bh ffff880111e6bf40 state 2
     irq/17-dwc3-3579  [003] d..1 21167.016149: bulk_out_complete: compl: bh ffff880111e6bf40 state 1
    file-storage-3578  [003] .... 21167.016159: fsg_main_thread: next: bh ffff880111e6bf40 state 1
    file-storage-3578  [003] .... 21167.016243: fsg_main_thread: next: bh ffff880111e68000 state 2
     irq/17-dwc3-3579  [003] d..1 21167.016301: bulk_out_complete: compl: bh ffff880111e68000 state 1
    file-storage-3578  [003] .... 21167.016305: fsg_main_thread: next: bh ffff880111e68000 state 1
    file-storage-3578  [003] .... 21167.016388: fsg_main_thread: next: bh ffff880111e68080 state 2
     irq/17-dwc3-3579  [003] d..1 21167.016446: bulk_out_complete: compl: bh ffff880111e68080 state 1
    file-storage-3578  [003] .... 21167.016450: fsg_main_thread: next: bh ffff880111e68080 state 1
    file-storage-3578  [003] .... 21167.016533: fsg_main_thread: next: bh ffff880111e68100 state 2
     irq/17-dwc3-3579  [003] d..1 21167.016591: bulk_out_complete: compl: bh ffff880111e68100 state 1
    file-storage-3578  [003] .... 21167.016595: fsg_main_thread: next: bh ffff880111e68100 state 1
    file-storage-3578  [003] .... 21167.016672: fsg_main_thread: next: bh ffff880111e68180 state 2
     irq/17-dwc3-3579  [003] d..1 21167.016729: bulk_out_complete: compl: bh ffff880111e68180 state 1
    file-storage-3578  [003] .... 21167.016733: fsg_main_thread: next: bh ffff880111e68180 state 1
    file-storage-3578  [003] .... 21167.016811: fsg_main_thread: next: bh ffff880111e68200 state 2
     irq/17-dwc3-3579  [003] d..1 21167.016869: bulk_out_complete: compl: bh ffff880111e68200 state 1
    file-storage-3578  [003] .... 21167.016873: fsg_main_thread: next: bh ffff880111e68200 state 1
    file-storage-3578  [003] .... 21167.016950: fsg_main_thread: next: bh ffff880111e68280 state 2
     irq/17-dwc3-3579  [003] d..1 21167.017008: bulk_out_complete: compl: bh ffff880111e68280 state 1
    file-storage-3578  [003] .... 21167.017012: fsg_main_thread: next: bh ffff880111e68280 state 1
    file-storage-3578  [003] .... 21167.017089: fsg_main_thread: next: bh ffff880111e68300 state 2
     irq/17-dwc3-3579  [003] d..1 21167.017179: bulk_out_complete: compl: bh ffff880111e68300 state 1
    file-storage-3578  [003] .... 21167.017183: fsg_main_thread: next: bh ffff880111e68300 state 1
    file-storage-3578  [003] .... 21167.017277: fsg_main_thread: next: bh ffff880111e68380 state 2
     irq/17-dwc3-3579  [003] d..1 21167.017373: bulk_out_complete: compl: bh ffff880111e68380 state 1
    file-storage-3578  [003] .... 21167.017376: fsg_main_thread: next: bh ffff880111e68380 state 1
    file-storage-3578  [003] .... 21167.017473: fsg_main_thread: next: bh ffff880111e68400 state 2
     irq/17-dwc3-3579  [003] d..1 21167.017548: bulk_out_complete: compl: bh ffff880111e68400 state 1
    file-storage-3578  [003] .... 21167.017552: fsg_main_thread: next: bh ffff880111e68400 state 1
    file-storage-3578  [003] .... 21167.017646: fsg_main_thread: next: bh ffff880111e68480 state 2
     irq/17-dwc3-3579  [003] d..1 21167.017728: bulk_out_complete: compl: bh ffff880111e68480 state 1
    file-storage-3578  [003] .... 21167.017732: fsg_main_thread: next: bh ffff880111e68480 state 1
    file-storage-3578  [003] .... 21167.017828: fsg_main_thread: next: bh ffff880111e68500 state 2
     irq/17-dwc3-3579  [003] d..1 21167.017904: bulk_out_complete: compl: bh ffff880111e68500 state 1
    file-storage-3578  [003] .... 21167.017908: fsg_main_thread: next: bh ffff880111e68500 state 1
    file-storage-3578  [003] .... 21167.018002: fsg_main_thread: next: bh ffff880111e68580 state 2
     irq/17-dwc3-3579  [003] d..1 21167.018078: bulk_out_complete: compl: bh ffff880111e68580 state 1
    file-storage-3578  [003] .... 21167.018087: fsg_main_thread: next: bh ffff880111e68580 state 1
    file-storage-3578  [003] .... 21167.018182: fsg_main_thread: next: bh ffff880111e68600 state 2
     irq/17-dwc3-3579  [003] d..1 21167.018261: bulk_out_complete: compl: bh ffff880111e68600 state 1
    file-storage-3578  [003] .... 21167.018264: fsg_main_thread: next: bh ffff880111e68600 state 1
    file-storage-3578  [003] .... 21167.018358: fsg_main_thread: next: bh ffff880111e68680 state 2
     irq/17-dwc3-3579  [003] d..1 21167.018435: bulk_out_complete: compl: bh ffff880111e68680 state 1
    file-storage-3578  [003] .... 21167.018439: fsg_main_thread: next: bh ffff880111e68680 state 1
    file-storage-3578  [003] .... 21167.018533: fsg_main_thread: next: bh ffff880111e68700 state 2
     irq/17-dwc3-3579  [003] d..1 21167.018609: bulk_out_complete: compl: bh ffff880111e68700 state 1
    file-storage-3578  [003] .... 21167.018613: fsg_main_thread: next: bh ffff880111e68700 state 1
    file-storage-3578  [003] .... 21167.018708: fsg_main_thread: next: bh ffff880111e68780 state 2
     irq/17-dwc3-3579  [003] d..1 21167.018782: bulk_out_complete: compl: bh ffff880111e68780 state 1
    file-storage-3578  [003] .... 21167.018785: fsg_main_thread: next: bh ffff880111e68780 state 1
    file-storage-3578  [003] .... 21167.018879: fsg_main_thread: next: bh ffff880111e68800 state 2
     irq/17-dwc3-3579  [003] d..1 21167.018955: bulk_out_complete: compl: bh ffff880111e68800 state 1
    file-storage-3578  [003] .... 21167.018959: fsg_main_thread: next: bh ffff880111e68800 state 1
    file-storage-3578  [003] .... 21167.019052: fsg_main_thread: next: bh ffff880111e68880 state 2
     irq/17-dwc3-3579  [003] d..1 21167.019128: bulk_out_complete: compl: bh ffff880111e68880 state 1
    file-storage-3578  [003] .... 21167.019138: fsg_main_thread: next: bh ffff880111e68880 state 1
    file-storage-3578  [003] .... 21167.019234: fsg_main_thread: next: bh ffff880111e68900 state 2
     irq/17-dwc3-3579  [003] d..1 21167.020016: bulk_out_complete: compl: bh ffff880111e68900 state 1
    file-storage-3578  [003] .... 21167.020025: fsg_main_thread: next: bh ffff880111e68900 state 1
    file-storage-3578  [003] .... 21167.020166: fsg_main_thread: next: bh ffff880111e68980 state 2
     irq/17-dwc3-3579  [003] d..1 21167.020230: bulk_out_complete: compl: bh ffff880111e68980 state 1
    file-storage-3578  [003] .... 21167.020240: fsg_main_thread: next: bh ffff880111e68980 state 1
    file-storage-3578  [003] .... 21167.020335: fsg_main_thread: next: bh ffff880111e68a00 state 2
     irq/17-dwc3-3579  [003] d..1 21167.020396: bulk_out_complete: compl: bh ffff880111e68a00 state 1
    file-storage-3578  [003] .... 21167.020400: fsg_main_thread: next: bh ffff880111e68a00 state 1
    file-storage-3578  [003] .... 21167.020510: fsg_main_thread: next: bh ffff880111e68a80 state 2
     irq/17-dwc3-3579  [003] d..1 21167.020571: bulk_out_complete: compl: bh ffff880111e68a80 state 1
    file-storage-3578  [003] .... 21167.020575: fsg_main_thread: next: bh ffff880111e68a80 state 1
    file-storage-3578  [003] .... 21167.020711: fsg_main_thread: next: bh ffff880111e68b00 state 2
     irq/17-dwc3-3579  [003] d..1 21167.020770: bulk_out_complete: compl: bh ffff880111e68b00 state 1
    file-storage-3578  [003] .... 21167.020774: fsg_main_thread: next: bh ffff880111e68b00 state 1
    file-storage-3578  [003] .... 21167.020912: fsg_main_thread: next: bh ffff880111e68c80 state 2
    file-storage-3578  [003] .... 21167.021015: fsg_main_thread: next: bh ffff880111e68c80 state 2
    file-storage-3578  [003] .... 21167.021076: fsg_main_thread: next: bh ffff880111e68c80 state 2
     irq/17-dwc3-3579  [003] d..1 21167.021214: bulk_out_complete: compl: bh ffff880111e68c80 state 1
    file-storage-3578  [003] .... 21167.021219: fsg_main_thread: next: bh ffff880111e68c80 state 1
    file-storage-3578  [003] .... 21167.021363: fsg_main_thread: next: bh ffff880111e68d00 state 2
     irq/17-dwc3-3579  [003] d..1 21167.021424: bulk_out_complete: compl: bh ffff880111e68d00 state 1
    file-storage-3578  [003] .... 21167.021434: fsg_main_thread: next: bh ffff880111e68d00 state 1
    file-storage-3578  [003] .... 21167.021570: fsg_main_thread: next: bh ffff880111e68dc0 state 2
     irq/17-dwc3-3579  [003] d..1 21167.021629: bulk_out_complete: compl: bh ffff880111e68dc0 state 1
    file-storage-3578  [003] .... 21167.021633: fsg_main_thread: next: bh ffff880111e68dc0 state 1
    file-storage-3578  [003] .... 21167.021767: fsg_main_thread: next: bh ffff880111e68ec0 state 2
    file-storage-3578  [003] .... 21167.021831: fsg_main_thread: next: bh ffff880111e68ec0 state 2
     irq/17-dwc3-3579  [003] d..1 21167.021892: bulk_out_complete: compl: bh ffff880111e68ec0 state 1
    file-storage-3578  [003] .... 21167.021896: fsg_main_thread: next: bh ffff880111e68ec0 state 1
    file-storage-3578  [003] .... 21167.022038: fsg_main_thread: next: bh ffff880111e68fc0 state 2
    file-storage-3578  [003] .... 21167.022102: fsg_main_thread: next: bh ffff880111e68fc0 state 2
     irq/17-dwc3-3579  [003] d..1 21167.022255: bulk_out_complete: compl: bh ffff880111e68fc0 state 1
    file-storage-3578  [003] .... 21167.022259: fsg_main_thread: next: bh ffff880111e68fc0 state 1
    file-storage-3578  [003] .... 21167.022400: fsg_main_thread: next: bh ffff880111e69100 state 2
    file-storage-3578  [003] .... 21167.022490: fsg_main_thread: next: bh ffff880111e69100 state 2
     irq/17-dwc3-3579  [003] d..1 21167.022549: bulk_out_complete: compl: bh ffff880111e69100 state 1
    file-storage-3578  [003] .... 21167.022552: fsg_main_thread: next: bh ffff880111e69100 state 1
    file-storage-3578  [003] .... 21167.022695: fsg_main_thread: next: bh ffff880111e69240 state 2
    file-storage-3578  [003] .... 21167.022788: fsg_main_thread: next: bh ffff880111e69240 state 2
    file-storage-3578  [003] .... 21167.022852: fsg_main_thread: next: bh ffff880111e69240 state 2
     irq/17-dwc3-3579  [003] d..1 21167.023005: bulk_out_complete: compl: bh ffff880111e69240 state 1
    file-storage-3578  [003] .... 21167.023012: fsg_main_thread: next: bh ffff880111e69240 state 1
    file-storage-3578  [003] .... 21167.023148: fsg_main_thread: next: bh ffff880111e69380 state 2
    file-storage-3578  [003] .... 21167.023239: fsg_main_thread: next: bh ffff880111e69380 state 2
     irq/17-dwc3-3579  [003] d..1 21167.023299: bulk_out_complete: compl: bh ffff880111e69380 state 1
    file-storage-3578  [003] .... 21167.023305: fsg_main_thread: next: bh ffff880111e69380 state 1
    file-storage-3578  [003] .... 21167.023450: fsg_main_thread: next: bh ffff880111e69440 state 2
     irq/17-dwc3-3579  [003] d..1 21167.023508: bulk_out_complete: compl: bh ffff880111e69440 state 1
    file-storage-3578  [003] .... 21167.023514: fsg_main_thread: next: bh ffff880111e69440 state 1
    file-storage-3578  [003] .... 21167.023665: fsg_main_thread: next: bh ffff880111e69500 state 2
     irq/17-dwc3-3579  [003] d..1 21167.023850: bulk_out_complete: compl: bh ffff880111e69500 state 1
    file-storage-3578  [003] .... 21167.023860: fsg_main_thread: next: bh ffff880111e69500 state 1
    file-storage-3578  [003] .... 21167.023939: fsg_main_thread: next: bh ffff880111e69540 state 2
     irq/17-dwc3-3579  [003] d..1 21167.024901: bulk_out_complete: compl: bh ffff880111e69540 state 1
    file-storage-3578  [003] .... 21167.025129: fsg_main_thread: next: bh ffff880111e69540 state 1
    file-storage-3578  [003] .... 21167.025257: fsg_main_thread: next: bh ffff880111e69580 state 2
     irq/17-dwc3-3579  [003] d..1 21167.025333: bulk_out_complete: compl: bh ffff880111e69580 state 1
    file-storage-3578  [000] .... 21167.025366: fsg_main_thread: next: bh ffff880111e69580 state 1
    file-storage-3578  [000] .... 21167.027653: fsg_main_thread: next: bh ffff880111e6a5c0 state 2
    file-storage-3578  [000] .... 21167.027717: fsg_main_thread: next: bh ffff880111e6a5c0 state 2
     irq/17-dwc3-3579  [003] d..1 21167.027926: bulk_out_complete: compl: bh ffff880111e6a5c0 state 1
    file-storage-3578  [000] .... 21167.027929: fsg_main_thread: next: bh ffff880111e6a5c0 state 1
    file-storage-3578  [002] .... 21167.030290: fsg_main_thread: next: bh ffff880111e6b600 state 2
     irq/17-dwc3-3579  [003] d..1 21167.030485: bulk_out_complete: compl: bh ffff880111e6b600 state 1
    file-storage-3578  [002] .... 21167.030488: fsg_main_thread: next: bh ffff880111e6b600 state 1
    file-storage-3578  [000] .... 21167.032852: fsg_main_thread: next: bh ffff880111e68680 state 2
     irq/17-dwc3-3579  [003] d..1 21167.033063: bulk_out_complete: compl: bh ffff880111e68680 state 1
    file-storage-3578  [000] .... 21167.033066: fsg_main_thread: next: bh ffff880111e68680 state 1
    file-storage-3578  [002] .... 21167.035429: fsg_main_thread: next: bh ffff880111e696c0 state 2
     irq/17-dwc3-3579  [003] d..1 21167.035629: bulk_out_complete: compl: bh ffff880111e696c0 state 1
    file-storage-3578  [002] .... 21167.035632: fsg_main_thread: next: bh ffff880111e696c0 state 1
    file-storage-3578  [002] .... 21167.037982: fsg_main_thread: next: bh ffff880111e6a700 state 2
     irq/17-dwc3-3579  [003] d..1 21167.038173: bulk_out_complete: compl: bh ffff880111e6a700 state 1
    file-storage-3578  [002] .... 21167.038177: fsg_main_thread: next: bh ffff880111e6a700 state 1
    file-storage-3578  [000] .... 21167.040538: fsg_main_thread: next: bh ffff880111e6b740 state 2
     irq/17-dwc3-3579  [003] d..1 21167.040738: bulk_out_complete: compl: bh ffff880111e6b740 state 1
    file-storage-3578  [000] .... 21167.040741: fsg_main_thread: next: bh ffff880111e6b740 state 1
    file-storage-3578  [000] .... 21167.043368: fsg_main_thread: next: bh ffff880111e687c0 state 2
     irq/17-dwc3-3579  [003] d..1 21167.043435: bulk_out_complete: compl: bh ffff880111e687c0 state 1
    file-storage-3578  [000] .... 21167.043439: fsg_main_thread: next: bh ffff880111e687c0 state 1
    file-storage-3578  [000] .... 21167.046046: fsg_main_thread: next: bh ffff880111e69800 state 2
     irq/17-dwc3-3579  [003] d..1 21167.046113: bulk_out_complete: compl: bh ffff880111e69800 state 1
    file-storage-3578  [000] .... 21167.046118: fsg_main_thread: next: bh ffff880111e69800 state 1
    file-storage-3578  [000] .... 21167.048720: fsg_main_thread: next: bh ffff880111e6a840 state 2
     irq/17-dwc3-3579  [003] d..1 21167.048787: bulk_out_complete: compl: bh ffff880111e6a840 state 1
    file-storage-3578  [000] .... 21167.048790: fsg_main_thread: next: bh ffff880111e6a840 state 1
    file-storage-3578  [000] .... 21167.051423: fsg_main_thread: next: bh ffff880111e6b880 state 2
     irq/17-dwc3-3579  [003] d..1 21167.051490: bulk_out_complete: compl: bh ffff880111e6b880 state 1
    file-storage-3578  [000] .... 21167.051493: fsg_main_thread: next: bh ffff880111e6b880 state 1
    file-storage-3578  [000] .... 21167.054116: fsg_main_thread: next: bh ffff880111e68900 state 2
     irq/17-dwc3-3579  [003] d..1 21167.054183: bulk_out_complete: compl: bh ffff880111e68900 state 1
    file-storage-3578  [000] .... 21167.054187: fsg_main_thread: next: bh ffff880111e68900 state 1
    file-storage-3578  [000] .... 21167.056808: fsg_main_thread: next: bh ffff880111e69940 state 2
     irq/17-dwc3-3579  [003] d..1 21167.056874: bulk_out_complete: compl: bh ffff880111e69940 state 1
    file-storage-3578  [000] .... 21167.056878: fsg_main_thread: next: bh ffff880111e69940 state 1
    file-storage-3578  [000] .... 21167.059493: fsg_main_thread: next: bh ffff880111e6a980 state 2
     irq/17-dwc3-3579  [003] d..1 21167.059560: bulk_out_complete: compl: bh ffff880111e6a980 state 1
    file-storage-3578  [000] .... 21167.059564: fsg_main_thread: next: bh ffff880111e6a980 state 1
    file-storage-3578  [002] .... 21167.061913: fsg_main_thread: next: bh ffff880111e6b9c0 state 2
     irq/17-dwc3-3579  [003] d..1 21167.062146: bulk_out_complete: compl: bh ffff880111e6b9c0 state 1
    file-storage-3578  [002] .... 21167.062150: fsg_main_thread: next: bh ffff880111e6b9c0 state 1
    file-storage-3578  [000] .... 21167.064505: fsg_main_thread: next: bh ffff880111e68a40 state 2
     irq/17-dwc3-3579  [003] d..1 21167.064711: bulk_out_complete: compl: bh ffff880111e68a40 state 1
    file-storage-3578  [000] .... 21167.064716: fsg_main_thread: next: bh ffff880111e68a40 state 1
    file-storage-3578  [000] .... 21167.066999: fsg_main_thread: next: bh ffff880111e69a80 state 2
    file-storage-3578  [002] .... 21167.067065: fsg_main_thread: next: bh ffff880111e69a80 state 2
     irq/17-dwc3-3579  [003] d..1 21167.067266: bulk_out_complete: compl: bh ffff880111e69a80 state 1
    file-storage-3578  [002] .... 21167.067269: fsg_main_thread: next: bh ffff880111e69a80 state 1
    file-storage-3578  [000] .... 21167.069635: fsg_main_thread: next: bh ffff880111e6aac0 state 2
     irq/17-dwc3-3579  [003] d..1 21167.069820: bulk_out_complete: compl: bh ffff880111e6aac0 state 1
    file-storage-3578  [000] .... 21167.069823: fsg_main_thread: next: bh ffff880111e6aac0 state 1
    file-storage-3578  [002] .... 21167.072173: fsg_main_thread: next: bh ffff880111e6bb00 state 2
     irq/17-dwc3-3579  [003] d..1 21167.072365: bulk_out_complete: compl: bh ffff880111e6bb00 state 1
    file-storage-3578  [002] .... 21167.072368: fsg_main_thread: next: bh ffff880111e6bb00 state 1
    file-storage-3578  [000] .... 21167.074736: fsg_main_thread: next: bh ffff880111e68b80 state 2
     irq/17-dwc3-3579  [003] d..1 21167.074932: bulk_out_complete: compl: bh ffff880111e68b80 state 1
    file-storage-3578  [000] .... 21167.074936: fsg_main_thread: next: bh ffff880111e68b80 state 1
    file-storage-3578  [000] .... 21167.077333: fsg_main_thread: next: bh ffff880111e69bc0 state 2
     irq/17-dwc3-3579  [003] d..1 21167.077539: bulk_out_complete: compl: bh ffff880111e69bc0 state 1
    file-storage-3578  [000] .... 21167.077543: fsg_main_thread: next: bh ffff880111e69bc0 state 1
    file-storage-3578  [000] .... 21167.080182: fsg_main_thread: next: bh ffff880111e6ac00 state 2
     irq/17-dwc3-3579  [003] d..1 21167.080250: bulk_out_complete: compl: bh ffff880111e6ac00 state 1
    file-storage-3578  [000] .... 21167.080255: fsg_main_thread: next: bh ffff880111e6ac00 state 1
    file-storage-3578  [000] .... 21167.082860: fsg_main_thread: next: bh ffff880111e6bc40 state 2
     irq/17-dwc3-3579  [003] d..1 21167.082926: bulk_out_complete: compl: bh ffff880111e6bc40 state 1
    file-storage-3578  [000] .... 21167.082930: fsg_main_thread: next: bh ffff880111e6bc40 state 1
    file-storage-3578  [000] .... 21167.085562: fsg_main_thread: next: bh ffff880111e68cc0 state 2
     irq/17-dwc3-3579  [003] d..1 21167.085629: bulk_out_complete: compl: bh ffff880111e68cc0 state 1
    file-storage-3578  [000] .... 21167.085632: fsg_main_thread: next: bh ffff880111e68cc0 state 1
    file-storage-3578  [000] .... 21167.088282: fsg_main_thread: next: bh ffff880111e69d00 state 2
     irq/17-dwc3-3579  [003] d..1 21167.088346: bulk_out_complete: compl: bh ffff880111e69d00 state 1
    file-storage-3578  [000] .... 21167.088350: fsg_main_thread: next: bh ffff880111e69d00 state 1
    file-storage-3578  [000] .... 21167.090988: fsg_main_thread: next: bh ffff880111e6ad40 state 2
     irq/17-dwc3-3579  [003] d..1 21167.091056: bulk_out_complete: compl: bh ffff880111e6ad40 state 1
    file-storage-3578  [000] .... 21167.091060: fsg_main_thread: next: bh ffff880111e6ad40 state 1
    file-storage-3578  [000] .... 21167.093680: fsg_main_thread: next: bh ffff880111e6bd80 state 2
     irq/17-dwc3-3579  [003] d..1 21167.093748: bulk_out_complete: compl: bh ffff880111e6bd80 state 1
    file-storage-3578  [000] .... 21167.093752: fsg_main_thread: next: bh ffff880111e6bd80 state 1
    file-storage-3578  [002] .... 21167.096100: fsg_main_thread: next: bh ffff880111e68e00 state 2
     irq/17-dwc3-3579  [003] d..1 21167.096324: bulk_out_complete: compl: bh ffff880111e68e00 state 1
    file-storage-3578  [002] .... 21167.096327: fsg_main_thread: next: bh ffff880111e68e00 state 1
    file-storage-3578  [000] .... 21167.098682: fsg_main_thread: next: bh ffff880111e69e40 state 2
     irq/17-dwc3-3579  [003] d..1 21167.098885: bulk_out_complete: compl: bh ffff880111e69e40 state 1
    file-storage-3578  [000] .... 21167.098890: fsg_main_thread: next: bh ffff880111e69e40 state 1
    file-storage-3578  [000] .... 21167.101268: fsg_main_thread: next: bh ffff880111e6ae80 state 2
     irq/17-dwc3-3579  [003] d..1 21167.101473: bulk_out_complete: compl: bh ffff880111e6ae80 state 1
    file-storage-3578  [000] .... 21167.101476: fsg_main_thread: next: bh ffff880111e6ae80 state 1
    file-storage-3578  [000] .... 21167.103842: fsg_main_thread: next: bh ffff880111e6bec0 state 2
     irq/17-dwc3-3579  [003] d..1 21167.104002: bulk_out_complete: compl: bh ffff880111e6bec0 state 1
    file-storage-3578  [000] .... 21167.104006: fsg_main_thread: next: bh ffff880111e6bec0 state 1
    file-storage-3578  [000] .... 21167.106391: fsg_main_thread: next: bh ffff880111e68f40 state 2
     irq/17-dwc3-3579  [003] d..1 21167.106599: bulk_out_complete: compl: bh ffff880111e68f40 state 1
    file-storage-3578  [000] .... 21167.106603: fsg_main_thread: next: bh ffff880111e68f40 state 1
    file-storage-3578  [000] .... 21167.108889: fsg_main_thread: next: bh ffff880111e69f80 state 2
    file-storage-3578  [002] .... 21167.108990: fsg_main_thread: next: bh ffff880111e69f80 state 2
     irq/17-dwc3-3579  [003] d..1 21167.109135: bulk_out_complete: compl: bh ffff880111e69f80 state 1
    file-storage-3578  [002] .... 21167.109139: fsg_main_thread: next: bh ffff880111e69f80 state 1
    file-storage-3578  [002] .... 21167.111491: fsg_main_thread: next: bh ffff880111e6afc0 state 2
     irq/17-dwc3-3579  [003] d..1 21167.111653: bulk_out_complete: compl: bh ffff880111e6afc0 state 1
    file-storage-3578  [002] .... 21167.111656: fsg_main_thread: next: bh ffff880111e6afc0 state 1
    file-storage-3578  [002] .... 21167.114030: fsg_main_thread: next: bh ffff880111e68040 state 2
     irq/17-dwc3-3579  [003] d..1 21167.114207: bulk_out_complete: compl: bh ffff880111e68040 state 1
    file-storage-3578  [002] .... 21167.114210: fsg_main_thread: next: bh ffff880111e68040 state 1
    file-storage-3578  [002] .... 21167.116585: fsg_main_thread: next: bh ffff880111e69080 state 2
     irq/17-dwc3-3579  [003] d..1 21167.116747: bulk_out_complete: compl: bh ffff880111e69080 state 1
    file-storage-3578  [002] .... 21167.116751: fsg_main_thread: next: bh ffff880111e69080 state 1
    file-storage-3578  [002] .... 21167.119105: fsg_main_thread: next: bh ffff880111e6a0c0 state 2
     irq/17-dwc3-3579  [003] d..1 21167.119305: bulk_out_complete: compl: bh ffff880111e6a0c0 state 1
    file-storage-3578  [002] .... 21167.119309: fsg_main_thread: next: bh ffff880111e6a0c0 state 1
    file-storage-3578  [000] .... 21167.121615: fsg_main_thread: next: bh ffff880111e6b100 state 2
    file-storage-3578  [000] .... 21167.121681: fsg_main_thread: next: bh ffff880111e6b100 state 2
     irq/17-dwc3-3579  [003] d..1 21167.121840: bulk_out_complete: compl: bh ffff880111e6b100 state 1
    file-storage-3578  [000] .... 21167.121844: fsg_main_thread: next: bh ffff880111e6b100 state 1
    file-storage-3578  [000] .... 21167.124192: fsg_main_thread: next: bh ffff880111e68180 state 2
     irq/17-dwc3-3579  [003] d..1 21167.124349: bulk_out_complete: compl: bh ffff880111e68180 state 1
    file-storage-3578  [000] .... 21167.124352: fsg_main_thread: next: bh ffff880111e68180 state 1
    file-storage-3578  [000] .... 21167.126725: fsg_main_thread: next: bh ffff880111e691c0 state 2
     irq/17-dwc3-3579  [003] d..1 21167.126933: bulk_out_complete: compl: bh ffff880111e691c0 state 1
    file-storage-3578  [000] .... 21167.126936: fsg_main_thread: next: bh ffff880111e691c0 state 1
    file-storage-3578  [000] .... 21167.129325: fsg_main_thread: next: bh ffff880111e6a200 state 2
     irq/17-dwc3-3579  [003] d..1 21167.129520: bulk_out_complete: compl: bh ffff880111e6a200 state 1
    file-storage-3578  [000] .... 21167.129524: fsg_main_thread: next: bh ffff880111e6a200 state 1
    file-storage-3578  [000] .... 21167.131886: fsg_main_thread: next: bh ffff880111e6b240 state 2
     irq/17-dwc3-3579  [003] d..1 21167.132101: bulk_out_complete: compl: bh ffff880111e6b240 state 1
    file-storage-3578  [000] .... 21167.132106: fsg_main_thread: next: bh ffff880111e6b240 state 1
    file-storage-3578  [000] .... 21167.134480: fsg_main_thread: next: bh ffff880111e682c0 state 2
     irq/17-dwc3-3579  [003] d..1 21167.134700: bulk_out_complete: compl: bh ffff880111e682c0 state 1
    file-storage-3578  [000] .... 21167.134704: fsg_main_thread: next: bh ffff880111e682c0 state 1
    file-storage-3578  [000] .... 21167.137086: fsg_main_thread: next: bh ffff880111e69300 state 2
     irq/17-dwc3-3579  [003] d..1 21167.137261: bulk_out_complete: compl: bh ffff880111e69300 state 1
    file-storage-3578  [000] .... 21167.137264: fsg_main_thread: next: bh ffff880111e69300 state 1
    file-storage-3578  [002] .... 21167.139628: fsg_main_thread: next: bh ffff880111e6a340 state 2
     irq/17-dwc3-3579  [003] d..1 21167.139804: bulk_out_complete: compl: bh ffff880111e6a340 state 1
    file-storage-3578  [002] .... 21167.139807: fsg_main_thread: next: bh ffff880111e6a340 state 1
    file-storage-3578  [002] .... 21167.142179: fsg_main_thread: next: bh ffff880111e6b380 state 2
     irq/17-dwc3-3579  [003] d..1 21167.142368: bulk_out_complete: compl: bh ffff880111e6b380 state 1
    file-storage-3578  [002] .... 21167.142373: fsg_main_thread: next: bh ffff880111e6b380 state 1
    file-storage-3578  [002] .... 21167.144749: fsg_main_thread: next: bh ffff880111e68400 state 2
     irq/17-dwc3-3579  [003] d..1 21167.144975: bulk_out_complete: compl: bh ffff880111e68400 state 1
    file-storage-3578  [002] .... 21167.144978: fsg_main_thread: next: bh ffff880111e68400 state 1
    file-storage-3578  [000] .... 21167.147387: fsg_main_thread: next: bh ffff880111e69440 state 2
     irq/17-dwc3-3579  [003] d..1 21167.147557: bulk_out_complete: compl: bh ffff880111e69440 state 1
    file-storage-3578  [000] .... 21167.147561: fsg_main_thread: next: bh ffff880111e69440 state 1
    file-storage-3578  [000] .... 21167.149935: fsg_main_thread: next: bh ffff880111e6a480 state 2
     irq/17-dwc3-3579  [003] d..1 21167.150156: bulk_out_complete: compl: bh ffff880111e6a480 state 1
    file-storage-3578  [000] .... 21167.150161: fsg_main_thread: next: bh ffff880111e6a480 state 1
    file-storage-3578  [000] .... 21167.152451: fsg_main_thread: next: bh ffff880111e6b4c0 state 2
    file-storage-3578  [000] .... 21167.152516: fsg_main_thread: next: bh ffff880111e6b4c0 state 2
     irq/17-dwc3-3579  [003] d..1 21167.152678: bulk_out_complete: compl: bh ffff880111e6b4c0 state 1
    file-storage-3578  [000] .... 21167.152682: fsg_main_thread: next: bh ffff880111e6b4c0 state 1
    file-storage-3578  [000] .... 21167.155048: fsg_main_thread: next: bh ffff880111e68540 state 2
     irq/17-dwc3-3579  [003] d..1 21167.155260: bulk_out_complete: compl: bh ffff880111e68540 state 1
    file-storage-3578  [000] .... 21167.155264: fsg_main_thread: next: bh ffff880111e68540 state 1
    file-storage-3578  [000] .... 21167.157640: fsg_main_thread: next: bh ffff880111e69580 state 2
     irq/17-dwc3-3579  [003] d..1 21167.157852: bulk_out_complete: compl: bh ffff880111e69580 state 1
    file-storage-3578  [000] .... 21167.157855: fsg_main_thread: next: bh ffff880111e69580 state 1
    file-storage-3578  [000] .... 21167.160132: fsg_main_thread: next: bh ffff880111e6a5c0 state 2
    file-storage-3578  [002] .... 21167.160197: fsg_main_thread: next: bh ffff880111e6a5c0 state 2
     irq/17-dwc3-3579  [003] d..1 21167.160375: bulk_out_complete: compl: bh ffff880111e6a5c0 state 1
    file-storage-3578  [002] .... 21167.160379: fsg_main_thread: next: bh ffff880111e6a5c0 state 1
    file-storage-3578  [002] .... 21167.162730: fsg_main_thread: next: bh ffff880111e6b600 state 2
     irq/17-dwc3-3579  [003] d..1 21167.162900: bulk_out_complete: compl: bh ffff880111e6b600 state 1
    file-storage-3578  [002] .... 21167.162904: fsg_main_thread: next: bh ffff880111e6b600 state 1
    file-storage-3578  [002] .... 21167.165295: fsg_main_thread: next: bh ffff880111e68680 state 2
     irq/17-dwc3-3579  [003] d..1 21167.165466: bulk_out_complete: compl: bh ffff880111e68680 state 1
    file-storage-3578  [002] .... 21167.165469: fsg_main_thread: next: bh ffff880111e68680 state 1
    file-storage-3578  [002] .... 21167.167838: fsg_main_thread: next: bh ffff880111e696c0 state 2
     irq/17-dwc3-3579  [003] d..1 21167.168105: bulk_out_complete: compl: bh ffff880111e696c0 state 1
    file-storage-3578  [001] .... 21167.171118: fsg_main_thread: next: bh ffff880111e696c0 state 1
    file-storage-3578  [001] .... 21167.173506: fsg_main_thread: next: bh ffff880111e6a700 state 2
     irq/17-dwc3-3579  [003] d..1 21167.173741: bulk_out_complete: compl: bh ffff880111e6a700 state 1
    file-storage-3578  [001] .... 21167.173744: fsg_main_thread: next: bh ffff880111e6a700 state 1
    file-storage-3578  [001] .... 21167.176031: fsg_main_thread: next: bh ffff880111e6b740 state 2
    file-storage-3578  [000] .... 21167.176097: fsg_main_thread: next: bh ffff880111e6b740 state 2
     irq/17-dwc3-3579  [003] d..1 21167.176275: bulk_out_complete: compl: bh ffff880111e6b740 state 1
    file-storage-3578  [000] .... 21167.176279: fsg_main_thread: next: bh ffff880111e6b740 state 1
    file-storage-3578  [002] .... 21167.178642: fsg_main_thread: next: bh ffff880111e687c0 state 2
     irq/17-dwc3-3579  [003] d..1 21167.178793: bulk_out_complete: compl: bh ffff880111e687c0 state 1
    file-storage-3578  [002] .... 21167.178796: fsg_main_thread: next: bh ffff880111e687c0 state 1
    file-storage-3578  [002] .... 21167.181152: fsg_main_thread: next: bh ffff880111e69800 state 2
     irq/17-dwc3-3579  [003] d..1 21167.181351: bulk_out_complete: compl: bh ffff880111e69800 state 1
    file-storage-3578  [002] .... 21167.181354: fsg_main_thread: next: bh ffff880111e69800 state 1
    file-storage-3578  [000] .... 21167.183665: fsg_main_thread: next: bh ffff880111e6a840 state 2
    file-storage-3578  [000] .... 21167.183731: fsg_main_thread: next: bh ffff880111e6a840 state 2
     irq/17-dwc3-3579  [003] d..1 21167.183931: bulk_out_complete: compl: bh ffff880111e6a840 state 1
    file-storage-3578  [000] .... 21167.183935: fsg_main_thread: next: bh ffff880111e6a840 state 1
    file-storage-3578  [000] .... 21167.186311: fsg_main_thread: next: bh ffff880111e6b880 state 2
     irq/17-dwc3-3579  [003] d..1 21167.186482: bulk_out_complete: compl: bh ffff880111e6b880 state 1
    file-storage-3578  [000] .... 21167.186485: fsg_main_thread: next: bh ffff880111e6b880 state 1
    file-storage-3578  [000] .... 21167.188899: fsg_main_thread: next: bh ffff880111e68900 state 2
     irq/17-dwc3-3579  [003] d..1 21167.189117: bulk_out_complete: compl: bh ffff880111e68900 state 1
    file-storage-3578  [000] .... 21167.189122: fsg_main_thread: next: bh ffff880111e68900 state 1
    file-storage-3578  [000] .... 21167.191502: fsg_main_thread: next: bh ffff880111e69940 state 2
     irq/17-dwc3-3579  [003] d..1 21167.191673: bulk_out_complete: compl: bh ffff880111e69940 state 1
    file-storage-3578  [000] .... 21167.191676: fsg_main_thread: next: bh ffff880111e69940 state 1
    file-storage-3578  [000] .... 21167.194058: fsg_main_thread: next: bh ffff880111e6a980 state 2
     irq/17-dwc3-3579  [003] d..1 21167.194228: bulk_out_complete: compl: bh ffff880111e6a980 state 1
    file-storage-3578  [000] .... 21167.194231: fsg_main_thread: next: bh ffff880111e6a980 state 1
    file-storage-3578  [000] .... 21167.196618: fsg_main_thread: next: bh ffff880111e6b9c0 state 2
     irq/17-dwc3-3579  [003] d..1 21167.196833: bulk_out_complete: compl: bh ffff880111e6b9c0 state 1
    file-storage-3578  [000] .... 21167.196837: fsg_main_thread: next: bh ffff880111e6b9c0 state 1
    file-storage-3578  [002] .... 21167.199213: fsg_main_thread: next: bh ffff880111e68a40 state 2
     irq/17-dwc3-3579  [003] d..1 21167.199395: bulk_out_complete: compl: bh ffff880111e68a40 state 1
    file-storage-3578  [002] .... 21167.199398: fsg_main_thread: next: bh ffff880111e68a40 state 1
    file-storage-3578  [002] .... 21167.201796: fsg_main_thread: next: bh ffff880111e69a80 state 2
     irq/17-dwc3-3579  [003] d..1 21167.201961: bulk_out_complete: compl: bh ffff880111e69a80 state 1
    file-storage-3578  [002] .... 21167.201964: fsg_main_thread: next: bh ffff880111e69a80 state 1
    file-storage-3578  [002] .... 21167.204326: fsg_main_thread: next: bh ffff880111e6aac0 state 2
     irq/17-dwc3-3579  [003] d..1 21167.204509: bulk_out_complete: compl: bh ffff880111e6aac0 state 1
    file-storage-3578  [002] .... 21167.204512: fsg_main_thread: next: bh ffff880111e6aac0 state 1
    file-storage-3578  [002] .... 21167.206913: fsg_main_thread: next: bh ffff880111e6bb00 state 2
     irq/17-dwc3-3579  [003] d..1 21167.207077: bulk_out_complete: compl: bh ffff880111e6bb00 state 1
    file-storage-3578  [002] .... 21167.207080: fsg_main_thread: next: bh ffff880111e6bb00 state 1
    file-storage-3578  [002] .... 21167.209421: fsg_main_thread: next: bh ffff880111e68b80 state 2
     irq/17-dwc3-3579  [003] d..1 21167.209622: bulk_out_complete: compl: bh ffff880111e68b80 state 1
    file-storage-3578  [002] .... 21167.209626: fsg_main_thread: next: bh ffff880111e68b80 state 1
    file-storage-3578  [002] .... 21167.211990: fsg_main_thread: next: bh ffff880111e69bc0 state 2
     irq/17-dwc3-3579  [003] d..1 21167.212238: bulk_out_complete: compl: bh ffff880111e69bc0 state 1
    file-storage-3578  [002] .... 21167.212241: fsg_main_thread: next: bh ffff880111e69bc0 state 1
    file-storage-3578  [000] .... 21167.214551: fsg_main_thread: next: bh ffff880111e6ac00 state 2
    file-storage-3578  [000] .... 21167.214615: fsg_main_thread: next: bh ffff880111e6ac00 state 2
     irq/17-dwc3-3579  [003] d..1 21167.214829: bulk_out_complete: compl: bh ffff880111e6ac00 state 1
    file-storage-3578  [000] .... 21167.214833: fsg_main_thread: next: bh ffff880111e6ac00 state 1
    file-storage-3578  [000] .... 21167.217206: fsg_main_thread: next: bh ffff880111e6bc40 state 2
     irq/17-dwc3-3579  [003] d..1 21167.217389: bulk_out_complete: compl: bh ffff880111e6bc40 state 1
    file-storage-3578  [000] .... 21167.217394: fsg_main_thread: next: bh ffff880111e6bc40 state 1
    file-storage-3578  [000] .... 21167.219810: fsg_main_thread: next: bh ffff880111e68cc0 state 2
     irq/17-dwc3-3579  [003] d..1 21167.219947: bulk_out_complete: compl: bh ffff880111e68cc0 state 1
    file-storage-3578  [000] .... 21167.219951: fsg_main_thread: next: bh ffff880111e68cc0 state 1
    file-storage-3578  [000] .... 21167.222332: fsg_main_thread: next: bh ffff880111e69d00 state 2
     irq/17-dwc3-3579  [003] d..1 21167.222494: bulk_out_complete: compl: bh ffff880111e69d00 state 1
    file-storage-3578  [000] .... 21167.222498: fsg_main_thread: next: bh ffff880111e69d00 state 1
    file-storage-3578  [000] .... 21167.224779: fsg_main_thread: next: bh ffff880111e6ad40 state 2
    file-storage-3578  [002] .... 21167.224846: fsg_main_thread: next: bh ffff880111e6ad40 state 2
     irq/17-dwc3-3579  [003] d..1 21167.225106: bulk_out_complete: compl: bh ffff880111e6ad40 state 1
    file-storage-3578  [002] .... 21167.225110: fsg_main_thread: next: bh ffff880111e6ad40 state 1
    file-storage-3578  [000] .... 21167.227470: fsg_main_thread: next: bh ffff880111e6bd80 state 2
     irq/17-dwc3-3579  [003] d..1 21167.227651: bulk_out_complete: compl: bh ffff880111e6bd80 state 1
    file-storage-3578  [000] .... 21167.227656: fsg_main_thread: next: bh ffff880111e6bd80 state 1
    file-storage-3578  [000] .... 21167.230010: fsg_main_thread: next: bh ffff880111e68e00 state 2
     irq/17-dwc3-3579  [003] d..1 21167.230181: bulk_out_complete: compl: bh ffff880111e68e00 state 1
    file-storage-3578  [000] .... 21167.230185: fsg_main_thread: next: bh ffff880111e68e00 state 1
    file-storage-3578  [002] .... 21167.232528: fsg_main_thread: next: bh ffff880111e69e40 state 2
     irq/17-dwc3-3579  [003] d..1 21167.232697: bulk_out_complete: compl: bh ffff880111e69e40 state 1
    file-storage-3578  [002] .... 21167.232701: fsg_main_thread: next: bh ffff880111e69e40 state 1
    file-storage-3578  [002] .... 21167.235063: fsg_main_thread: next: bh ffff880111e6ae80 state 2
     irq/17-dwc3-3579  [003] d..1 21167.235238: bulk_out_complete: compl: bh ffff880111e6ae80 state 1
    file-storage-3578  [002] .... 21167.235241: fsg_main_thread: next: bh ffff880111e6ae80 state 1
    file-storage-3578  [000] .... 21167.237637: fsg_main_thread: next: bh ffff880111e6bec0 state 2
     irq/17-dwc3-3579  [003] d..1 21167.237793: bulk_out_complete: compl: bh ffff880111e6bec0 state 1
    file-storage-3578  [000] .... 21167.237797: fsg_main_thread: next: bh ffff880111e6bec0 state 1
    file-storage-3578  [000] .... 21167.240185: fsg_main_thread: next: bh ffff880111e68f40 state 2
     irq/17-dwc3-3579  [003] d..1 21167.240399: bulk_out_complete: compl: bh ffff880111e68f40 state 1
    file-storage-3578  [000] .... 21167.240403: fsg_main_thread: next: bh ffff880111e68f40 state 1
    file-storage-3578  [000] .... 21167.242780: fsg_main_thread: next: bh ffff880111e69f80 state 2
     irq/17-dwc3-3579  [003] d..1 21167.242951: bulk_out_complete: compl: bh ffff880111e69f80 state 1
    file-storage-3578  [000] .... 21167.242955: fsg_main_thread: next: bh ffff880111e69f80 state 1
    file-storage-3578  [000] .... 21167.245313: fsg_main_thread: next: bh ffff880111e6afc0 state 2
     irq/17-dwc3-3579  [003] d..1 21167.245475: bulk_out_complete: compl: bh ffff880111e6afc0 state 1
    file-storage-3578  [000] .... 21167.245479: fsg_main_thread: next: bh ffff880111e6afc0 state 1
    file-storage-3578  [002] .... 21167.247829: fsg_main_thread: next: bh ffff880111e68040 state 2
     irq/17-dwc3-3579  [003] d..1 21167.248012: bulk_out_complete: compl: bh ffff880111e68040 state 1
    file-storage-3578  [002] .... 21167.248015: fsg_main_thread: next: bh ffff880111e68040 state 1
    file-storage-3578  [002] .... 21167.250370: fsg_main_thread: next: bh ffff880111e69080 state 2
     irq/17-dwc3-3579  [003] d..1 21167.250551: bulk_out_complete: compl: bh ffff880111e69080 state 1
    file-storage-3578  [002] .... 21167.250554: fsg_main_thread: next: bh ffff880111e69080 state 1
    file-storage-3578  [002] .... 21167.252826: fsg_main_thread: next: bh ffff880111e6a0c0 state 2
    file-storage-3578  [002] .... 21167.252891: fsg_main_thread: next: bh ffff880111e6a0c0 state 2
     irq/17-dwc3-3579  [003] d..1 21167.253067: bulk_out_complete: compl: bh ffff880111e6a0c0 state 1
    file-storage-3578  [000] .... 21167.253073: fsg_main_thread: next: bh ffff880111e6a0c0 state 1
    file-storage-3578  [000] .... 21167.255747: fsg_main_thread: next: bh ffff880111e6b100 state 2
     irq/17-dwc3-3579  [003] d..1 21167.255815: bulk_out_complete: compl: bh ffff880111e6b100 state 1
    file-storage-3578  [000] .... 21167.255819: fsg_main_thread: next: bh ffff880111e6b100 state 1
    file-storage-3578  [000] .... 21167.258108: fsg_main_thread: next: bh ffff880111e68180 state 2
     irq/17-dwc3-3579  [003] d..1 21167.258387: bulk_out_complete: compl: bh ffff880111e68180 state 1
    file-storage-3578  [000] .... 21167.258387: fsg_main_thread: next: bh ffff880111e68180 state 1
    file-storage-3578  [000] .... 21167.260758: fsg_main_thread: next: bh ffff880111e691c0 state 2
     irq/17-dwc3-3579  [003] d..1 21167.260947: bulk_out_complete: compl: bh ffff880111e691c0 state 1
    file-storage-3578  [000] .... 21167.260951: fsg_main_thread: next: bh ffff880111e691c0 state 1
    file-storage-3578  [002] .... 21167.263384: fsg_main_thread: next: bh ffff880111e6a200 state 2
     irq/17-dwc3-3579  [003] d..1 21167.263555: bulk_out_complete: compl: bh ffff880111e6a200 state 1
    file-storage-3578  [002] .... 21167.263559: fsg_main_thread: next: bh ffff880111e6a200 state 1
    file-storage-3578  [002] .... 21167.265909: fsg_main_thread: next: bh ffff880111e6b240 state 2
     irq/17-dwc3-3579  [003] d..1 21167.266088: bulk_out_complete: compl: bh ffff880111e6b240 state 1
    file-storage-3578  [002] .... 21167.266092: fsg_main_thread: next: bh ffff880111e6b240 state 1
    file-storage-3578  [002] .... 21167.268478: fsg_main_thread: next: bh ffff880111e682c0 state 2
     irq/17-dwc3-3579  [003] d..1 21167.268684: bulk_out_complete: compl: bh ffff880111e682c0 state 1
    file-storage-3578  [002] .... 21167.268687: fsg_main_thread: next: bh ffff880111e682c0 state 1
    file-storage-3578  [000] .... 21167.271091: fsg_main_thread: next: bh ffff880111e69300 state 2
     irq/17-dwc3-3579  [003] d..1 21167.271262: bulk_out_complete: compl: bh ffff880111e69300 state 1
    file-storage-3578  [000] .... 21167.271266: fsg_main_thread: next: bh ffff880111e69300 state 1
    file-storage-3578  [000] .... 21167.273640: fsg_main_thread: next: bh ffff880111e6a340 state 2
     irq/17-dwc3-3579  [003] d..1 21167.273806: bulk_out_complete: compl: bh ffff880111e6a340 state 1
    file-storage-3578  [000] .... 21167.273810: fsg_main_thread: next: bh ffff880111e6a340 state 1
    file-storage-3578  [000] .... 21167.276199: fsg_main_thread: next: bh ffff880111e6b380 state 2
     irq/17-dwc3-3579  [003] d..1 21167.276386: bulk_out_complete: compl: bh ffff880111e6b380 state 1
    file-storage-3578  [000] .... 21167.276389: fsg_main_thread: next: bh ffff880111e6b380 state 1
    file-storage-3578  [002] .... 21167.278749: fsg_main_thread: next: bh ffff880111e68400 state 2
     irq/17-dwc3-3579  [003] d..1 21167.278975: bulk_out_complete: compl: bh ffff880111e68400 state 1
    file-storage-3578  [002] .... 21167.278979: fsg_main_thread: next: bh ffff880111e68400 state 1
    file-storage-3578  [002] .... 21167.281336: fsg_main_thread: next: bh ffff880111e69440 state 2
     irq/17-dwc3-3579  [003] d..1 21167.281506: bulk_out_complete: compl: bh ffff880111e69440 state 1
    file-storage-3578  [002] .... 21167.281509: fsg_main_thread: next: bh ffff880111e69440 state 1
    file-storage-3578  [002] .... 21167.283877: fsg_main_thread: next: bh ffff880111e6a480 state 2
     irq/17-dwc3-3579  [003] d..1 21167.284047: bulk_out_complete: compl: bh ffff880111e6a480 state 1
    file-storage-3578  [002] .... 21167.284050: fsg_main_thread: next: bh ffff880111e6a480 state 1
    file-storage-3578  [002] .... 21167.286423: fsg_main_thread: next: bh ffff880111e6b4c0 state 2
     irq/17-dwc3-3579  [003] d..1 21167.286636: bulk_out_complete: compl: bh ffff880111e6b4c0 state 1
    file-storage-3578  [002] .... 21167.286640: fsg_main_thread: next: bh ffff880111e6b4c0 state 1
    file-storage-3578  [002] .... 21167.289042: fsg_main_thread: next: bh ffff880111e68540 state 2
     irq/17-dwc3-3579  [003] d..1 21167.289212: bulk_out_complete: compl: bh ffff880111e68540 state 1
    file-storage-3578  [002] .... 21167.289215: fsg_main_thread: next: bh ffff880111e68540 state 1
    file-storage-3578  [002] .... 21167.291584: fsg_main_thread: next: bh ffff880111e69580 state 2
     irq/17-dwc3-3579  [003] d..1 21167.291751: bulk_out_complete: compl: bh ffff880111e69580 state 1
    file-storage-3578  [002] .... 21167.291755: fsg_main_thread: next: bh ffff880111e69580 state 1
    file-storage-3578  [002] .... 21167.294170: fsg_main_thread: next: bh ffff880111e6a5c0 state 2
     irq/17-dwc3-3579  [003] d..1 21167.294358: bulk_out_complete: compl: bh ffff880111e6a5c0 state 1
    file-storage-3578  [002] .... 21167.294362: fsg_main_thread: next: bh ffff880111e6a5c0 state 1
    file-storage-3578  [002] .... 21167.296734: fsg_main_thread: next: bh ffff880111e6b600 state 2
     irq/17-dwc3-3579  [003] d..1 21167.296906: bulk_out_complete: compl: bh ffff880111e6b600 state 1
    file-storage-3578  [002] .... 21167.296910: fsg_main_thread: next: bh ffff880111e6b600 state 1
    file-storage-3578  [002] .... 21167.299184: fsg_main_thread: next: bh ffff880111e68680 state 2
    file-storage-3578  [002] .... 21167.299247: fsg_main_thread: next: bh ffff880111e68680 state 2
     irq/17-dwc3-3579  [003] d..1 21167.299414: bulk_out_complete: compl: bh ffff880111e68680 state 1
    file-storage-3578  [002] .... 21167.299417: fsg_main_thread: next: bh ffff880111e68680 state 1
    file-storage-3578  [002] .... 21167.301696: fsg_main_thread: next: bh ffff880111e696c0 state 2
    file-storage-3578  [002] .... 21167.301761: fsg_main_thread: next: bh ffff880111e696c0 state 2
     irq/17-dwc3-3579  [003] d..1 21167.301973: bulk_out_complete: compl: bh ffff880111e696c0 state 1
    file-storage-3578  [002] .... 21167.301977: fsg_main_thread: next: bh ffff880111e696c0 state 1
    file-storage-3578  [002] .... 21167.304276: fsg_main_thread: next: bh ffff880111e6a700 state 2
    file-storage-3578  [000] .... 21167.304343: fsg_main_thread: next: bh ffff880111e6a700 state 2
     irq/17-dwc3-3579  [003] d..1 21167.304522: bulk_out_complete: compl: bh ffff880111e6a700 state 1
    file-storage-3578  [000] .... 21167.304526: fsg_main_thread: next: bh ffff880111e6a700 state 1
    file-storage-3578  [000] .... 21167.306888: fsg_main_thread: next: bh ffff880111e6b740 state 2
     irq/17-dwc3-3579  [003] d..1 21167.307062: bulk_out_complete: compl: bh ffff880111e6b740 state 1
    file-storage-3578  [000] .... 21167.307065: fsg_main_thread: next: bh ffff880111e6b740 state 1
    file-storage-3578  [000] .... 21167.309454: fsg_main_thread: next: bh ffff880111e687c0 state 2
     irq/17-dwc3-3579  [003] d..1 21167.309664: bulk_out_complete: compl: bh ffff880111e687c0 state 1
    file-storage-3578  [000] .... 21167.309668: fsg_main_thread: next: bh ffff880111e687c0 state 1
    file-storage-3578  [002] .... 21167.312042: fsg_main_thread: next: bh ffff880111e69800 state 2
     irq/17-dwc3-3579  [003] d..1 21167.312227: bulk_out_complete: compl: bh ffff880111e69800 state 1
    file-storage-3578  [002] .... 21167.312231: fsg_main_thread: next: bh ffff880111e69800 state 1
    file-storage-3578  [002] .... 21167.314569: fsg_main_thread: next: bh ffff880111e6a840 state 2
     irq/17-dwc3-3579  [003] d..1 21167.314727: bulk_out_complete: compl: bh ffff880111e6a840 state 1
    file-storage-3578  [002] .... 21167.314731: fsg_main_thread: next: bh ffff880111e6a840 state 1
    file-storage-3578  [002] .... 21167.317083: fsg_main_thread: next: bh ffff880111e6b880 state 2
     irq/17-dwc3-3579  [003] d..1 21167.317267: bulk_out_complete: compl: bh ffff880111e6b880 state 1
    file-storage-3578  [002] .... 21167.317270: fsg_main_thread: next: bh ffff880111e6b880 state 1
    file-storage-3578  [002] .... 21167.319623: fsg_main_thread: next: bh ffff880111e68900 state 2
     irq/17-dwc3-3579  [003] d..1 21167.319790: bulk_out_complete: compl: bh ffff880111e68900 state 1
    file-storage-3578  [002] .... 21167.319793: fsg_main_thread: next: bh ffff880111e68900 state 1
    file-storage-3578  [002] .... 21167.322177: fsg_main_thread: next: bh ffff880111e69940 state 2
     irq/17-dwc3-3579  [003] d..1 21167.322358: bulk_out_complete: compl: bh ffff880111e69940 state 1
    file-storage-3578  [002] .... 21167.322361: fsg_main_thread: next: bh ffff880111e69940 state 1
    file-storage-3578  [002] .... 21167.324760: fsg_main_thread: next: bh ffff880111e6a980 state 2
     irq/17-dwc3-3579  [003] d..1 21167.324951: bulk_out_complete: compl: bh ffff880111e6a980 state 1
    file-storage-3578  [002] .... 21167.324955: fsg_main_thread: next: bh ffff880111e6a980 state 1
    file-storage-3578  [000] .... 21167.327324: fsg_main_thread: next: bh ffff880111e6b9c0 state 2
     irq/17-dwc3-3579  [003] d..1 21167.327534: bulk_out_complete: compl: bh ffff880111e6b9c0 state 1
    file-storage-3578  [000] .... 21167.327539: fsg_main_thread: next: bh ffff880111e6b9c0 state 1
    file-storage-3578  [002] .... 21167.329977: fsg_main_thread: next: bh ffff880111e68a40 state 2
     irq/17-dwc3-3579  [003] d..1 21167.330160: bulk_out_complete: compl: bh ffff880111e68a40 state 1
    file-storage-3578  [002] .... 21167.330164: fsg_main_thread: next: bh ffff880111e68a40 state 1
    file-storage-3578  [002] .... 21167.332535: fsg_main_thread: next: bh ffff880111e69a80 state 2
     irq/17-dwc3-3579  [003] d..1 21167.332710: bulk_out_complete: compl: bh ffff880111e69a80 state 1
    file-storage-3578  [002] .... 21167.332713: fsg_main_thread: next: bh ffff880111e69a80 state 1
    file-storage-3578  [002] .... 21167.335103: fsg_main_thread: next: bh ffff880111e6aac0 state 2
     irq/17-dwc3-3579  [003] d..1 21167.335286: bulk_out_complete: compl: bh ffff880111e6aac0 state 1
    file-storage-3578  [000] .... 21167.335290: fsg_main_thread: next: bh ffff880111e6aac0 state 1
     irq/17-dwc3-3579  [003] d..1 21167.337859: bulk_out_complete: compl: bh ffff880111e6bb00 state 1
    file-storage-3578  [000] .... 21167.337859: fsg_main_thread: next: bh ffff880111e6bb00 state 2
    file-storage-3578  [000] .... 21167.340471: fsg_main_thread: next: bh ffff880111e68b80 state 2
     irq/17-dwc3-3579  [003] d..1 21167.340544: bulk_out_complete: compl: bh ffff880111e68b80 state 1
    file-storage-3578  [000] .... 21167.340548: fsg_main_thread: next: bh ffff880111e68b80 state 1
    file-storage-3578  [000] .... 21167.343168: fsg_main_thread: next: bh ffff880111e69bc0 state 2
     irq/17-dwc3-3579  [003] d..1 21167.343241: bulk_out_complete: compl: bh ffff880111e69bc0 state 1
    file-storage-3578  [000] .... 21167.343245: fsg_main_thread: next: bh ffff880111e69bc0 state 1
    file-storage-3578  [000] .... 21167.345623: fsg_main_thread: next: bh ffff880111e6ac00 state 2
     irq/17-dwc3-3579  [003] d..1 21167.345846: bulk_out_complete: compl: bh ffff880111e6ac00 state 1
    file-storage-3578  [000] .... 21167.345849: fsg_main_thread: next: bh ffff880111e6ac00 state 1
    file-storage-3578  [000] .... 21167.348211: fsg_main_thread: next: bh ffff880111e6bc40 state 2
     irq/17-dwc3-3579  [003] d..1 21167.348368: bulk_out_complete: compl: bh ffff880111e6bc40 state 1
    file-storage-3578  [000] .... 21167.348371: fsg_main_thread: next: bh ffff880111e6bc40 state 1
    file-storage-3578  [000] .... 21167.350662: fsg_main_thread: next: bh ffff880111e68cc0 state 2
    file-storage-3578  [002] .... 21167.350727: fsg_main_thread: next: bh ffff880111e68cc0 state 2
     irq/17-dwc3-3579  [003] d..1 21167.350908: bulk_out_complete: compl: bh ffff880111e68cc0 state 1
    file-storage-3578  [002] .... 21167.350912: fsg_main_thread: next: bh ffff880111e68cc0 state 1
    file-storage-3578  [002] .... 21167.353282: fsg_main_thread: next: bh ffff880111e69d00 state 2
     irq/17-dwc3-3579  [003] d..1 21167.353458: bulk_out_complete: compl: bh ffff880111e69d00 state 1
    file-storage-3578  [002] .... 21167.353461: fsg_main_thread: next: bh ffff880111e69d00 state 1
    file-storage-3578  [000] .... 21167.355809: fsg_main_thread: next: bh ffff880111e6ad40 state 2
     irq/17-dwc3-3579  [003] d..1 21167.355966: bulk_out_complete: compl: bh ffff880111e6ad40 state 1
    file-storage-3578  [000] .... 21167.355970: fsg_main_thread: next: bh ffff880111e6ad40 state 1
    file-storage-3578  [000] .... 21167.358322: fsg_main_thread: next: bh ffff880111e6bd80 state 2
     irq/17-dwc3-3579  [003] d..1 21167.358494: bulk_out_complete: compl: bh ffff880111e6bd80 state 1
    file-storage-3578  [002] .... 21167.358500: fsg_main_thread: next: bh ffff880111e6bd80 state 1
    file-storage-3578  [000] .... 21167.360869: fsg_main_thread: next: bh ffff880111e68e00 state 2
     irq/17-dwc3-3579  [003] d..1 21167.361040: bulk_out_complete: compl: bh ffff880111e68e00 state 1
    file-storage-3578  [000] .... 21167.361044: fsg_main_thread: next: bh ffff880111e68e00 state 1
    file-storage-3578  [000] .... 21167.363400: fsg_main_thread: next: bh ffff880111e69e40 state 2
     irq/17-dwc3-3579  [003] d..1 21167.363566: bulk_out_complete: compl: bh ffff880111e69e40 state 1
    file-storage-3578  [000] .... 21167.363569: fsg_main_thread: next: bh ffff880111e69e40 state 1
    file-storage-3578  [002] .... 21167.365920: fsg_main_thread: next: bh ffff880111e6ae80 state 2
     irq/17-dwc3-3579  [003] d..1 21167.366121: bulk_out_complete: compl: bh ffff880111e6ae80 state 1
    file-storage-3578  [002] .... 21167.366125: fsg_main_thread: next: bh ffff880111e6ae80 state 1
    file-storage-3578  [000] .... 21167.368480: fsg_main_thread: next: bh ffff880111e6bec0 state 2
     irq/17-dwc3-3579  [003] d..1 21167.368662: bulk_out_complete: compl: bh ffff880111e6bec0 state 1
    file-storage-3578  [000] .... 21167.368667: fsg_main_thread: next: bh ffff880111e6bec0 state 1
    file-storage-3578  [000] .... 21167.370944: fsg_main_thread: next: bh ffff880111e68f40 state 2
    file-storage-3578  [000] .... 21167.371007: fsg_main_thread: next: bh ffff880111e68f40 state 2
     irq/17-dwc3-3579  [003] d..1 21167.371181: bulk_out_complete: compl: bh ffff880111e68f40 state 1
    file-storage-3578  [000] .... 21167.371184: fsg_main_thread: next: bh ffff880111e68f40 state 1
    file-storage-3578  [002] .... 21167.373521: fsg_main_thread: next: bh ffff880111e69f80 state 2
     irq/17-dwc3-3579  [003] d..1 21167.373693: bulk_out_complete: compl: bh ffff880111e69f80 state 1
    file-storage-3578  [002] .... 21167.373696: fsg_main_thread: next: bh ffff880111e69f80 state 1
    file-storage-3578  [002] .... 21167.376049: fsg_main_thread: next: bh ffff880111e6afc0 state 2
     irq/17-dwc3-3579  [003] d..1 21167.376222: bulk_out_complete: compl: bh ffff880111e6afc0 state 1
    file-storage-3578  [002] .... 21167.376225: fsg_main_thread: next: bh ffff880111e6afc0 state 1
    file-storage-3578  [002] .... 21167.378598: fsg_main_thread: next: bh ffff880111e68040 state 2
     irq/17-dwc3-3579  [003] d..1 21167.378779: bulk_out_complete: compl: bh ffff880111e68040 state 1
    file-storage-3578  [002] .... 21167.378782: fsg_main_thread: next: bh ffff880111e68040 state 1
    file-storage-3578  [002] .... 21167.381157: fsg_main_thread: next: bh ffff880111e69080 state 2
     irq/17-dwc3-3579  [003] d..1 21167.381350: bulk_out_complete: compl: bh ffff880111e69080 state 1
    file-storage-3578  [002] .... 21167.381354: fsg_main_thread: next: bh ffff880111e69080 state 1
    file-storage-3578  [000] .... 21167.383722: fsg_main_thread: next: bh ffff880111e6a0c0 state 2
     irq/17-dwc3-3579  [003] d..1 21167.383890: bulk_out_complete: compl: bh ffff880111e6a0c0 state 1
    file-storage-3578  [000] .... 21167.383894: fsg_main_thread: next: bh ffff880111e6a0c0 state 1
    file-storage-3578  [000] .... 21167.386249: fsg_main_thread: next: bh ffff880111e6b100 state 2
     irq/17-dwc3-3579  [003] d..1 21167.386421: bulk_out_complete: compl: bh ffff880111e6b100 state 1
    file-storage-3578  [000] .... 21167.386425: fsg_main_thread: next: bh ffff880111e6b100 state 1
    file-storage-3578  [002] .... 21167.388785: fsg_main_thread: next: bh ffff880111e68180 state 2
     irq/17-dwc3-3579  [003] d..1 21167.388982: bulk_out_complete: compl: bh ffff880111e68180 state 1
    file-storage-3578  [002] .... 21167.388986: fsg_main_thread: next: bh ffff880111e68180 state 1
    file-storage-3578  [000] .... 21167.391335: fsg_main_thread: next: bh ffff880111e691c0 state 2
     irq/17-dwc3-3579  [003] d..1 21167.391509: bulk_out_complete: compl: bh ffff880111e691c0 state 1
    file-storage-3578  [000] .... 21167.391514: fsg_main_thread: next: bh ffff880111e691c0 state 1
    file-storage-3578  [000] .... 21167.393892: fsg_main_thread: next: bh ffff880111e6a200 state 2
     irq/17-dwc3-3579  [003] d..1 21167.394057: bulk_out_complete: compl: bh ffff880111e6a200 state 1
    file-storage-3578  [000] .... 21167.394060: fsg_main_thread: next: bh ffff880111e6a200 state 1
    file-storage-3578  [002] .... 21167.396393: fsg_main_thread: next: bh ffff880111e6b240 state 2
     irq/17-dwc3-3579  [003] d..1 21167.396568: bulk_out_complete: compl: bh ffff880111e6b240 state 1
    file-storage-3578  [002] .... 21167.396572: fsg_main_thread: next: bh ffff880111e6b240 state 1
    file-storage-3578  [002] .... 21167.398949: fsg_main_thread: next: bh ffff880111e682c0 state 2
     irq/17-dwc3-3579  [003] d..1 21167.399144: bulk_out_complete: compl: bh ffff880111e682c0 state 1
    file-storage-3578  [002] .... 21167.399147: fsg_main_thread: next: bh ffff880111e682c0 state 1
    file-storage-3578  [002] .... 21167.401519: fsg_main_thread: next: bh ffff880111e69300 state 2
     irq/17-dwc3-3579  [003] d..1 21167.401693: bulk_out_complete: compl: bh ffff880111e69300 state 1
    file-storage-3578  [002] .... 21167.401696: fsg_main_thread: next: bh ffff880111e69300 state 1
    file-storage-3578  [002] .... 21167.404028: fsg_main_thread: next: bh ffff880111e6a340 state 2
     irq/17-dwc3-3579  [003] d..1 21167.404215: bulk_out_complete: compl: bh ffff880111e6a340 state 1
    file-storage-3578  [002] .... 21167.404218: fsg_main_thread: next: bh ffff880111e6a340 state 1
    file-storage-3578  [002] .... 21167.406621: fsg_main_thread: next: bh ffff880111e6b380 state 2
     irq/17-dwc3-3579  [003] d..1 21167.406813: bulk_out_complete: compl: bh ffff880111e6b380 state 1
    file-storage-3578  [002] .... 21167.406816: fsg_main_thread: next: bh ffff880111e6b380 state 1
    file-storage-3578  [002] .... 21167.409220: fsg_main_thread: next: bh ffff880111e68400 state 2
     irq/17-dwc3-3579  [003] d..1 21167.409396: bulk_out_complete: compl: bh ffff880111e68400 state 1
    file-storage-3578  [002] .... 21167.409399: fsg_main_thread: next: bh ffff880111e68400 state 1
    file-storage-3578  [002] .... 21167.411786: fsg_main_thread: next: bh ffff880111e69440 state 2
     irq/17-dwc3-3579  [003] d..1 21167.411958: bulk_out_complete: compl: bh ffff880111e69440 state 1
    file-storage-3578  [002] .... 21167.411961: fsg_main_thread: next: bh ffff880111e69440 state 1
    file-storage-3578  [000] .... 21167.414345: fsg_main_thread: next: bh ffff880111e6a480 state 2
     irq/17-dwc3-3579  [003] d..1 21167.414594: bulk_out_complete: compl: bh ffff880111e6a480 state 1
    file-storage-3578  [000] .... 21167.414599: fsg_main_thread: next: bh ffff880111e6a480 state 1
    file-storage-3578  [002] .... 21167.416994: fsg_main_thread: next: bh ffff880111e6b4c0 state 2
     irq/17-dwc3-3579  [003] d..1 21167.417176: bulk_out_complete: compl: bh ffff880111e6b4c0 state 1
    file-storage-3578  [000] .... 21167.417180: fsg_main_thread: next: bh ffff880111e6b4c0 state 1
    file-storage-3578  [000] .... 21167.419551: fsg_main_thread: next: bh ffff880111e68540 state 2
     irq/17-dwc3-3579  [003] d..1 21167.419747: bulk_out_complete: compl: bh ffff880111e68540 state 1
    file-storage-3578  [000] .... 21167.419752: fsg_main_thread: next: bh ffff880111e68540 state 1
    file-storage-3578  [000] .... 21167.422125: fsg_main_thread: next: bh ffff880111e69580 state 2
     irq/17-dwc3-3579  [003] d..1 21167.422336: bulk_out_complete: compl: bh ffff880111e69580 state 1
    file-storage-3578  [000] .... 21167.422339: fsg_main_thread: next: bh ffff880111e69580 state 1
    file-storage-3578  [000] .... 21167.424708: fsg_main_thread: next: bh ffff880111e6a5c0 state 2
     irq/17-dwc3-3579  [003] d..1 21167.424903: bulk_out_complete: compl: bh ffff880111e6a5c0 state 1
    file-storage-3578  [000] .... 21167.424907: fsg_main_thread: next: bh ffff880111e6a5c0 state 1
    file-storage-3578  [002] .... 21167.427376: fsg_main_thread: next: bh ffff880111e6b600 state 2
     irq/17-dwc3-3579  [003] d..1 21167.427434: bulk_out_complete: compl: bh ffff880111e6b600 state 1
    file-storage-3578  [002] .... 21167.427437: fsg_main_thread: next: bh ffff880111e6b600 state 1
    file-storage-3578  [002] .... 21167.429791: fsg_main_thread: next: bh ffff880111e68680 state 2
     irq/17-dwc3-3579  [003] d..1 21167.430006: bulk_out_complete: compl: bh ffff880111e68680 state 1
    file-storage-3578  [002] .... 21167.430010: fsg_main_thread: next: bh ffff880111e68680 state 1
    file-storage-3578  [002] .... 21167.432386: fsg_main_thread: next: bh ffff880111e696c0 state 2
     irq/17-dwc3-3579  [003] d..1 21167.432577: bulk_out_complete: compl: bh ffff880111e696c0 state 1
    file-storage-3578  [002] .... 21167.432581: fsg_main_thread: next: bh ffff880111e696c0 state 1
    file-storage-3578  [002] .... 21167.434936: fsg_main_thread: next: bh ffff880111e6a700 state 2
     irq/17-dwc3-3579  [003] d..1 21167.435108: bulk_out_complete: compl: bh ffff880111e6a700 state 1
    file-storage-3578  [002] .... 21167.435112: fsg_main_thread: next: bh ffff880111e6a700 state 1
    file-storage-3578  [002] .... 21167.437460: fsg_main_thread: next: bh ffff880111e6b740 state 2
     irq/17-dwc3-3579  [003] d..1 21167.437674: bulk_out_complete: compl: bh ffff880111e6b740 state 1
    file-storage-3578  [002] .... 21167.437678: fsg_main_thread: next: bh ffff880111e6b740 state 1
    file-storage-3578  [000] .... 21167.440041: fsg_main_thread: next: bh ffff880111e687c0 state 2
     irq/17-dwc3-3579  [003] d..1 21167.440259: bulk_out_complete: compl: bh ffff880111e687c0 state 1
    file-storage-3578  [000] .... 21167.440264: fsg_main_thread: next: bh ffff880111e687c0 state 1
    file-storage-3578  [000] .... 21167.442617: fsg_main_thread: next: bh ffff880111e69800 state 2
     irq/17-dwc3-3579  [003] d..1 21167.442789: bulk_out_complete: compl: bh ffff880111e69800 state 1
    file-storage-3578  [000] .... 21167.442792: fsg_main_thread: next: bh ffff880111e69800 state 1
    file-storage-3578  [002] .... 21167.445191: fsg_main_thread: next: bh ffff880111e6a840 state 2
     irq/17-dwc3-3579  [003] d..1 21167.445408: bulk_out_complete: compl: bh ffff880111e6a840 state 1
    file-storage-3578  [002] .... 21167.445412: fsg_main_thread: next: bh ffff880111e6a840 state 1
    file-storage-3578  [000] .... 21167.447774: fsg_main_thread: next: bh ffff880111e6b880 state 2
     irq/17-dwc3-3579  [003] d..1 21167.447950: bulk_out_complete: compl: bh ffff880111e6b880 state 1
    file-storage-3578  [000] .... 21167.447954: fsg_main_thread: next: bh ffff880111e6b880 state 1
    file-storage-3578  [000] .... 21167.450326: fsg_main_thread: next: bh ffff880111e68900 state 2
     irq/17-dwc3-3579  [003] d..1 21167.450496: bulk_out_complete: compl: bh ffff880111e68900 state 1
    file-storage-3578  [000] .... 21167.450499: fsg_main_thread: next: bh ffff880111e68900 state 1
    file-storage-3578  [002] .... 21167.452862: fsg_main_thread: next: bh ffff880111e69940 state 2
     irq/17-dwc3-3579  [003] d..1 21167.453039: bulk_out_complete: compl: bh ffff880111e69940 state 1
    file-storage-3578  [002] .... 21167.453043: fsg_main_thread: next: bh ffff880111e69940 state 1
    file-storage-3578  [000] .... 21167.455350: fsg_main_thread: next: bh ffff880111e6a980 state 2
    file-storage-3578  [000] .... 21167.455415: fsg_main_thread: next: bh ffff880111e6a980 state 2
     irq/17-dwc3-3579  [003] d..1 21167.455618: bulk_out_complete: compl: bh ffff880111e6a980 state 1
    file-storage-3578  [000] .... 21167.455622: fsg_main_thread: next: bh ffff880111e6a980 state 1
    file-storage-3578  [002] .... 21167.457982: fsg_main_thread: next: bh ffff880111e6b9c0 state 2
     irq/17-dwc3-3579  [003] d..1 21167.458160: bulk_out_complete: compl: bh ffff880111e6b9c0 state 1
    file-storage-3578  [002] .... 21167.458163: fsg_main_thread: next: bh ffff880111e6b9c0 state 1
    file-storage-3578  [002] .... 21167.460526: fsg_main_thread: next: bh ffff880111e68a40 state 2
     irq/17-dwc3-3579  [003] d..1 21167.460702: bulk_out_complete: compl: bh ffff880111e68a40 state 1
    file-storage-3578  [002] .... 21167.460705: fsg_main_thread: next: bh ffff880111e68a40 state 1
    file-storage-3578  [002] .... 21167.463087: fsg_main_thread: next: bh ffff880111e69a80 state 2
     irq/17-dwc3-3579  [003] d..1 21167.463300: bulk_out_complete: compl: bh ffff880111e69a80 state 1
    file-storage-3578  [002] .... 21167.463304: fsg_main_thread: next: bh ffff880111e69a80 state 1
    file-storage-3578  [002] .... 21167.465667: fsg_main_thread: next: bh ffff880111e6aac0 state 2
     irq/17-dwc3-3579  [003] d..1 21167.465839: bulk_out_complete: compl: bh ffff880111e6aac0 state 1
    file-storage-3578  [002] .... 21167.465843: fsg_main_thread: next: bh ffff880111e6aac0 state 1
    file-storage-3578  [002] .... 21167.468201: fsg_main_thread: next: bh ffff880111e6bb00 state 2
     irq/17-dwc3-3579  [003] d..1 21167.468384: bulk_out_complete: compl: bh ffff880111e6bb00 state 1
    file-storage-3578  [002] .... 21167.468388: fsg_main_thread: next: bh ffff880111e6bb00 state 1
    file-storage-3578  [000] .... 21167.470770: fsg_main_thread: next: bh ffff880111e68b80 state 2
     irq/17-dwc3-3579  [003] d..1 21167.471021: bulk_out_complete: compl: bh ffff880111e68b80 state 1
    file-storage-3578  [000] .... 21167.471026: fsg_main_thread: next: bh ffff880111e68b80 state 1
    file-storage-3578  [000] .... 21167.473417: fsg_main_thread: next: bh ffff880111e69bc0 state 2
     irq/17-dwc3-3579  [003] d..1 21167.473642: bulk_out_complete: compl: bh ffff880111e69bc0 state 1
    file-storage-3578  [000] .... 21167.473646: fsg_main_thread: next: bh ffff880111e69bc0 state 1
    file-storage-3578  [000] .... 21167.476024: fsg_main_thread: next: bh ffff880111e6ac00 state 2
     irq/17-dwc3-3579  [003] d..1 21167.476198: bulk_out_complete: compl: bh ffff880111e6ac00 state 1
    file-storage-3578  [000] .... 21167.476202: fsg_main_thread: next: bh ffff880111e6ac00 state 1
    file-storage-3578  [002] .... 21167.478566: fsg_main_thread: next: bh ffff880111e6bc40 state 2
     irq/17-dwc3-3579  [003] d..1 21167.478734: bulk_out_complete: compl: bh ffff880111e6bc40 state 1
    file-storage-3578  [002] .... 21167.478738: fsg_main_thread: next: bh ffff880111e6bc40 state 1
    file-storage-3578  [002] .... 21167.481112: fsg_main_thread: next: bh ffff880111e68cc0 state 2
     irq/17-dwc3-3579  [003] d..1 21167.481290: bulk_out_complete: compl: bh ffff880111e68cc0 state 1
    file-storage-3578  [002] .... 21167.481293: fsg_main_thread: next: bh ffff880111e68cc0 state 1
    file-storage-3578  [002] .... 21167.483676: fsg_main_thread: next: bh ffff880111e69d00 state 2
     irq/17-dwc3-3579  [003] d..1 21167.483847: bulk_out_complete: compl: bh ffff880111e69d00 state 1
    file-storage-3578  [002] .... 21167.483851: fsg_main_thread: next: bh ffff880111e69d00 state 1
    file-storage-3578  [002] .... 21167.486217: fsg_main_thread: next: bh ffff880111e6ad40 state 2
     irq/17-dwc3-3579  [003] d..1 21167.486391: bulk_out_complete: compl: bh ffff880111e6ad40 state 1
    file-storage-3578  [002] .... 21167.486395: fsg_main_thread: next: bh ffff880111e6ad40 state 1
    file-storage-3578  [002] .... 21167.488766: fsg_main_thread: next: bh ffff880111e6bd80 state 2
     irq/17-dwc3-3579  [003] d..1 21167.488950: bulk_out_complete: compl: bh ffff880111e6bd80 state 1
    file-storage-3578  [002] .... 21167.488953: fsg_main_thread: next: bh ffff880111e6bd80 state 1
    file-storage-3578  [002] .... 21167.491346: fsg_main_thread: next: bh ffff880111e68e00 state 2
     irq/17-dwc3-3579  [003] d..1 21167.491512: bulk_out_complete: compl: bh ffff880111e68e00 state 1
    file-storage-3578  [002] .... 21167.491516: fsg_main_thread: next: bh ffff880111e68e00 state 1
    file-storage-3578  [002] .... 21167.493906: fsg_main_thread: next: bh ffff880111e69e40 state 2
     irq/17-dwc3-3579  [003] d..1 21167.494125: bulk_out_complete: compl: bh ffff880111e69e40 state 1
    file-storage-3578  [002] .... 21167.494129: fsg_main_thread: next: bh ffff880111e69e40 state 1
    file-storage-3578  [000] .... 21167.496425: fsg_main_thread: next: bh ffff880111e6ae80 state 2
    file-storage-3578  [000] .... 21167.496489: fsg_main_thread: next: bh ffff880111e6ae80 state 2
     irq/17-dwc3-3579  [003] d..1 21167.496718: bulk_out_complete: compl: bh ffff880111e6ae80 state 1
    file-storage-3578  [000] .... 21167.496722: fsg_main_thread: next: bh ffff880111e6ae80 state 1
    file-storage-3578  [000] .... 21167.499096: fsg_main_thread: next: bh ffff880111e6bec0 state 2
     irq/17-dwc3-3579  [003] d..1 21167.499280: bulk_out_complete: compl: bh ffff880111e6bec0 state 1
    file-storage-3578  [000] .... 21167.499285: fsg_main_thread: next: bh ffff880111e6bec0 state 1
    file-storage-3578  [000] .... 21167.501779: fsg_main_thread: next: bh ffff880111e68f40 state 2
     irq/17-dwc3-3579  [003] d..1 21167.501831: bulk_out_complete: compl: bh ffff880111e68f40 state 1
    file-storage-3578  [000] .... 21167.501835: fsg_main_thread: next: bh ffff880111e68f40 state 1
    file-storage-3578  [000] .... 21167.504206: fsg_main_thread: next: bh ffff880111e69f80 state 2
     irq/17-dwc3-3579  [003] d..1 21167.504406: bulk_out_complete: compl: bh ffff880111e69f80 state 1
    file-storage-3578  [000] .... 21167.504410: fsg_main_thread: next: bh ffff880111e69f80 state 1
    file-storage-3578  [000] .... 21167.506918: fsg_main_thread: next: bh ffff880111e6afc0 state 2
     irq/17-dwc3-3579  [003] d..1 21167.506971: bulk_out_complete: compl: bh ffff880111e6afc0 state 1
    file-storage-3578  [000] .... 21167.506974: fsg_main_thread: next: bh ffff880111e6afc0 state 1
    file-storage-3578  [000] .... 21167.509360: fsg_main_thread: next: bh ffff880111e68040 state 2
     irq/17-dwc3-3579  [003] d..1 21167.509568: bulk_out_complete: compl: bh ffff880111e68040 state 1
    file-storage-3578  [000] .... 21167.509572: fsg_main_thread: next: bh ffff880111e68040 state 1
    file-storage-3578  [002] .... 21167.511933: fsg_main_thread: next: bh ffff880111e69080 state 2
     irq/17-dwc3-3579  [003] d..1 21167.512110: bulk_out_complete: compl: bh ffff880111e69080 state 1
    file-storage-3578  [002] .... 21167.512114: fsg_main_thread: next: bh ffff880111e69080 state 1
    file-storage-3578  [002] .... 21167.514514: fsg_main_thread: next: bh ffff880111e6a0c0 state 2
     irq/17-dwc3-3579  [003] d..1 21167.514689: bulk_out_complete: compl: bh ffff880111e6a0c0 state 1
    file-storage-3578  [002] .... 21167.514692: fsg_main_thread: next: bh ffff880111e6a0c0 state 1
    file-storage-3578  [002] .... 21167.517039: fsg_main_thread: next: bh ffff880111e6b100 state 2
     irq/17-dwc3-3579  [003] d..1 21167.517245: bulk_out_complete: compl: bh ffff880111e6b100 state 1
    file-storage-3578  [002] .... 21167.517249: fsg_main_thread: next: bh ffff880111e6b100 state 1
    file-storage-3578  [000] .... 21167.519659: fsg_main_thread: next: bh ffff880111e68180 state 2
     irq/17-dwc3-3579  [003] d..1 21167.519825: bulk_out_complete: compl: bh ffff880111e68180 state 1
    file-storage-3578  [000] .... 21167.519829: fsg_main_thread: next: bh ffff880111e68180 state 1
    file-storage-3578  [002] .... 21167.522186: fsg_main_thread: next: bh ffff880111e691c0 state 2
     irq/17-dwc3-3579  [003] d..1 21167.522354: bulk_out_complete: compl: bh ffff880111e691c0 state 1
    file-storage-3578  [002] .... 21167.522357: fsg_main_thread: next: bh ffff880111e691c0 state 1
    file-storage-3578  [002] .... 21167.524702: fsg_main_thread: next: bh ffff880111e6a200 state 2
     irq/17-dwc3-3579  [003] d..1 21167.524906: bulk_out_complete: compl: bh ffff880111e6a200 state 1
    file-storage-3578  [002] .... 21167.524910: fsg_main_thread: next: bh ffff880111e6a200 state 1
    file-storage-3578  [002] .... 21167.527194: fsg_main_thread: next: bh ffff880111e6b240 state 2
    file-storage-3578  [000] .... 21167.527260: fsg_main_thread: next: bh ffff880111e6b240 state 2
     irq/17-dwc3-3579  [003] d..1 21167.527482: bulk_out_complete: compl: bh ffff880111e6b240 state 1
    file-storage-3578  [000] .... 21167.527486: fsg_main_thread: next: bh ffff880111e6b240 state 1
    file-storage-3578  [000] .... 21167.529849: fsg_main_thread: next: bh ffff880111e682c0 state 2
     irq/17-dwc3-3579  [003] d..1 21167.530016: bulk_out_complete: compl: bh ffff880111e682c0 state 1
    file-storage-3578  [000] .... 21167.530020: fsg_main_thread: next: bh ffff880111e682c0 state 1
    file-storage-3578  [000] .... 21167.532405: fsg_main_thread: next: bh ffff880111e69300 state 2
     irq/17-dwc3-3579  [003] d..1 21167.532605: bulk_out_complete: compl: bh ffff880111e69300 state 1
    file-storage-3578  [000] .... 21167.532609: fsg_main_thread: next: bh ffff880111e69300 state 1
    file-storage-3578  [000] .... 21167.534996: fsg_main_thread: next: bh ffff880111e6a340 state 2
     irq/17-dwc3-3579  [003] d..1 21167.535213: bulk_out_complete: compl: bh ffff880111e6a340 state 1
    file-storage-3578  [000] .... 21167.535217: fsg_main_thread: next: bh ffff880111e6a340 state 1
    file-storage-3578  [000] .... 21167.537630: fsg_main_thread: next: bh ffff880111e6b380 state 2
     irq/17-dwc3-3579  [003] d..1 21167.537853: bulk_out_complete: compl: bh ffff880111e6b380 state 1
    file-storage-3578  [000] .... 21167.537858: fsg_main_thread: next: bh ffff880111e6b380 state 1
    file-storage-3578  [000] .... 21167.540222: fsg_main_thread: next: bh ffff880111e68400 state 2
     irq/17-dwc3-3579  [003] d..1 21167.540391: bulk_out_complete: compl: bh ffff880111e68400 state 1
    file-storage-3578  [000] .... 21167.540395: fsg_main_thread: next: bh ffff880111e68400 state 1
    file-storage-3578  [000] .... 21167.542775: fsg_main_thread: next: bh ffff880111e69440 state 2
     irq/17-dwc3-3579  [003] d..1 21167.542997: bulk_out_complete: compl: bh ffff880111e69440 state 1
    file-storage-3578  [000] .... 21167.543000: fsg_main_thread: next: bh ffff880111e69440 state 1
    file-storage-3578  [000] .... 21167.545375: fsg_main_thread: next: bh ffff880111e6a480 state 2
     irq/17-dwc3-3579  [003] d..1 21167.545593: bulk_out_complete: compl: bh ffff880111e6a480 state 1
    file-storage-3578  [000] .... 21167.545597: fsg_main_thread: next: bh ffff880111e6a480 state 1
    file-storage-3578  [002] .... 21167.547962: fsg_main_thread: next: bh ffff880111e6b4c0 state 2
     irq/17-dwc3-3579  [003] d..1 21167.548155: bulk_out_complete: compl: bh ffff880111e6b4c0 state 1
    file-storage-3578  [002] .... 21167.548159: fsg_main_thread: next: bh ffff880111e6b4c0 state 1
    file-storage-3578  [000] .... 21167.550548: fsg_main_thread: next: bh ffff880111e68540 state 2
     irq/17-dwc3-3579  [003] d..1 21167.550743: bulk_out_complete: compl: bh ffff880111e68540 state 1
    file-storage-3578  [000] .... 21167.550750: fsg_main_thread: next: bh ffff880111e68540 state 1
    file-storage-3578  [000] .... 21167.553207: fsg_main_thread: next: bh ffff880111e69580 state 2
     irq/17-dwc3-3579  [003] d..1 21167.553440: bulk_out_complete: compl: bh ffff880111e69580 state 1
    file-storage-3578  [000] .... 21167.553444: fsg_main_thread: next: bh ffff880111e69580 state 1
    file-storage-3578  [000] .... 21167.555742: fsg_main_thread: next: bh ffff880111e6a5c0 state 2
    file-storage-3578  [000] .... 21167.555807: fsg_main_thread: next: bh ffff880111e6a5c0 state 2
     irq/17-dwc3-3579  [003] d..1 21167.555996: bulk_out_complete: compl: bh ffff880111e6a5c0 state 1
    file-storage-3578  [000] .... 21167.556001: fsg_main_thread: next: bh ffff880111e6a5c0 state 1
    file-storage-3578  [000] .... 21167.558364: fsg_main_thread: next: bh ffff880111e6b600 state 2
     irq/17-dwc3-3579  [003] d..1 21167.558542: bulk_out_complete: compl: bh ffff880111e6b600 state 1
    file-storage-3578  [000] .... 21167.558546: fsg_main_thread: next: bh ffff880111e6b600 state 1
    file-storage-3578  [000] .... 21167.560901: fsg_main_thread: next: bh ffff880111e68680 state 2
     irq/17-dwc3-3579  [003] d..1 21167.561077: bulk_out_complete: compl: bh ffff880111e68680 state 1
    file-storage-3578  [000] .... 21167.561081: fsg_main_thread: next: bh ffff880111e68680 state 1
    file-storage-3578  [000] .... 21167.563485: fsg_main_thread: next: bh ffff880111e696c0 state 2
     irq/17-dwc3-3579  [003] d..1 21167.563703: bulk_out_complete: compl: bh ffff880111e696c0 state 1
    file-storage-3578  [000] .... 21167.563707: fsg_main_thread: next: bh ffff880111e696c0 state 1
    file-storage-3578  [000] .... 21167.565997: fsg_main_thread: next: bh ffff880111e6a700 state 2
    file-storage-3578  [000] .... 21167.566061: fsg_main_thread: next: bh ffff880111e6a700 state 2
     irq/17-dwc3-3579  [003] d..1 21167.566340: bulk_out_complete: compl: bh ffff880111e6a700 state 1
    file-storage-3578  [000] .... 21167.566345: fsg_main_thread: next: bh ffff880111e6a700 state 1
    file-storage-3578  [000] .... 21167.568795: fsg_main_thread: next: bh ffff880111e6b740 state 2
     irq/17-dwc3-3579  [003] d..1 21167.569033: bulk_out_complete: compl: bh ffff880111e6b740 state 1
    file-storage-3578  [000] .... 21167.569037: fsg_main_thread: next: bh ffff880111e6b740 state 1
    file-storage-3578  [000] .... 21167.571427: fsg_main_thread: next: bh ffff880111e687c0 state 2
     irq/17-dwc3-3579  [003] d..1 21167.571614: bulk_out_complete: compl: bh ffff880111e687c0 state 1
    file-storage-3578  [000] .... 21167.571618: fsg_main_thread: next: bh ffff880111e687c0 state 1
    file-storage-3578  [000] .... 21167.573990: fsg_main_thread: next: bh ffff880111e69800 state 2
     irq/17-dwc3-3579  [003] d..1 21167.574167: bulk_out_complete: compl: bh ffff880111e69800 state 1
    file-storage-3578  [000] .... 21167.574172: fsg_main_thread: next: bh ffff880111e69800 state 1
    file-storage-3578  [000] .... 21167.576560: fsg_main_thread: next: bh ffff880111e6a840 state 2
     irq/17-dwc3-3579  [003] d..1 21167.576777: bulk_out_complete: compl: bh ffff880111e6a840 state 1
    file-storage-3578  [000] .... 21167.576782: fsg_main_thread: next: bh ffff880111e6a840 state 1
    file-storage-3578  [002] .... 21167.579170: fsg_main_thread: next: bh ffff880111e6b880 state 2
     irq/17-dwc3-3579  [003] d..1 21167.579347: bulk_out_complete: compl: bh ffff880111e6b880 state 1
    file-storage-3578  [002] .... 21167.579350: fsg_main_thread: next: bh ffff880111e6b880 state 1
    file-storage-3578  [002] .... 21167.581733: fsg_main_thread: next: bh ffff880111e68900 state 2
     irq/17-dwc3-3579  [003] d..1 21167.581942: bulk_out_complete: compl: bh ffff880111e68900 state 1
    file-storage-3578  [000] .... 21167.581948: fsg_main_thread: next: bh ffff880111e68900 state 1
    file-storage-3578  [000] .... 21167.584321: fsg_main_thread: next: bh ffff880111e69940 state 2
     irq/17-dwc3-3579  [003] d..1 21167.584528: bulk_out_complete: compl: bh ffff880111e69940 state 1
    file-storage-3578  [000] .... 21167.584532: fsg_main_thread: next: bh ffff880111e69940 state 1
    file-storage-3578  [000] .... 21167.586814: fsg_main_thread: next: bh ffff880111e6a980 state 2
    file-storage-3578  [000] .... 21167.587040: fsg_main_thread: next: bh ffff880111e6a980 state 2
     irq/17-dwc3-3579  [003] d..1 21167.587100: bulk_out_complete: compl: bh ffff880111e6a980 state 1
    file-storage-3578  [000] .... 21167.587104: fsg_main_thread: next: bh ffff880111e6a980 state 1
    file-storage-3578  [001] .... 21167.589396: fsg_main_thread: next: bh ffff880111e6b9c0 state 2
    file-storage-3578  [001] .... 21167.589638: fsg_main_thread: next: bh ffff880111e6b9c0 state 2
     irq/17-dwc3-3579  [003] d..1 21167.589687: bulk_out_complete: compl: bh ffff880111e6b9c0 state 1
    file-storage-3578  [001] .... 21167.589695: fsg_main_thread: next: bh ffff880111e6b9c0 state 1
    file-storage-3578  [001] .... 21167.592084: fsg_main_thread: next: bh ffff880111e68a40 state 2
     irq/17-dwc3-3579  [003] d..1 21167.592344: bulk_out_complete: compl: bh ffff880111e68a40 state 1
    file-storage-3578  [001] .... 21167.592349: fsg_main_thread: next: bh ffff880111e68a40 state 1
    file-storage-3578  [000] .... 21167.594734: fsg_main_thread: next: bh ffff880111e69a80 state 2
     irq/17-dwc3-3579  [003] d..1 21167.594902: bulk_out_complete: compl: bh ffff880111e69a80 state 1
    file-storage-3578  [000] .... 21167.594905: fsg_main_thread: next: bh ffff880111e69a80 state 1
    file-storage-3578  [002] .... 21167.597277: fsg_main_thread: next: bh ffff880111e6aac0 state 2
     irq/17-dwc3-3579  [003] d..1 21167.597498: bulk_out_complete: compl: bh ffff880111e6aac0 state 1
    file-storage-3578  [002] .... 21167.597503: fsg_main_thread: next: bh ffff880111e6aac0 state 1
    file-storage-3578  [002] .... 21167.599871: fsg_main_thread: next: bh ffff880111e6bb00 state 2
     irq/17-dwc3-3579  [003] d..1 21167.600051: bulk_out_complete: compl: bh ffff880111e6bb00 state 1
    file-storage-3578  [002] .... 21167.600055: fsg_main_thread: next: bh ffff880111e6bb00 state 1
    file-storage-3578  [000] .... 21167.602428: fsg_main_thread: next: bh ffff880111e68b80 state 2
     irq/17-dwc3-3579  [003] d..1 21167.602612: bulk_out_complete: compl: bh ffff880111e68b80 state 1
    file-storage-3578  [000] .... 21167.602616: fsg_main_thread: next: bh ffff880111e68b80 state 1
    file-storage-3578  [002] .... 21167.604996: fsg_main_thread: next: bh ffff880111e69bc0 state 2
     irq/17-dwc3-3579  [003] d..1 21167.605196: bulk_out_complete: compl: bh ffff880111e69bc0 state 1
    file-storage-3578  [002] .... 21167.605200: fsg_main_thread: next: bh ffff880111e69bc0 state 1
    file-storage-3578  [002] .... 21167.607854: fsg_main_thread: next: bh ffff880111e6ac00 state 2
     irq/17-dwc3-3579  [003] d..1 21167.607903: bulk_out_complete: compl: bh ffff880111e6ac00 state 1
    file-storage-3578  [002] .... 21167.607908: fsg_main_thread: next: bh ffff880111e6ac00 state 1
    file-storage-3578  [002] .... 21167.610293: fsg_main_thread: next: bh ffff880111e6bc40 state 2
     irq/17-dwc3-3579  [003] d..1 21167.610481: bulk_out_complete: compl: bh ffff880111e6bc40 state 1
    file-storage-3578  [002] .... 21167.610485: fsg_main_thread: next: bh ffff880111e6bc40 state 1
    file-storage-3578  [002] .... 21167.612872: fsg_main_thread: next: bh ffff880111e68cc0 state 2
     irq/17-dwc3-3579  [003] d..1 21167.613057: bulk_out_complete: compl: bh ffff880111e68cc0 state 1
    file-storage-3578  [002] .... 21167.613061: fsg_main_thread: next: bh ffff880111e68cc0 state 1
    file-storage-3578  [002] .... 21167.615439: fsg_main_thread: next: bh ffff880111e69d00 state 2
     irq/17-dwc3-3579  [003] d..1 21167.615622: bulk_out_complete: compl: bh ffff880111e69d00 state 1
    file-storage-3578  [002] .... 21167.615626: fsg_main_thread: next: bh ffff880111e69d00 state 1
    file-storage-3578  [000] .... 21167.618013: fsg_main_thread: next: bh ffff880111e6ad40 state 2
     irq/17-dwc3-3579  [003] d..1 21167.618241: bulk_out_complete: compl: bh ffff880111e6ad40 state 1
    file-storage-3578  [000] .... 21167.618245: fsg_main_thread: next: bh ffff880111e6ad40 state 1
    file-storage-3578  [002] .... 21167.620629: fsg_main_thread: next: bh ffff880111e6bd80 state 2
     irq/17-dwc3-3579  [003] d..1 21167.620821: bulk_out_complete: compl: bh ffff880111e6bd80 state 1
    file-storage-3578  [002] .... 21167.620828: fsg_main_thread: next: bh ffff880111e6bd80 state 1
    file-storage-3578  [002] .... 21167.623233: fsg_main_thread: next: bh ffff880111e68e00 state 2
     irq/17-dwc3-3579  [003] d..1 21167.623430: bulk_out_complete: compl: bh ffff880111e68e00 state 1
    file-storage-3578  [002] .... 21167.623435: fsg_main_thread: next: bh ffff880111e68e00 state 1
    file-storage-3578  [002] .... 21167.625741: fsg_main_thread: next: bh ffff880111e69e40 state 2
    file-storage-3578  [000] .... 21167.625808: fsg_main_thread: next: bh ffff880111e69e40 state 2
     irq/17-dwc3-3579  [003] d..1 21167.625997: bulk_out_complete: compl: bh ffff880111e69e40 state 1
    file-storage-3578  [000] .... 21167.626001: fsg_main_thread: next: bh ffff880111e69e40 state 1
    file-storage-3578  [000] .... 21167.628425: fsg_main_thread: next: bh ffff880111e6ae80 state 2
     irq/17-dwc3-3579  [003] d..1 21167.628646: bulk_out_complete: compl: bh ffff880111e6ae80 state 1
    file-storage-3578  [000] .... 21167.628650: fsg_main_thread: next: bh ffff880111e6ae80 state 1
    file-storage-3578  [000] .... 21167.631048: fsg_main_thread: next: bh ffff880111e6bec0 state 2
     irq/17-dwc3-3579  [003] d..1 21167.631272: bulk_out_complete: compl: bh ffff880111e6bec0 state 1
    file-storage-3578  [000] .... 21167.631276: fsg_main_thread: next: bh ffff880111e6bec0 state 1
    file-storage-3578  [000] .... 21167.633650: fsg_main_thread: next: bh ffff880111e68f40 state 2
     irq/17-dwc3-3579  [003] d..1 21167.633834: bulk_out_complete: compl: bh ffff880111e68f40 state 1
    file-storage-3578  [000] .... 21167.633837: fsg_main_thread: next: bh ffff880111e68f40 state 1
    file-storage-3578  [002] .... 21167.636199: fsg_main_thread: next: bh ffff880111e69f80 state 2
     irq/17-dwc3-3579  [003] d..1 21167.636413: bulk_out_complete: compl: bh ffff880111e69f80 state 1
    file-storage-3578  [002] .... 21167.636418: fsg_main_thread: next: bh ffff880111e69f80 state 1
    file-storage-3578  [002] .... 21167.638954: fsg_main_thread: next: bh ffff880111e6afc0 state 2
     irq/17-dwc3-3579  [003] d..1 21167.639017: bulk_out_complete: compl: bh ffff880111e6afc0 state 1
    file-storage-3578  [002] .... 21167.639021: fsg_main_thread: next: bh ffff880111e6afc0 state 1
    file-storage-3578  [002] .... 21167.641367: fsg_main_thread: next: bh ffff880111e68040 state 2
     irq/17-dwc3-3579  [003] d..1 21167.641551: bulk_out_complete: compl: bh ffff880111e68040 state 1
    file-storage-3578  [002] .... 21167.641555: fsg_main_thread: next: bh ffff880111e68040 state 1
    file-storage-3578  [002] .... 21167.643911: fsg_main_thread: next: bh ffff880111e69080 state 2
     irq/17-dwc3-3579  [003] d..1 21167.644091: bulk_out_complete: compl: bh ffff880111e69080 state 1
    file-storage-3578  [002] .... 21167.644095: fsg_main_thread: next: bh ffff880111e69080 state 1
    file-storage-3578  [002] .... 21167.646466: fsg_main_thread: next: bh ffff880111e6a0c0 state 2
     irq/17-dwc3-3579  [003] d..1 21167.646641: bulk_out_complete: compl: bh ffff880111e6a0c0 state 1
    file-storage-3578  [002] .... 21167.646644: fsg_main_thread: next: bh ffff880111e6a0c0 state 1
    file-storage-3578  [002] .... 21167.649025: fsg_main_thread: next: bh ffff880111e6b100 state 2
     irq/17-dwc3-3579  [003] d..1 21167.649216: bulk_out_complete: compl: bh ffff880111e6b100 state 1
    file-storage-3578  [002] .... 21167.649220: fsg_main_thread: next: bh ffff880111e6b100 state 1
    file-storage-3578  [002] .... 21167.651583: fsg_main_thread: next: bh ffff880111e68180 state 2
     irq/17-dwc3-3579  [003] d..1 21167.651766: bulk_out_complete: compl: bh ffff880111e68180 state 1
    file-storage-3578  [002] .... 21167.651771: fsg_main_thread: next: bh ffff880111e68180 state 1
    file-storage-3578  [002] .... 21167.654164: fsg_main_thread: next: bh ffff880111e691c0 state 2
     irq/17-dwc3-3579  [003] d..1 21167.654352: bulk_out_complete: compl: bh ffff880111e691c0 state 1
    file-storage-3578  [002] .... 21167.654357: fsg_main_thread: next: bh ffff880111e691c0 state 1
    file-storage-3578  [002] .... 21167.656735: fsg_main_thread: next: bh ffff880111e6a200 state 2
     irq/17-dwc3-3579  [003] d..1 21167.656957: bulk_out_complete: compl: bh ffff880111e6a200 state 1
    file-storage-3578  [002] .... 21167.656962: fsg_main_thread: next: bh ffff880111e6a200 state 1
    file-storage-3578  [002] .... 21167.659244: fsg_main_thread: next: bh ffff880111e6b240 state 2
    file-storage-3578  [002] .... 21167.659309: fsg_main_thread: next: bh ffff880111e6b240 state 2
     irq/17-dwc3-3579  [003] d..1 21167.659501: bulk_out_complete: compl: bh ffff880111e6b240 state 1
    file-storage-3578  [002] .... 21167.659505: fsg_main_thread: next: bh ffff880111e6b240 state 1
    file-storage-3578  [002] .... 21167.661818: fsg_main_thread: next: bh ffff880111e682c0 state 2
    file-storage-3578  [002] .... 21167.661883: fsg_main_thread: next: bh ffff880111e682c0 state 2
     irq/17-dwc3-3579  [003] d..1 21167.662073: bulk_out_complete: compl: bh ffff880111e682c0 state 1
    file-storage-3578  [002] .... 21167.662078: fsg_main_thread: next: bh ffff880111e682c0 state 1
    file-storage-3578  [002] .... 21167.664457: fsg_main_thread: next: bh ffff880111e69300 state 2
     irq/17-dwc3-3579  [003] d..1 21167.664634: bulk_out_complete: compl: bh ffff880111e69300 state 1
    file-storage-3578  [000] .... 21167.664641: fsg_main_thread: next: bh ffff880111e69300 state 1
    file-storage-3578  [000] .... 21167.667013: fsg_main_thread: next: bh ffff880111e6a340 state 2
     irq/17-dwc3-3579  [003] d..1 21167.667216: bulk_out_complete: compl: bh ffff880111e6a340 state 1
    file-storage-3578  [000] .... 21167.667221: fsg_main_thread: next: bh ffff880111e6a340 state 1
    file-storage-3578  [000] .... 21167.669540: fsg_main_thread: next: bh ffff880111e6b380 state 2
    file-storage-3578  [000] .... 21167.669856: fsg_main_thread: next: bh ffff880111e6b380 state 2
     irq/17-dwc3-3579  [003] d..1 21167.669929: bulk_out_complete: compl: bh ffff880111e6b380 state 1
    file-storage-3578  [000] .... 21167.669933: fsg_main_thread: next: bh ffff880111e6b380 state 1
    file-storage-3578  [000] .... 21167.672585: fsg_main_thread: next: bh ffff880111e68400 state 2
     irq/17-dwc3-3579  [003] d..1 21167.672652: bulk_out_complete: compl: bh ffff880111e68400 state 1
    file-storage-3578  [000] .... 21167.672658: fsg_main_thread: next: bh ffff880111e68400 state 1
    file-storage-3578  [000] .... 21167.675043: fsg_main_thread: next: bh ffff880111e69440 state 2
     irq/17-dwc3-3579  [003] d..1 21167.675270: bulk_out_complete: compl: bh ffff880111e69440 state 1
    file-storage-3578  [000] .... 21167.675275: fsg_main_thread: next: bh ffff880111e69440 state 1
    file-storage-3578  [000] .... 21167.677670: fsg_main_thread: next: bh ffff880111e6a480 state 2
     irq/17-dwc3-3579  [003] d..1 21167.677858: bulk_out_complete: compl: bh ffff880111e6a480 state 1
    file-storage-3578  [002] .... 21167.677864: fsg_main_thread: next: bh ffff880111e6a480 state 1
    file-storage-3578  [002] .... 21167.680223: fsg_main_thread: next: bh ffff880111e6b4c0 state 2
     irq/17-dwc3-3579  [003] d..1 21167.680404: bulk_out_complete: compl: bh ffff880111e6b4c0 state 1
    file-storage-3578  [002] .... 21167.680408: fsg_main_thread: next: bh ffff880111e6b4c0 state 1
    file-storage-3578  [002] .... 21167.682783: fsg_main_thread: next: bh ffff880111e68540 state 2
     irq/17-dwc3-3579  [003] d..1 21167.682965: bulk_out_complete: compl: bh ffff880111e68540 state 1
    file-storage-3578  [002] .... 21167.682969: fsg_main_thread: next: bh ffff880111e68540 state 1
    file-storage-3578  [002] .... 21167.685308: fsg_main_thread: next: bh ffff880111e69580 state 2
     irq/17-dwc3-3579  [003] d..1 21167.685528: bulk_out_complete: compl: bh ffff880111e69580 state 1
    file-storage-3578  [002] .... 21167.685532: fsg_main_thread: next: bh ffff880111e69580 state 1
    file-storage-3578  [002] .... 21167.687918: fsg_main_thread: next: bh ffff880111e6a5c0 state 2
     irq/17-dwc3-3579  [003] d..1 21167.688148: bulk_out_complete: compl: bh ffff880111e6a5c0 state 1
    file-storage-3578  [002] .... 21167.688152: fsg_main_thread: next: bh ffff880111e6a5c0 state 1
    file-storage-3578  [002] .... 21167.690581: fsg_main_thread: next: bh ffff880111e6b600 state 2
     irq/17-dwc3-3579  [003] d..1 21167.690773: bulk_out_complete: compl: bh ffff880111e6b600 state 1
    file-storage-3578  [002] .... 21167.690778: fsg_main_thread: next: bh ffff880111e6b600 state 1
    file-storage-3578  [002] .... 21167.693182: fsg_main_thread: next: bh ffff880111e68680 state 2
     irq/17-dwc3-3579  [003] d..1 21167.693360: bulk_out_complete: compl: bh ffff880111e68680 state 1
    file-storage-3578  [002] .... 21167.693364: fsg_main_thread: next: bh ffff880111e68680 state 1
    file-storage-3578  [002] .... 21167.695807: fsg_main_thread: next: bh ffff880111e696c0 state 2
     irq/17-dwc3-3579  [003] d..1 21167.695990: bulk_out_complete: compl: bh ffff880111e696c0 state 1
    file-storage-3578  [002] .... 21167.695994: fsg_main_thread: next: bh ffff880111e696c0 state 1
    file-storage-3578  [002] .... 21167.698368: fsg_main_thread: next: bh ffff880111e6a700 state 2
     irq/17-dwc3-3579  [003] d..1 21167.698544: bulk_out_complete: compl: bh ffff880111e6a700 state 1
    file-storage-3578  [002] .... 21167.698548: fsg_main_thread: next: bh ffff880111e6a700 state 1
    file-storage-3578  [002] .... 21167.700937: fsg_main_thread: next: bh ffff880111e6b740 state 2
     irq/17-dwc3-3579  [003] d..1 21167.701155: bulk_out_complete: compl: bh ffff880111e6b740 state 1
    file-storage-3578  [002] .... 21167.701159: fsg_main_thread: next: bh ffff880111e6b740 state 1
    file-storage-3578  [002] .... 21167.703553: fsg_main_thread: next: bh ffff880111e687c0 state 2
     irq/17-dwc3-3579  [003] d..1 21167.703773: bulk_out_complete: compl: bh ffff880111e687c0 state 1
    file-storage-3578  [002] .... 21167.703778: fsg_main_thread: next: bh ffff880111e687c0 state 1
    file-storage-3578  [002] .... 21167.706161: fsg_main_thread: next: bh ffff880111e69800 state 2
     irq/17-dwc3-3579  [003] d..1 21167.706364: bulk_out_complete: compl: bh ffff880111e69800 state 1
    file-storage-3578  [002] .... 21167.706368: fsg_main_thread: next: bh ffff880111e69800 state 1
    file-storage-3578  [002] .... 21167.708740: fsg_main_thread: next: bh ffff880111e6a840 state 2
     irq/17-dwc3-3579  [003] d..1 21167.708922: bulk_out_complete: compl: bh ffff880111e6a840 state 1
    file-storage-3578  [002] .... 21167.708926: fsg_main_thread: next: bh ffff880111e6a840 state 1
    file-storage-3578  [002] .... 21167.711338: fsg_main_thread: next: bh ffff880111e6b880 state 2
     irq/17-dwc3-3579  [003] d..1 21167.711519: bulk_out_complete: compl: bh ffff880111e6b880 state 1
    file-storage-3578  [002] .... 21167.711523: fsg_main_thread: next: bh ffff880111e6b880 state 1
    file-storage-3578  [002] .... 21167.713897: fsg_main_thread: next: bh ffff880111e68900 state 2
     irq/17-dwc3-3579  [003] d..1 21167.714076: bulk_out_complete: compl: bh ffff880111e68900 state 1
    file-storage-3578  [002] .... 21167.714080: fsg_main_thread: next: bh ffff880111e68900 state 1
    file-storage-3578  [002] .... 21167.716436: fsg_main_thread: next: bh ffff880111e69940 state 2
     irq/17-dwc3-3579  [003] d..1 21167.716639: bulk_out_complete: compl: bh ffff880111e69940 state 1
    file-storage-3578  [002] .... 21167.716643: fsg_main_thread: next: bh ffff880111e69940 state 1
    file-storage-3578  [002] .... 21167.719034: fsg_main_thread: next: bh ffff880111e6a980 state 2
     irq/17-dwc3-3579  [003] d..1 21167.719256: bulk_out_complete: compl: bh ffff880111e6a980 state 1
    file-storage-3578  [002] .... 21167.719260: fsg_main_thread: next: bh ffff880111e6a980 state 1
    file-storage-3578  [000] .... 21167.721547: fsg_main_thread: next: bh ffff880111e6b9c0 state 2
    file-storage-3578  [000] .... 21167.721612: fsg_main_thread: next: bh ffff880111e6b9c0 state 2
     irq/17-dwc3-3579  [003] d..1 21167.721855: bulk_out_complete: compl: bh ffff880111e6b9c0 state 1
    file-storage-3578  [000] .... 21167.721859: fsg_main_thread: next: bh ffff880111e6b9c0 state 1
    file-storage-3578  [000] .... 21167.724148: fsg_main_thread: next: bh ffff880111e68a40 state 2
    file-storage-3578  [000] .... 21167.724213: fsg_main_thread: next: bh ffff880111e68a40 state 2
     irq/17-dwc3-3579  [003] d..1 21167.724413: bulk_out_complete: compl: bh ffff880111e68a40 state 1
    file-storage-3578  [002] .... 21167.724419: fsg_main_thread: next: bh ffff880111e68a40 state 1
    file-storage-3578  [002] .... 21167.726827: fsg_main_thread: next: bh ffff880111e69a80 state 2
     irq/17-dwc3-3579  [003] d..1 21167.727066: bulk_out_complete: compl: bh ffff880111e69a80 state 1
    file-storage-3578  [002] .... 21167.727072: fsg_main_thread: next: bh ffff880111e69a80 state 1
    file-storage-3578  [002] .... 21167.729458: fsg_main_thread: next: bh ffff880111e6aac0 state 2
     irq/17-dwc3-3579  [003] d..1 21167.729666: bulk_out_complete: compl: bh ffff880111e6aac0 state 1
    file-storage-3578  [002] .... 21167.729670: fsg_main_thread: next: bh ffff880111e6aac0 state 1


-- 
balbi

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 800 bytes --]

Re: Memory barrier needed with wake_up_process()?

[Alan Stern]

On Fri, 9 Sep 2016, Felipe Balbi wrote:

> Finally :-) Here's the diff I used:
> 
> diff --git a/drivers/usb/gadget/function/f_mass_storage.c b/drivers/usb/gadget/function/f_mass_storage.c
> index 8f3659b65f53..0716024f6b65 100644
> --- a/drivers/usb/gadget/function/f_mass_storage.c
> +++ b/drivers/usb/gadget/function/f_mass_storage.c
> @@ -481,6 +481,8 @@ static void bulk_out_complete(struct usb_ep *ep, struct usb_request *req)
>         spin_lock(&common->lock);
>         bh->outreq_busy = 0;
>         bh->state = BUF_STATE_FULL;
> +       if (bh->bulk_out_intended_length == US_BULK_CB_WRAP_LEN)
> +               trace_printk("compl: bh %p state %d\n", bh, bh->state);
>         wakeup_thread(common);
>         spin_unlock(&common->lock);
>  }
> @@ -2208,6 +2210,7 @@ static int get_next_command(struct fsg_common *common)
>                 rc = sleep_thread(common, true);
>                 if (rc)
>                         return rc;
> +               trace_printk("next: bh %p state %d\n", bh, bh->state);
>         }
>         smp_rmb();
>         rc = fsg_is_set(common) ? received_cbw(common->fsg, bh) : -EIO;
> 
> 
> And here's trace output:
> 
> # tracer: nop
> #
> # entries-in-buffer/entries-written: 1002/1002   #P:4
> #
> #                              _-----=> irqs-off
> #                             / _----=> need-resched
> #                            | / _---=> hardirq/softirq
> #                            || / _--=> preempt-depth
> #                            ||| /     delay
> #           TASK-PID   CPU#  ||||    TIMESTAMP  FUNCTION
> #              | |       |   ||||       |         |
>     file-storage-3578  [000] .... 21166.789127: fsg_main_thread: next: bh ffff880111e69a00 state 2
>     file-storage-3578  [000] .... 21166.789312: fsg_main_thread: next: bh ffff880111e69a00 state 2
>      irq/17-dwc3-3579  [003] d..1 21166.789395: bulk_out_complete: compl: bh ffff880111e69a00 state 1
>     file-storage-3578  [000] .... 21166.789445: fsg_main_thread: next: bh ffff880111e69a00 state 1

Okay, that's normal.  2 = BUF_STATE_BUSY, 1 = BUF_STATE_FULL.  So we get woken
up a couple of times while the transfer is in progress (probably because some
earlier buffers have finished transferring), then the CBW transfer completes
and the buffer is read.

...

>     file-storage-3578  [002] .... 21167.726827: fsg_main_thread: next: bh ffff880111e69a80 state 2
>      irq/17-dwc3-3579  [003] d..1 21167.727066: bulk_out_complete: compl: bh ffff880111e69a80 state 1
>     file-storage-3578  [002] .... 21167.727072: fsg_main_thread: next: bh ffff880111e69a80 state 1
>     file-storage-3578  [002] .... 21167.729458: fsg_main_thread: next: bh ffff880111e6aac0 state 2
>      irq/17-dwc3-3579  [003] d..1 21167.729666: bulk_out_complete: compl: bh ffff880111e6aac0 state 1
>     file-storage-3578  [002] .... 21167.729670: fsg_main_thread: next: bh ffff880111e6aac0 state 1

And this is where everything stopped?

This also looks normal.  So the question is what happened when 
get_next_command() returned after this?

Felipe, maybe the patch below (in place of your current patch) will
help.  Since the events that it logs are all supposed to be unusual,
you can use printk if you want, but I wrote it with trace_printk.

Alan Stern



Index: usb-4.x/drivers/usb/gadget/function/f_mass_storage.c
===================================================================
--- usb-4.x.orig/drivers/usb/gadget/function/f_mass_storage.c
+++ usb-4.x/drivers/usb/gadget/function/f_mass_storage.c
@@ -415,6 +415,7 @@ static void raise_exception(struct fsg_c
 	 * If a lower-or-equal priority exception is in progress, preempt it
 	 * and notify the main thread by sending it a signal.
 	 */
+	trace_printk("raise_exception %d\n", new_state);
 	spin_lock_irqsave(&common->lock, flags);
 	if (common->state <= new_state) {
 		common->exception_req_tag = common->ep0_req_tag;
@@ -2495,6 +2496,7 @@ static void handle_exception(struct fsg_
 static int fsg_main_thread(void *common_)
 {
 	struct fsg_common	*common = common_;
+	int rc;
 
 	/*
 	 * Allow the thread to be killed by a signal, but set the signal mask
@@ -2518,6 +2520,7 @@ static int fsg_main_thread(void *common_
 	/* The main loop */
 	while (common->state != FSG_STATE_TERMINATED) {
 		if (exception_in_progress(common) || signal_pending(current)) {
+			trace_printk("handling exception\n");
 			handle_exception(common);
 			continue;
 		}
@@ -2527,24 +2530,38 @@ static int fsg_main_thread(void *common_
 			continue;
 		}
 
-		if (get_next_command(common))
+		rc = get_next_command(common);
+		if (rc) {
+			trace_printk("get_next_command -> %d\n", rc);
 			continue;
+		}
 
 		spin_lock_irq(&common->lock);
 		if (!exception_in_progress(common))
 			common->state = FSG_STATE_DATA_PHASE;
 		spin_unlock_irq(&common->lock);
 
-		if (do_scsi_command(common) || finish_reply(common))
+		rc = do_scsi_command(common);
+		if (rc) {
+			trace_printk("do_scsi_command -> %d\n", rc);
+			continue;
+		}
+		rc = finish_reply(common);
+		if (rc) {
+			trace_printk("finish_reply -> %d\n", rc);
 			continue;
+		}
 
 		spin_lock_irq(&common->lock);
 		if (!exception_in_progress(common))
 			common->state = FSG_STATE_STATUS_PHASE;
 		spin_unlock_irq(&common->lock);
 
-		if (send_status(common))
+		rc = send_status(common);
+		if (rc) {
+			trace_printk("send_status -> %d\n", rc);
 			continue;
+		}
 
 		spin_lock_irq(&common->lock);
 		if (!exception_in_progress(common))

Re: Memory barrier needed with wake_up_process()?

[Felipe Balbi]

[-- Attachment #1: Type: text/plain, Size: 9575 bytes --]


Hi Alan,

Alan Stern <stern@rowland.harvard.edu> writes:
> On Fri, 9 Sep 2016, Felipe Balbi wrote:
>
>> Finally :-) Here's the diff I used:
>> 
>> diff --git a/drivers/usb/gadget/function/f_mass_storage.c b/drivers/usb/gadget/function/f_mass_storage.c
>> index 8f3659b65f53..0716024f6b65 100644
>> --- a/drivers/usb/gadget/function/f_mass_storage.c
>> +++ b/drivers/usb/gadget/function/f_mass_storage.c
>> @@ -481,6 +481,8 @@ static void bulk_out_complete(struct usb_ep *ep, struct usb_request *req)
>>         spin_lock(&common->lock);
>>         bh->outreq_busy = 0;
>>         bh->state = BUF_STATE_FULL;
>> +       if (bh->bulk_out_intended_length == US_BULK_CB_WRAP_LEN)
>> +               trace_printk("compl: bh %p state %d\n", bh, bh->state);
>>         wakeup_thread(common);
>>         spin_unlock(&common->lock);
>>  }
>> @@ -2208,6 +2210,7 @@ static int get_next_command(struct fsg_common *common)
>>                 rc = sleep_thread(common, true);
>>                 if (rc)
>>                         return rc;
>> +               trace_printk("next: bh %p state %d\n", bh, bh->state);
>>         }
>>         smp_rmb();
>>         rc = fsg_is_set(common) ? received_cbw(common->fsg, bh) : -EIO;
>> 
>> 
>> And here's trace output:
>> 
>> # tracer: nop
>> #
>> # entries-in-buffer/entries-written: 1002/1002   #P:4
>> #
>> #                              _-----=> irqs-off
>> #                             / _----=> need-resched
>> #                            | / _---=> hardirq/softirq
>> #                            || / _--=> preempt-depth
>> #                            ||| /     delay
>> #           TASK-PID   CPU#  ||||    TIMESTAMP  FUNCTION
>> #              | |       |   ||||       |         |
>>     file-storage-3578  [000] .... 21166.789127: fsg_main_thread: next: bh ffff880111e69a00 state 2
>>     file-storage-3578  [000] .... 21166.789312: fsg_main_thread: next: bh ffff880111e69a00 state 2
>>      irq/17-dwc3-3579  [003] d..1 21166.789395: bulk_out_complete: compl: bh ffff880111e69a00 state 1
>>     file-storage-3578  [000] .... 21166.789445: fsg_main_thread: next: bh ffff880111e69a00 state 1
>
> Okay, that's normal.  2 = BUF_STATE_BUSY, 1 = BUF_STATE_FULL.  So we get woken
> up a couple of times while the transfer is in progress (probably because some
> earlier buffers have finished transferring), then the CBW transfer completes
> and the buffer is read.
>
> ...
>
>>     file-storage-3578  [002] .... 21167.726827: fsg_main_thread: next: bh ffff880111e69a80 state 2
>>      irq/17-dwc3-3579  [003] d..1 21167.727066: bulk_out_complete: compl: bh ffff880111e69a80 state 1
>>     file-storage-3578  [002] .... 21167.727072: fsg_main_thread: next: bh ffff880111e69a80 state 1
>>     file-storage-3578  [002] .... 21167.729458: fsg_main_thread: next: bh ffff880111e6aac0 state 2
>>      irq/17-dwc3-3579  [003] d..1 21167.729666: bulk_out_complete: compl: bh ffff880111e6aac0 state 1
>>     file-storage-3578  [002] .... 21167.729670: fsg_main_thread: next: bh ffff880111e6aac0 state 1
>
> And this is where everything stopped?

yeah, that's everything.

> This also looks normal.  So the question is what happened when 
> get_next_command() returned after this?
>
> Felipe, maybe the patch below (in place of your current patch) will
> help.  Since the events that it logs are all supposed to be unusual,
> you can use printk if you want, but I wrote it with trace_printk.

I've applied your patch and it wasn't giving me any output, which hinted
that g_mass_storage wasn't returning any failures. So I enabled dwc3's
traces to get more data out of it. Here's the final snippet (with
comments, again). Let me know if you want the entire thing (it's
~14MiB).

> irq/17-dwc3-3527  [003] d..1    34.215214: dwc3_event: event (00110301): Link Change [U1]
> irq/17-dwc3-3527  [003] d..1    34.215214: dwc3_event: event (00004086): ep1in: Transfer In-Progress
> irq/17-dwc3-3527  [003] d..1    34.215214: dwc3_complete_trb: ep1in: 2/255 trb ffff880084813e70 buf 0000000080808000 size 0 ctrl 00000c18 (hlcS:SC:normal)
> irq/17-dwc3-3527  [003] d..1    34.215214: dwc3_gadget_giveback: ep1in: req ffff88016d046840 length 16384/16384 zsI ==> 0
> irq/17-dwc3-3527  [003] d...    34.215215: usb_gadget_giveback_request: ep1in: length 16384/16384 sgs 0/0 stream 0 zsI status 0 --> 0
> irq/17-dwc3-3527  [003] d..1    34.215218: dwc3_gadget_ep_cmd: ep1in: cmd 'Update Transfer' [196615] params 00000000 00000000 00000000 --> status: Successful
> irq/17-dwc3-3527  [003] d..1    34.215281: dwc3_event: event (00100301): Link Change [U0]
> irq/17-dwc3-3527  [003] d..1    34.215281: dwc3_event: event (00110301): Link Change [U1]
> irq/17-dwc3-3527  [003] d..1    34.215282: dwc3_event: event (00004086): ep1in: Transfer In-Progress
> irq/17-dwc3-3527  [003] d..1    34.215282: dwc3_complete_trb: ep1in: 1/255 trb ffff880084813e80 buf 000000008080c000 size 0 ctrl 00000c18 (hlcS:SC:normal)
> irq/17-dwc3-3527  [003] d..1    34.215282: dwc3_gadget_giveback: ep1in: req ffff88016d046f00 length 13/13 zsI ==> 0
> irq/17-dwc3-3527  [003] d...    34.215283: usb_gadget_giveback_request: ep1in: length 13/13 sgs 0/0 stream 0 zsI status 0 --> 0
> irq/17-dwc3-3527  [003] d..1    34.215284: dwc3_event: event (00100301): Link Change [U0]
> irq/17-dwc3-3527  [003] d..1    34.215284: dwc3_event: event (00110301): Link Change [U1]
> irq/17-dwc3-3527  [003] d..1    34.215284: dwc3_event: event (00006084): ep1out: Transfer In-Progress
> irq/17-dwc3-3527  [003] d..1    34.215284: dwc3_complete_trb: ep1out: 1/255 trb ffff880084804320 buf 000000008080c800 size 993 ctrl 00000c18 (hlcS:SC:normal)
> irq/17-dwc3-3527  [003] d..1    34.215285: dwc3_gadget_giveback: ep1out: req ffff880154205300 length 31/1024 zsI ==> 0
> irq/17-dwc3-3527  [003] d...    34.215285: usb_gadget_giveback_request: ep1out: length 31/1024 sgs 0/0 stream 0 zsI status 0 --> 0

completed and gave back CBW.

> irq/17-dwc3-3527  [003] d..1    34.215349: dwc3_event: event (00100301): Link Change [U0]
> irq/17-dwc3-3527  [003] d..1    34.215349: dwc3_event: event (00110301): Link Change [U1]
> irq/17-dwc3-3527  [003] d..1    34.225616: dwc3_event: event (00120301): Link Change [U2]
> irq/17-dwc3-3527  [003] d...    34.225617: usb_gadget_vbus_draw: speed 5/5 state 7 0mA [sg:out_aligned:self-powered:activated:connected] --> -95
> irq/17-dwc3-3527  [003] d..1    64.832221: dwc3_event: event (00100301): Link Change [U0]

30 seconds of nothing.

> irq/17-dwc3-3527  [003] d..1    64.832240: dwc3_event: event (0000c040): ep0out: Transfer Complete
> irq/17-dwc3-3527  [003] d..1    64.832243: dwc3_ep0: ep0out: Transfer Complete: state 'Setup Phase'
> irq/17-dwc3-3527  [003] d..1    64.832243: dwc3_ep0: Setup Phase
> irq/17-dwc3-3527  [003] d..1    64.832244: dwc3_ctrl_req: bRequestType 00 bRequest 01 wValue 0030 wIndex 0000 wLength 0
> irq/17-dwc3-3527  [003] d..1    64.832244: dwc3_ep0: USB_REQ_CLEAR_FEATURE
> irq/17-dwc3-3527  [003] d..1    64.832253: dwc3_event: event (000020c2): ep0in: Transfer Not Ready (Not Active)
> irq/17-dwc3-3527  [003] d..1    64.832254: dwc3_ep0: ep0in: Transfer Not Ready (Not Active): state 'Setup Phase'
> irq/17-dwc3-3527  [003] d..1    64.832254: dwc3_ep0: Control Status
> irq/17-dwc3-3527  [003] d..1    64.832255: dwc3_prepare_trb: ep0in: 0/0 trb ffff880084802000 buf 0000000084801000 size 0 ctrl 00000c33 (HLcs:SC:status2)
> irq/17-dwc3-3527  [003] d..1    64.832258: dwc3_gadget_ep_cmd: ep0in: cmd 'Start Transfer' [6] params 00000000 84802000 00000000 --> status: Successful
> irq/17-dwc3-3527  [003] d..1    64.832348: dwc3_event: event (0000c042): ep0in: Transfer Complete
> irq/17-dwc3-3527  [003] d..1    64.832349: dwc3_ep0: ep0in: Transfer Complete: state 'Status Phase'
> irq/17-dwc3-3527  [003] d..1    64.832350: dwc3_ep0: Status Phase
> irq/17-dwc3-3527  [003] d..1    64.832350: dwc3_complete_trb: ep0out: 0/1 trb ffff880084802000 buf 0000000084801000 size 0 ctrl 00000c32 (hLcs:SC:status2)
> irq/17-dwc3-3527  [003] d..1    64.832351: dwc3_prepare_trb: ep0out: 0/1 trb ffff880084802000 buf 0000000084801000 size 8 ctrl 00000c23 (HLcs:SC:setup)
> irq/17-dwc3-3527  [003] d..1    64.832360: dwc3_gadget_ep_cmd: ep0out: cmd 'Start Transfer' [6] params 00000000 84802000 00000000 --> status: Successful
> irq/17-dwc3-3527  [003] d..1    64.832362: dwc3_event: event (00120301): Link Change [U2]
> irq/17-dwc3-3527  [003] d...    64.832364: usb_gadget_vbus_draw: speed 5/5 state 7 0mA [sg:out_aligned:self-powered:activated:connected] --> -95
> irq/17-dwc3-3527  [003] d..1    64.832718: dwc3_event: event (00100301): Link Change [U0]
> irq/17-dwc3-3527  [003] d..1    64.832737: dwc3_event: event (0000c040): ep0out: Transfer Complete

New enumeration. Nothing really screaming wrong here. The only errors
returned by mass storage happen after the 30s timeout:

$ grep -RnH "raise_exception\|fsg_main_thread" /tmp/trace.txt 
/tmp/trace.txt:111743:     irq/17-dwc3-3527  [003] d..1    64.833078: raise_exception: raise_exception 4
/tmp/trace.txt:111745:    file-storage-3526  [002] ....    64.833139: fsg_main_thread: get_next_command -> -4
/tmp/trace.txt:111746:    file-storage-3526  [002] ....    64.833140: fsg_main_thread: handling exception
/tmp/trace.txt:112950:     irq/17-dwc3-3527  [003] d..1    64.951349: raise_exception: raise_exception 4
/tmp/trace.txt:112956:    file-storage-3526  [002] ....    64.951401: fsg_main_thread: handling exception

Any ideas?

-- 
balbi

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 800 bytes --]

Re: Memory barrier needed with wake_up_process()?

[Alan Stern]

On Mon, 19 Sep 2016, Felipe Balbi wrote:

> >>     file-storage-3578  [002] .... 21167.727072: fsg_main_thread: next: bh ffff880111e69a80 state 1
> >>     file-storage-3578  [002] .... 21167.729458: fsg_main_thread: next: bh ffff880111e6aac0 state 2
> >>      irq/17-dwc3-3579  [003] d..1 21167.729666: bulk_out_complete: compl: bh ffff880111e6aac0 state 1
> >>     file-storage-3578  [002] .... 21167.729670: fsg_main_thread: next: bh ffff880111e6aac0 state 1
> >
> > And this is where everything stopped?
> 
> yeah, that's everything.
> 
> > This also looks normal.  So the question is what happened when 
> > get_next_command() returned after this?
> >
> > Felipe, maybe the patch below (in place of your current patch) will
> > help.  Since the events that it logs are all supposed to be unusual,
> > you can use printk if you want, but I wrote it with trace_printk.
> 
> I've applied your patch and it wasn't giving me any output, which hinted
> that g_mass_storage wasn't returning any failures. So I enabled dwc3's
> traces to get more data out of it. Here's the final snippet (with
> comments, again). Let me know if you want the entire thing (it's
> ~14MiB).
> 
> > irq/17-dwc3-3527  [003] d..1    34.215214: dwc3_event: event (00110301): Link Change [U1]
> > irq/17-dwc3-3527  [003] d..1    34.215214: dwc3_event: event (00004086): ep1in: Transfer In-Progress
> > irq/17-dwc3-3527  [003] d..1    34.215214: dwc3_complete_trb: ep1in: 2/255 trb ffff880084813e70 buf 0000000080808000 size 0 ctrl 00000c18 (hlcS:SC:normal)
> > irq/17-dwc3-3527  [003] d..1    34.215214: dwc3_gadget_giveback: ep1in: req ffff88016d046840 length 16384/16384 zsI ==> 0
> > irq/17-dwc3-3527  [003] d...    34.215215: usb_gadget_giveback_request: ep1in: length 16384/16384 sgs 0/0 stream 0 zsI status 0 --> 0
> > irq/17-dwc3-3527  [003] d..1    34.215218: dwc3_gadget_ep_cmd: ep1in: cmd 'Update Transfer' [196615] params 00000000 00000000 00000000 --> status: Successful
> > irq/17-dwc3-3527  [003] d..1    34.215281: dwc3_event: event (00100301): Link Change [U0]
> > irq/17-dwc3-3527  [003] d..1    34.215281: dwc3_event: event (00110301): Link Change [U1]
> > irq/17-dwc3-3527  [003] d..1    34.215282: dwc3_event: event (00004086): ep1in: Transfer In-Progress
> > irq/17-dwc3-3527  [003] d..1    34.215282: dwc3_complete_trb: ep1in: 1/255 trb ffff880084813e80 buf 000000008080c000 size 0 ctrl 00000c18 (hlcS:SC:normal)
> > irq/17-dwc3-3527  [003] d..1    34.215282: dwc3_gadget_giveback: ep1in: req ffff88016d046f00 length 13/13 zsI ==> 0
> > irq/17-dwc3-3527  [003] d...    34.215283: usb_gadget_giveback_request: ep1in: length 13/13 sgs 0/0 stream 0 zsI status 0 --> 0
> > irq/17-dwc3-3527  [003] d..1    34.215284: dwc3_event: event (00100301): Link Change [U0]
> > irq/17-dwc3-3527  [003] d..1    34.215284: dwc3_event: event (00110301): Link Change [U1]
> > irq/17-dwc3-3527  [003] d..1    34.215284: dwc3_event: event (00006084): ep1out: Transfer In-Progress
> > irq/17-dwc3-3527  [003] d..1    34.215284: dwc3_complete_trb: ep1out: 1/255 trb ffff880084804320 buf 000000008080c800 size 993 ctrl 00000c18 (hlcS:SC:normal)
> > irq/17-dwc3-3527  [003] d..1    34.215285: dwc3_gadget_giveback: ep1out: req ffff880154205300 length 31/1024 zsI ==> 0
> > irq/17-dwc3-3527  [003] d...    34.215285: usb_gadget_giveback_request: ep1out: length 31/1024 sgs 0/0 stream 0 zsI status 0 --> 0
> 
> completed and gave back CBW.
> 
> > irq/17-dwc3-3527  [003] d..1    34.215349: dwc3_event: event (00100301): Link Change [U0]
> > irq/17-dwc3-3527  [003] d..1    34.215349: dwc3_event: event (00110301): Link Change [U1]
> > irq/17-dwc3-3527  [003] d..1    34.225616: dwc3_event: event (00120301): Link Change [U2]
> > irq/17-dwc3-3527  [003] d...    34.225617: usb_gadget_vbus_draw: speed 5/5 state 7 0mA [sg:out_aligned:self-powered:activated:connected] --> -95
> > irq/17-dwc3-3527  [003] d..1    64.832221: dwc3_event: event (00100301): Link Change [U0]
> 
> 30 seconds of nothing.
> 
> > irq/17-dwc3-3527  [003] d..1    64.832240: dwc3_event: event (0000c040): ep0out: Transfer Complete
> > irq/17-dwc3-3527  [003] d..1    64.832243: dwc3_ep0: ep0out: Transfer Complete: state 'Setup Phase'
> > irq/17-dwc3-3527  [003] d..1    64.832243: dwc3_ep0: Setup Phase
> > irq/17-dwc3-3527  [003] d..1    64.832244: dwc3_ctrl_req: bRequestType 00 bRequest 01 wValue 0030 wIndex 0000 wLength 0
> > irq/17-dwc3-3527  [003] d..1    64.832244: dwc3_ep0: USB_REQ_CLEAR_FEATURE
> > irq/17-dwc3-3527  [003] d..1    64.832253: dwc3_event: event (000020c2): ep0in: Transfer Not Ready (Not Active)
> > irq/17-dwc3-3527  [003] d..1    64.832254: dwc3_ep0: ep0in: Transfer Not Ready (Not Active): state 'Setup Phase'
> > irq/17-dwc3-3527  [003] d..1    64.832254: dwc3_ep0: Control Status
> > irq/17-dwc3-3527  [003] d..1    64.832255: dwc3_prepare_trb: ep0in: 0/0 trb ffff880084802000 buf 0000000084801000 size 0 ctrl 00000c33 (HLcs:SC:status2)
> > irq/17-dwc3-3527  [003] d..1    64.832258: dwc3_gadget_ep_cmd: ep0in: cmd 'Start Transfer' [6] params 00000000 84802000 00000000 --> status: Successful
> > irq/17-dwc3-3527  [003] d..1    64.832348: dwc3_event: event (0000c042): ep0in: Transfer Complete
> > irq/17-dwc3-3527  [003] d..1    64.832349: dwc3_ep0: ep0in: Transfer Complete: state 'Status Phase'
> > irq/17-dwc3-3527  [003] d..1    64.832350: dwc3_ep0: Status Phase
> > irq/17-dwc3-3527  [003] d..1    64.832350: dwc3_complete_trb: ep0out: 0/1 trb ffff880084802000 buf 0000000084801000 size 0 ctrl 00000c32 (hLcs:SC:status2)
> > irq/17-dwc3-3527  [003] d..1    64.832351: dwc3_prepare_trb: ep0out: 0/1 trb ffff880084802000 buf 0000000084801000 size 8 ctrl 00000c23 (HLcs:SC:setup)
> > irq/17-dwc3-3527  [003] d..1    64.832360: dwc3_gadget_ep_cmd: ep0out: cmd 'Start Transfer' [6] params 00000000 84802000 00000000 --> status: Successful
> > irq/17-dwc3-3527  [003] d..1    64.832362: dwc3_event: event (00120301): Link Change [U2]
> > irq/17-dwc3-3527  [003] d...    64.832364: usb_gadget_vbus_draw: speed 5/5 state 7 0mA [sg:out_aligned:self-powered:activated:connected] --> -95
> > irq/17-dwc3-3527  [003] d..1    64.832718: dwc3_event: event (00100301): Link Change [U0]
> > irq/17-dwc3-3527  [003] d..1    64.832737: dwc3_event: event (0000c040): ep0out: Transfer Complete
> 
> New enumeration. Nothing really screaming wrong here. The only errors
> returned by mass storage happen after the 30s timeout:
> 
> $ grep -RnH "raise_exception\|fsg_main_thread" /tmp/trace.txt 
> /tmp/trace.txt:111743:     irq/17-dwc3-3527  [003] d..1    64.833078: raise_exception: raise_exception 4
> /tmp/trace.txt:111745:    file-storage-3526  [002] ....    64.833139: fsg_main_thread: get_next_command -> -4
> /tmp/trace.txt:111746:    file-storage-3526  [002] ....    64.833140: fsg_main_thread: handling exception
> /tmp/trace.txt:112950:     irq/17-dwc3-3527  [003] d..1    64.951349: raise_exception: raise_exception 4
> /tmp/trace.txt:112956:    file-storage-3526  [002] ....    64.951401: fsg_main_thread: handling exception
> 
> Any ideas?

I'm afraid not.  The only thing I can think of to try next is complete 
tracing of fsg_main_thread, as in the patch below.  It will generate 
continuous output as long as the gadget is doing something, but there 
doesn't seem to be any way to avoid this.  At least when everything 
stops, it should be able to tell us exactly where and why.

Alan Stern



Index: usb-4.x/drivers/usb/gadget/function/f_mass_storage.c
===================================================================
--- usb-4.x.orig/drivers/usb/gadget/function/f_mass_storage.c
+++ usb-4.x/drivers/usb/gadget/function/f_mass_storage.c
@@ -415,6 +415,7 @@ static void raise_exception(struct fsg_c
 	 * If a lower-or-equal priority exception is in progress, preempt it
 	 * and notify the main thread by sending it a signal.
 	 */
+	trace_printk("raise_exception %d\n", new_state);
 	spin_lock_irqsave(&common->lock, flags);
 	if (common->state <= new_state) {
 		common->exception_req_tag = common->ep0_req_tag;
@@ -2495,6 +2496,7 @@ static void handle_exception(struct fsg_
 static int fsg_main_thread(void *common_)
 {
 	struct fsg_common	*common = common_;
+	int rc;
 
 	/*
 	 * Allow the thread to be killed by a signal, but set the signal mask
@@ -2518,16 +2520,20 @@ static int fsg_main_thread(void *common_
 	/* The main loop */
 	while (common->state != FSG_STATE_TERMINATED) {
 		if (exception_in_progress(common) || signal_pending(current)) {
+			trace_printk("handling exception\n");
 			handle_exception(common);
 			continue;
 		}
 
 		if (!common->running) {
+			trace_printk("fsg_main_thread not running...\n");
 			sleep_thread(common, true);
 			continue;
 		}
 
-		if (get_next_command(common))
+		rc = get_next_command(common);
+		trace_printk("get_next_command -> %d\n", rc);
+		if (rc)
 			continue;
 
 		spin_lock_irq(&common->lock);
@@ -2535,7 +2541,13 @@ static int fsg_main_thread(void *common_
 			common->state = FSG_STATE_DATA_PHASE;
 		spin_unlock_irq(&common->lock);
 
-		if (do_scsi_command(common) || finish_reply(common))
+		rc = do_scsi_command(common);
+		trace_printk("do_scsi_command -> %d\n", rc);
+		if (rc)
+			continue;
+		rc = finish_reply(common);
+		trace_printk("finish_reply -> %d\n", rc);
+		if (rc)
 			continue;
 
 		spin_lock_irq(&common->lock);
@@ -2543,7 +2555,9 @@ static int fsg_main_thread(void *common_
 			common->state = FSG_STATE_STATUS_PHASE;
 		spin_unlock_irq(&common->lock);
 
-		if (send_status(common))
+		rc = send_status(common);
+		trace_printk("send_status -> %d\n", rc);
+		if (rc)
 			continue;
 
 		spin_lock_irq(&common->lock);

Re: Memory barrier needed with wake_up_process()?

[Felipe Balbi]

[-- Attachment #1: Type: text/plain, Size: 240622 bytes --]


Hi,

Alan Stern <stern@rowland.harvard.edu> writes:

[...]

>> $ grep -RnH "raise_exception\|fsg_main_thread" /tmp/trace.txt 
>> /tmp/trace.txt:111743:     irq/17-dwc3-3527  [003] d..1    64.833078: raise_exception: raise_exception 4
>> /tmp/trace.txt:111745:    file-storage-3526  [002] ....    64.833139: fsg_main_thread: get_next_command -> -4
>> /tmp/trace.txt:111746:    file-storage-3526  [002] ....    64.833140: fsg_main_thread: handling exception
>> /tmp/trace.txt:112950:     irq/17-dwc3-3527  [003] d..1    64.951349: raise_exception: raise_exception 4
>> /tmp/trace.txt:112956:    file-storage-3526  [002] ....    64.951401: fsg_main_thread: handling exception
>> 
>> Any ideas?
>
> I'm afraid not.  The only thing I can think of to try next is complete 
> tracing of fsg_main_thread, as in the patch below.  It will generate 
> continuous output as long as the gadget is doing something, but there 
> doesn't seem to be any way to avoid this.  At least when everything 
> stops, it should be able to tell us exactly where and why.

tried with your changes plus a trace_printk() added to both
bulk_out_complete and bulk_in_complete. Here's the diff:

diff --git a/drivers/usb/gadget/function/f_mass_storage.c b/drivers/usb/gadget/function/f_mass_storage.c
index 8f3659b65f53..9556451ea584 100644
--- a/drivers/usb/gadget/function/f_mass_storage.c
+++ b/drivers/usb/gadget/function/f_mass_storage.c
@@ -411,6 +411,7 @@ static void raise_exception(struct fsg_common *common, enum fsg_state new_state)
 	 * If a lower-or-equal priority exception is in progress, preempt it
 	 * and notify the main thread by sending it a signal.
 	 */
+	trace_printk("raise_exception %d\n", new_state);
 	spin_lock_irqsave(&common->lock, flags);
 	if (common->state <= new_state) {
 		common->exception_req_tag = common->ep0_req_tag;
@@ -449,6 +450,8 @@ static void bulk_in_complete(struct usb_ep *ep, struct usb_request *req)
 	struct fsg_common	*common = ep->driver_data;
 	struct fsg_buffhd	*bh = req->context;
 
+	trace_printk("%d, %u/%u\n", req->status, req->actual, req->length);
+
 	if (req->status || req->actual != req->length)
 		DBG(common, "%s --> %d, %u/%u\n", __func__,
 		    req->status, req->actual, req->length);
@@ -470,6 +473,8 @@ static void bulk_out_complete(struct usb_ep *ep, struct usb_request *req)
 	struct fsg_buffhd	*bh = req->context;
 
 	dump_msg(common, "bulk-out", req->buf, req->actual);
+
+	trace_printk("%d, %u/%u\n", req->status, req->actual, bh->bulk_out_intended_length);
 	if (req->status || req->actual != bh->bulk_out_intended_length)
 		DBG(common, "%s --> %d, %u/%u\n", __func__,
 		    req->status, req->actual, bh->bulk_out_intended_length);
@@ -2496,6 +2501,7 @@ static void handle_exception(struct fsg_common *common)
 static int fsg_main_thread(void *common_)
 {
 	struct fsg_common	*common = common_;
+	int			rc;
 
 	/*
 	 * Allow the thread to be killed by a signal, but set the signal mask
@@ -2519,6 +2525,7 @@ static int fsg_main_thread(void *common_)
 	/* The main loop */
 	while (common->state != FSG_STATE_TERMINATED) {
 		if (exception_in_progress(common) || signal_pending(current)) {
+			trace_printk("handling exception\n");
 			handle_exception(common);
 			continue;
 		}
@@ -2528,7 +2535,9 @@ static int fsg_main_thread(void *common_)
 			continue;
 		}
 
-		if (get_next_command(common))
+		rc = get_next_command(common);
+		trace_printk("get_next_command -> %d\n", rc);
+		if (rc)
 			continue;
 
 		spin_lock_irq(&common->lock);
@@ -2536,16 +2545,24 @@ static int fsg_main_thread(void *common_)
 			common->state = FSG_STATE_DATA_PHASE;
 		spin_unlock_irq(&common->lock);
 
-		if (do_scsi_command(common) || finish_reply(common))
+		rc = do_scsi_command(common);
+		trace_printk("do_scsi_command -> %d\n", rc);
+		if (rc)
 			continue;
+		rc = finish_reply(common);
+		trace_printk("finish_reply -> %d\n", rc);
+		if (rc)
+		       continue;
 
 		spin_lock_irq(&common->lock);
 		if (!exception_in_progress(common))
 			common->state = FSG_STATE_STATUS_PHASE;
 		spin_unlock_irq(&common->lock);
 
-		if (send_status(common))
-			continue;
+		rc = send_status(common);
+		trace_printk("send_status -> %d\n", rc);
+		if (rc)
+		       continue;
 
 		spin_lock_irq(&common->lock);
 		if (!exception_in_progress(common))

And here's trace output (complete, scroll to bottom). It seems to me
like the thread didn't wake up at all. It didn't even try to
execute. I'll add some more traces and try to get better information
about what's going on.



# tracer: nop
#
# entries-in-buffer/entries-written: 2865/2865   #P:4
#
#                              _-----=> irqs-off
#                             / _----=> need-resched
#                            | / _---=> hardirq/softirq
#                            || / _--=> preempt-depth
#                            ||| /     delay
#           TASK-PID   CPU#  ||||    TIMESTAMP  FUNCTION
#              | |       |   ||||       |         |
     irq/17-dwc3-2522  [002] d...    43.387002: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.387043: fsg_main_thread: get_next_command -> 0
     irq/17-dwc3-2522  [002] d...    43.387148: bulk_in_complete: 0, 16384/16384
    file-storage-2521  [001] ....    43.387195: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.387202: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.387205: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.387246: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.387250: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.387363: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.387367: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.387478: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.387481: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.387575: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.387681: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.387685: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.387785: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.387788: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.387868: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.387950: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.387954: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.388070: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.388078: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.388270: bulk_out_complete: 0, 31/31
    file-storage-2521  [000] ....    43.388273: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [000] ....    43.388337: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [000] ....    43.388341: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [000] ....    43.388345: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.388363: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.388501: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.388505: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.388621: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.388626: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.388712: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.388715: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.388845: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.388850: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.388912: bulk_out_complete: 0, 31/31
    file-storage-2521  [000] ....    43.388914: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [000] ....    43.388973: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [000] ....    43.388976: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [000] ....    43.388980: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.388998: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.389098: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.389101: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.389194: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.389198: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.389283: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.389287: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.389377: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.389382: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.389444: bulk_out_complete: 0, 31/31
    file-storage-2521  [000] ....    43.389446: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [000] ....    43.389531: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [000] ....    43.389535: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [000] ....    43.389539: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.389544: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.389634: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.389637: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.389730: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.389733: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.389821: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.389825: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.389921: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.389945: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.390026: bulk_out_complete: 0, 31/31
    file-storage-2521  [000] ....    43.390028: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [000] ....    43.390109: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [000] ....    43.390116: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [000] ....    43.390120: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.390125: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.390214: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.390219: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.390312: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.390375: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.390379: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.390482: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.390636: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.390642: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.390707: bulk_out_complete: 0, 31/31
    file-storage-2521  [000] ....    43.390709: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [000] ....    43.390797: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [000] ....    43.390802: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [000] ....    43.390805: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.390806: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.390894: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.390899: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.390984: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.390988: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.391074: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.391077: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.391164: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.391169: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.391284: bulk_out_complete: 0, 31/31
    file-storage-2521  [000] ....    43.391287: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [000] ....    43.391365: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [000] ....    43.391369: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [000] ....    43.391372: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.391376: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.391473: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.391477: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.391566: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.391570: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.391657: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.391660: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.391755: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.391780: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.391860: bulk_out_complete: 0, 31/31
    file-storage-2521  [000] ....    43.391864: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [000] ....    43.391938: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [000] ....    43.391942: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [000] ....    43.391946: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.391950: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.392032: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.392099: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.392104: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.392192: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.392196: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.392279: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.392284: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.392372: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.392453: bulk_out_complete: 0, 31/31
    file-storage-2521  [000] ....    43.392456: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [000] ....    43.392537: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [000] ....    43.392541: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [000] ....    43.392544: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.392555: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.392645: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.392648: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.392734: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.392738: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.392824: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.392827: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.392920: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.392945: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.393026: bulk_out_complete: 0, 31/31
    file-storage-2521  [000] ....    43.393029: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [000] ....    43.393112: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [000] ....    43.393116: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [000] ....    43.393120: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.393124: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.393216: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.393221: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.393303: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.393308: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.393394: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.393398: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.393476: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.393481: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.393590: bulk_out_complete: 0, 31/31
    file-storage-2521  [000] ....    43.393592: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [000] ....    43.393688: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [000] ....    43.393692: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [000] ....    43.393695: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.393699: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.393794: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.393797: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.393884: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.393888: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.393973: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.393976: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.394070: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.394094: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.394178: bulk_out_complete: 0, 31/31
    file-storage-2521  [000] ....    43.394180: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [000] ....    43.394264: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [000] ....    43.394268: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [000] ....    43.394272: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.394277: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.394365: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.394369: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.394455: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.394459: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.394546: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.394549: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.394643: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.394668: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.394749: bulk_out_complete: 0, 31/31
    file-storage-2521  [000] ....    43.394751: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [000] ....    43.394847: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [000] ....    43.394851: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [000] ....    43.394855: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.394859: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.394949: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.394952: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.395037: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.395040: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.395126: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.395130: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.395241: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.395246: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.395342: bulk_out_complete: 0, 31/31
    file-storage-2521  [000] ....    43.395345: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [000] ....    43.395424: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [000] ....    43.395428: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [000] ....    43.395432: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.395443: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.395533: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.395537: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.395624: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.395629: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.395713: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.395717: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.395810: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.395835: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.395918: bulk_out_complete: 0, 31/31
    file-storage-2521  [000] ....    43.395921: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [000] ....    43.396020: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [000] ....    43.396024: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [000] ....    43.396027: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.396032: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.396124: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.396128: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.396214: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.396218: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.396305: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.396309: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.396401: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.396426: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.396507: bulk_out_complete: 0, 31/31
    file-storage-2521  [000] ....    43.396510: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [000] ....    43.396606: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [000] ....    43.396611: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [000] ....    43.396614: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.396619: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.396708: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.396712: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.396796: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.396800: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.396886: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.396890: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.396978: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.396982: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.397101: bulk_out_complete: 0, 31/31
    file-storage-2521  [000] ....    43.397104: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [000] ....    43.397195: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [000] ....    43.397201: fsg_main_thread: finish_reply -> 0
     irq/17-dwc3-2522  [002] d...    43.397202: bulk_in_complete: 0, 16384/16384
    file-storage-2521  [000] ....    43.397205: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.397291: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.397295: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.397380: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.397384: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.397477: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.397482: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.397579: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.397601: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.397681: bulk_out_complete: 0, 31/31
    file-storage-2521  [000] ....    43.397698: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [000] ....    43.397778: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [000] ....    43.397784: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [000] ....    43.397787: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.397799: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.397903: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.397907: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.397992: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.397997: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.398081: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.398085: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.398193: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.398197: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.398260: bulk_out_complete: 0, 31/31
    file-storage-2521  [000] ....    43.398262: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [000] ....    43.398348: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [000] ....    43.398351: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [000] ....    43.398355: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.398361: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.398447: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.398451: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.398537: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.398541: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.398628: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.398632: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.398737: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.398742: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.398804: bulk_out_complete: 0, 31/31
    file-storage-2521  [000] ....    43.398806: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [000] ....    43.398905: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [000] ....    43.398909: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [000] ....    43.398912: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.398917: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.399005: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.399009: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.399091: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.399096: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.399188: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.399192: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.399304: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.399312: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.399390: bulk_out_complete: 0, 31/31
    file-storage-2521  [000] ....    43.399392: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [000] ....    43.399464: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [000] ....    43.399468: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [000] ....    43.399472: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.399481: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.399570: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.399574: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.399660: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.399663: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.399748: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.399752: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.399837: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.399842: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.399965: bulk_out_complete: 0, 31/31
    file-storage-2521  [000] ....    43.399968: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [000] ....    43.400035: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [000] ....    43.400039: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [000] ....    43.400042: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.400067: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.400145: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.400213: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.400216: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.400301: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.400305: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.400394: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.400398: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.400496: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.400499: bulk_out_complete: 0, 31/31
    file-storage-2521  [000] ....    43.400500: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [000] ....    43.400593: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [000] ....    43.400598: fsg_main_thread: finish_reply -> 0
     irq/17-dwc3-2522  [002] d...    43.400599: bulk_in_complete: 0, 16384/16384
    file-storage-2521  [000] ....    43.400602: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.400689: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.400692: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.400778: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.400782: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.400868: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.400872: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.400969: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.400974: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.401037: bulk_out_complete: 0, 31/31
    file-storage-2521  [000] ....    43.401039: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [000] ....    43.401130: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [000] ....    43.401134: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [000] ....    43.401137: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.401146: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.401237: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.401301: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.401306: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.401383: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.401386: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.401472: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.401476: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.401561: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.401637: bulk_out_complete: 0, 31/31
    file-storage-2521  [000] ....    43.401640: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [000] ....    43.401742: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [000] ....    43.401746: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [000] ....    43.401748: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.401753: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.401841: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.401845: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.401930: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.401934: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.402023: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.402027: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.402113: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.402118: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.402182: bulk_out_complete: 0, 31/31
    file-storage-2521  [000] ....    43.402185: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [000] ....    43.402252: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [000] ....    43.402255: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [000] ....    43.402259: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.402270: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.402360: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.402364: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.402449: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.402453: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.402541: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.402544: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.402669: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.402674: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.402676: bulk_out_complete: 0, 31/31
    file-storage-2521  [000] ....    43.402678: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [000] ....    43.402745: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [000] ....    43.402750: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [000] ....    43.402753: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.402813: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.402817: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.402896: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.402900: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.402976: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.403041: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.403044: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.403132: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.403137: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.403202: bulk_out_complete: 0, 31/31
    file-storage-2521  [000] ....    43.403205: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [000] ....    43.403285: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [000] ....    43.403291: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [000] ....    43.403311: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.403317: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.403382: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.403464: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.403468: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.403553: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.403558: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.403645: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.403650: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.403734: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.403810: bulk_out_complete: 0, 31/31
    file-storage-2521  [000] ....    43.403813: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [000] ....    43.403912: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [000] ....    43.403919: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [000] ....    43.403922: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.403926: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.404014: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.404018: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.404101: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.404105: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.404192: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.404196: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.404282: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.404287: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.404351: bulk_out_complete: 0, 31/31
    file-storage-2521  [000] ....    43.404353: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [000] ....    43.404420: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [000] ....    43.404427: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [000] ....    43.404430: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.404439: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.404526: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.404591: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.404595: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.404682: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.404686: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.404790: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.404808: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.404818: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.404927: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.404931: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.405001: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.405005: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.405008: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.405019: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.405109: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.405113: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.405198: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.405202: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.405291: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.405295: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.405381: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.405386: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.405470: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.405472: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.405552: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.405556: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.405559: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.405567: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.405658: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.405662: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.405748: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.405752: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.405836: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.405839: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.405928: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.405933: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.406009: bulk_out_complete: 0, 31/31
    file-storage-2521  [000] ....    43.406011: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [000] ....    43.406077: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [000] ....    43.406081: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [000] ....    43.406084: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.406096: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.406187: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.406190: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.406286: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.406290: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.406383: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.406387: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.406472: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.406476: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.406597: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.406652: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.406734: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.406737: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.406741: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.406788: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.406792: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.406896: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.406987: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.406991: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.407056: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.407125: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.407128: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.407194: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.407270: bulk_out_complete: 0, 31/31
    file-storage-2521  [000] ....    43.407273: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [000] ....    43.407338: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [000] ....    43.407344: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [000] ....    43.407348: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.407360: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.407449: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.407454: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.407538: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.407542: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.407631: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.407635: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.407721: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.407731: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.407814: bulk_out_complete: 0, 31/31
    file-storage-2521  [000] ....    43.407816: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [000] ....    43.407883: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [000] ....    43.407887: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [000] ....    43.407890: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.407901: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.407991: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.407995: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.408081: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.408085: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.408175: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.408179: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.408263: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.408268: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.408371: bulk_out_complete: 0, 31/31
    file-storage-2521  [000] ....    43.408374: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [000] ....    43.408462: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [000] ....    43.408466: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [000] ....    43.408470: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.408478: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.408566: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.408569: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.408654: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.408658: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.408745: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.408749: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.408834: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.408838: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.408960: bulk_out_complete: 0, 31/31
    file-storage-2521  [000] ....    43.408963: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [000] ....    43.409046: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [000] ....    43.409050: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [000] ....    43.409054: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.409062: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.409151: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.409154: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.409242: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.409246: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.409331: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.409335: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.409419: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.409424: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.409544: bulk_out_complete: 0, 31/31
    file-storage-2521  [000] ....    43.409546: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [000] ....    43.409650: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [000] ....    43.409654: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [000] ....    43.409656: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.409660: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.409750: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.409753: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.409840: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.409844: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.409930: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.409934: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.410027: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.410052: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.410119: bulk_out_complete: 0, 31/31
    file-storage-2521  [000] ....    43.410122: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [000] ....    43.410213: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [000] ....    43.410218: fsg_main_thread: finish_reply -> 0
     irq/17-dwc3-2522  [002] d...    43.410219: bulk_in_complete: 0, 16384/16384
    file-storage-2521  [000] ....    43.410222: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.410309: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.410313: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.410398: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.410402: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.410489: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.410492: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.410589: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.410593: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.410710: bulk_out_complete: 0, 31/31
    file-storage-2521  [000] ....    43.410713: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [000] ....    43.410811: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [000] ....    43.410815: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [000] ....    43.410818: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.410824: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.410914: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.410918: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.411002: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.411006: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.411093: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.411096: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.411182: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.411186: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.411325: bulk_out_complete: 0, 31/31
    file-storage-2521  [000] ....    43.411328: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [000] ....    43.411396: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [000] ....    43.411400: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [000] ....    43.411404: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.411414: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.411508: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.411512: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.411598: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.411601: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.411686: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.411690: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.411777: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.411781: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.411902: bulk_out_complete: 0, 31/31
    file-storage-2521  [000] ....    43.411905: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [000] ....    43.411972: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [000] ....    43.411976: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [000] ....    43.411979: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.411991: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.412095: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.412160: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.412164: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.412243: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.412246: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.412336: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.412340: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.412460: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.412571: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.412622: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.412697: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.412703: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.412706: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.412713: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.412819: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.412823: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.412918: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.413010: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.413014: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.413129: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.413133: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.413238: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.413311: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.413315: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.413382: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.413388: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.413392: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.413402: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.413501: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.413505: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.413602: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.413606: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.413705: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.413709: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.413811: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.413939: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.414017: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.414066: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.414151: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.414165: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.414169: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.414170: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.414285: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.414288: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.414383: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.414470: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.414474: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.414568: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.414572: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.414673: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.414802: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.414852: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.414932: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.414936: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.414940: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.415032: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.415035: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.415127: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.415215: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.415219: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.415319: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.415323: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.415413: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.415516: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.415592: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.415596: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.415696: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.415700: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.415703: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.415708: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.415813: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.415816: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.415908: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.415911: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.415997: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.416088: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.416092: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.416208: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.416355: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.416383: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.416464: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.416468: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.416472: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.416482: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.416602: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.416680: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.416683: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.416765: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.416768: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.416855: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.416859: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.416954: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.417048: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.417051: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.417117: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.417123: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.417127: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.417134: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.417225: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.417228: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.417315: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.417318: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.417419: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.417423: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.417530: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.417534: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.417597: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.417600: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.417669: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.417675: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.417678: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.417686: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.417777: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.417780: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.417879: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.417883: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.417973: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.418039: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.418043: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.418129: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.418205: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.418208: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.418274: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.418281: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.418284: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.418292: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.418384: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.418387: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.418478: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.418555: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.418559: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.418648: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.418652: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.418735: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.418830: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.418834: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.418900: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.418903: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.418907: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.418917: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.419021: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.419025: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.419101: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.419166: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.419170: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.419260: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.419326: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.419423: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.419425: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.419426: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.419508: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.419512: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.419515: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.419560: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.419563: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.419650: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.419653: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.419740: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.419742: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.419829: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.419833: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.419928: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.420054: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.420057: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.420137: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.420142: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.420146: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.420211: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.420214: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.420306: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.420377: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.420381: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.420471: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.420539: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.420543: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.420627: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.420703: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.420706: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.420771: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.420775: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.420778: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.420788: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.420879: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.420882: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.420969: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.420973: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.421059: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.421062: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.421160: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.421165: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.421266: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.421269: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.421357: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.421362: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.421366: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.421410: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.421428: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.421492: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.421558: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.421562: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.421652: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.421717: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.421721: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.421786: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.421881: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.421883: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.421958: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.421964: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.421967: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.421975: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.422077: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.422081: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.422172: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.422241: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.422245: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.422322: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.422390: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.422394: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.422458: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.422461: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.422527: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.422531: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.422534: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.422545: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.422635: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.422639: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.422736: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.422740: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.422831: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.422897: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.422900: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.422985: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.423060: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.423063: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.423126: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.423129: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.423133: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.423146: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.423233: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.423309: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.423313: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.423410: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.423414: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.423499: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.423503: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.423579: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.423674: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.423677: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.423758: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.423764: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.423767: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.423772: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.423865: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.423869: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.423951: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.423955: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.424050: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.424054: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.424166: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.424171: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.424234: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.424237: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.424320: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.424323: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.424326: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.424330: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.424425: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.424428: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.424514: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.424517: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.424596: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.424682: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.424686: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.424768: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.424770: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.424772: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.424844: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.424849: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.424853: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.424901: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.424905: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.425000: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.425004: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.425092: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.425169: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.425173: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.425261: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.425265: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.425330: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.425333: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.425396: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.425399: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.425403: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.425416: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.425520: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.425524: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.425608: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.425611: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.425700: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.425804: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.425808: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.425925: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.425927: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.425929: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.426003: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.426009: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.426012: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.426020: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.426108: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.426174: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.426178: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.426265: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.426268: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.426354: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.426358: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.426454: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.426550: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.426553: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.426646: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.426653: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.426656: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.426664: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.426767: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.426771: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.426856: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.426860: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.426946: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.426949: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.427056: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.427061: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.427129: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.427132: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.427199: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.427203: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.427206: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.427211: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.427314: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.427318: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.427405: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.427409: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.427498: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.427502: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.427606: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.427611: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.427674: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.427677: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.427741: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.427746: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.427750: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.427760: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.427853: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.427857: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.427940: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.427944: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.428034: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.428038: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.428162: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.428166: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.428169: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.428170: fsg_main_thread: get_next_command -> 0
     irq/17-dwc3-2522  [002] d...    43.428268: bulk_in_complete: 0, 16384/16384
    file-storage-2521  [001] ....    43.428278: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.428282: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.428285: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.428376: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.428381: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.428466: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.428469: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.428569: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.428572: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.428658: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.428662: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.428727: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.428730: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.428822: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.428825: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.428829: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.428836: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.428928: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.428932: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.429030: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.429034: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.429121: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.429125: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.429203: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.429207: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.429305: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.429321: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.429383: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.429390: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.429408: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.429411: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.429488: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.429570: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.429574: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.429673: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.429678: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.429767: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.429833: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.429929: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.429931: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.429933: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.430011: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.430016: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.430020: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.430024: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.430128: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.430132: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.430228: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.430232: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.430327: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.430331: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.430432: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.430437: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.430500: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.430503: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.430595: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.430599: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.430602: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.430606: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.430704: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.430708: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.430800: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.430804: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.430891: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.430895: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.430979: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.430984: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.431069: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.431072: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.431140: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.431146: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.431152: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.431204: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.431208: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.431291: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.431295: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.431381: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.431385: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.431471: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.431474: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.431550: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.431666: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.431669: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.431749: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.431755: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.431758: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.431769: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.431866: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.431870: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.431973: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.431977: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.432087: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.432091: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.432218: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.432222: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.432224: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.432226: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.432328: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.432332: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.432335: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.432387: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.432391: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.432484: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.432488: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.432587: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.432590: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.432709: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.432713: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.432823: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.432825: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.432827: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.432908: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.432913: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.432917: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.432917: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.433019: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.433023: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.433108: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.433112: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.433200: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.433204: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.433306: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.433312: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.433314: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.433315: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.433396: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.433400: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.433403: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.433448: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.433452: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.433546: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.433610: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.433612: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.433705: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.433709: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.433827: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.433835: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.433900: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.433903: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.433999: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.434005: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.434009: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.434017: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.434120: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.434125: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.434203: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.434269: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.434273: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.434385: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.434390: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.434494: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.434569: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.434572: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.434638: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.434645: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.434649: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.434656: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.434760: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.434764: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.434860: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.434864: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.434952: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.434955: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.435067: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.435072: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.435135: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.435138: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.435233: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.435237: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.435240: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.435244: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.435338: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.435342: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.435437: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.435441: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.435529: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.435533: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.435631: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.435636: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.435739: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.435742: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.435838: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.435843: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.435847: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.435848: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.435943: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.436007: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.436010: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.436090: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.436093: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.436180: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.436184: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.436264: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.436339: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.436342: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.436409: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.436412: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.436416: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.436426: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.436530: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.436533: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.436617: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.436621: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.436713: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.436780: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.436783: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.436871: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.436965: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.436968: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.437034: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.437038: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.437041: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.437051: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.437144: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.437148: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.437245: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.437249: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.437327: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.437406: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.437410: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.437509: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.437511: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.437512: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.437589: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.437593: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.437596: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.437641: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.437645: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.437726: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.437791: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.437795: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.437882: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.437886: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.437951: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.438067: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.438069: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.438071: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.438149: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.438153: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.438160: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.438214: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.438218: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.438294: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.438375: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.438378: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.438442: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.438508: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.438512: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.438588: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.438717: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.438720: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.438798: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.438801: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.438805: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.438812: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.438916: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.438920: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.439016: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.439020: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.439114: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.439117: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.439231: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.439235: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.439298: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.439301: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.439368: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.439374: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.439378: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.439385: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.439488: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.439492: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.439577: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.439581: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.439659: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.439744: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.439748: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.439835: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.439837: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.439838: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.439916: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.439922: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.439926: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.439930: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.440035: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.440039: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.440129: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.440196: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.440199: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.440279: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.440347: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.440352: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.440416: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.440419: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.440515: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.440519: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.440522: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.440526: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.440620: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.440623: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.440718: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.440722: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.440808: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.440812: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.440896: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.440900: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.440984: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.440987: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.441081: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.441085: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.441088: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.441099: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.441205: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.441208: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.441299: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.441363: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.441367: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.441444: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.441516: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.441521: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.441585: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.441588: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.441680: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.441684: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.441687: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.441695: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.441789: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.441853: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.441857: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.441932: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.441936: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.442038: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.442042: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.442107: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.442110: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.442111: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.442185: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.442189: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.442192: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.442256: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.442260: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.442338: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.442415: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.442418: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.442506: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.442510: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.442604: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.442609: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.442673: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.442676: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.442738: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.442742: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.442745: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.442759: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.442848: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.442852: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.442960: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.442964: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.443066: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.443069: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.443151: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.443155: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.443249: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.443252: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.443345: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.443349: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.443352: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.443359: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.443463: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.443466: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.443556: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.443623: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.443626: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.443718: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.443783: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.443788: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.443893: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.443896: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.443977: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.443983: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.443986: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.443991: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.444095: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.444099: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.444185: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.444189: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.444276: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.444280: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.444384: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.444388: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.444452: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.444455: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.444521: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.444524: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.444528: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.444538: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.444631: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.444635: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.444717: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.444720: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.444820: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.444824: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.444935: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.444940: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.445003: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.445006: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.445071: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.445075: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.445078: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.445089: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.445196: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.445201: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.445284: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.445287: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.445372: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.445449: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.445453: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.445470: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.445573: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.445576: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.445641: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.445645: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.445649: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.445659: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.445751: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.445754: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.445852: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.445856: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.445952: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.445955: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.446042: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.446048: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.446111: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.446114: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.446181: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.446187: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.446190: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.446198: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.446292: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.446295: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.446380: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.446384: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.446478: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.446482: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.446598: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.446603: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.446666: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.446669: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.446763: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.446767: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.446770: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.446775: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.446871: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.446875: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.446964: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.447027: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.447031: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.447117: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.447121: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.447187: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.447269: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.447272: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.447353: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.447357: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.447360: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.447364: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.447467: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.447471: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.447569: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.447572: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.447658: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.447662: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.447751: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.447756: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.447820: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.447823: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.447891: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.447897: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.447900: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.447907: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.448011: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.448015: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.448100: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.448104: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.448183: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.448268: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.448272: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.448358: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.448360: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.448361: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.448440: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.448443: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.448447: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.448451: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.448555: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.448559: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.448643: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.448647: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.448733: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.448736: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.448833: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.448837: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.448958: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.448961: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.449042: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.449046: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.449049: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.449057: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.449144: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.449209: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.449213: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.449304: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.449308: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.449409: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.449413: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.449437: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.449531: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.449533: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.449634: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.449639: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.449643: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.449643: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.449757: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.449761: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.449869: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.449872: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.449973: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.449977: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.450077: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.450081: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.450164: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.450167: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.450249: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.450253: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.450258: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.450329: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.450333: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.450397: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.450461: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.450465: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.450549: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.450553: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.450645: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.450649: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.450735: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.450738: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.450834: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.450838: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.450841: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.450849: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.450954: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.450958: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.451043: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.451047: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.451145: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.451149: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.451215: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.451292: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.451294: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.451296: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.451372: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.451376: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.451380: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.451437: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.451440: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.451525: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.451602: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.451605: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.451689: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.451693: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.451796: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.451802: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.451804: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.451805: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.451888: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.451895: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.451898: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.451939: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.451943: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.452029: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.452033: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.452118: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.452121: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.452221: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.452225: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.452229: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.452349: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.452352: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.452418: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.452422: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.452426: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.452436: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.452534: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.452538: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.452630: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.452634: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.452741: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.452745: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.452833: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.452930: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.452931: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.452933: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.453011: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.453014: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.453018: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.453063: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.453067: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.453162: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.453226: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.453230: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.453312: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.453315: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.453402: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.453406: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.453482: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.453485: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.453553: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.453559: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.453564: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.453630: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.453634: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.453710: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.453775: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.453779: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.453865: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.453869: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.453992: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.453997: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.453998: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.453999: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.454074: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.454079: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.454082: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.454143: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.454147: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.454234: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.454238: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.454335: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.454338: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.454457: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.454461: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.454539: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.454540: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.454542: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.454627: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.454632: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.454635: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.454680: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.454684: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.454769: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.454773: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.454864: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.454939: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.454943: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.455031: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.455035: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.455131: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.455134: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.455214: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.455221: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.455225: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.455276: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.455279: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.455357: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.455421: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.455424: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.455514: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.455580: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.455584: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.455660: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.455781: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.455784: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.455867: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.455873: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.455877: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.455881: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.455986: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.455990: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.456081: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.456145: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.456149: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.456236: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.456240: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.456316: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.456405: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.456408: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.456485: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.456489: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.456492: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.456506: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.456610: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.456614: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.456697: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.456701: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.456788: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.456792: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.456889: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.456894: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.456957: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.456960: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.457021: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.457025: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.457028: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.457105: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.457108: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.457200: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.457204: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.457296: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.457300: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.457392: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.457396: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.457471: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.457570: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.457573: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.457639: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.457643: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.457648: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.457705: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.457709: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.457818: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.457895: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.457898: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.457988: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.458088: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.458093: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.458159: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.458161: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.458163: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.458238: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.458242: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.458245: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.458307: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.458311: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.458376: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.458464: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.458468: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.458545: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.458617: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.458621: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.458716: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.458823: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.458826: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.458889: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.458893: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.458896: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.458910: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.459011: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.459015: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.459101: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.459105: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.459206: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.459210: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.459332: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.459336: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.459338: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.459339: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.459415: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.459419: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.459422: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.459475: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.459478: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.459571: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.459575: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.459666: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.459741: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.459745: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.459835: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.459839: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.459924: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.459927: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.460003: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.460009: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.460012: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.460026: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.460127: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.460131: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.460230: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.460234: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.460332: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.460336: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.460423: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.460427: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.460525: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.460527: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.460597: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.460601: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.460618: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.460624: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.460689: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.460771: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.460775: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.460850: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.460917: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.460921: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.461011: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.461015: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.461097: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.461100: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.461183: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.461191: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.461195: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.461268: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.461273: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.461351: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.461355: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.461459: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.461463: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.461550: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.461553: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.461650: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.461652: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.461654: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.461728: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.461731: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.461735: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.461797: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.461801: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.461877: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.461956: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.461959: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.462058: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.462062: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.462150: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.462154: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.462241: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.462244: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.462312: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.462318: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.462321: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.462329: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.462431: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.462435: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.462525: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.462597: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.462601: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.462691: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.462777: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.462781: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.462784: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.462784: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.462858: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.462863: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.462867: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.462868: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.462968: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.462972: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.463058: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.463062: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.463151: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.463155: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.463260: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.463265: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.463329: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.463331: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.463398: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.463405: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.463408: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.463416: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.463507: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.463511: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.463594: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.463597: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.463698: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.463702: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.463811: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.463816: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.463879: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.463882: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.463975: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.463981: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.463984: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.463989: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.464080: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.464144: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.464147: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.464227: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.464231: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.464319: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.464323: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.464410: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.464413: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.464414: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.464479: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.464483: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.464489: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.464546: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.464550: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.464645: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.464722: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.464727: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.464813: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.464817: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.464900: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.464905: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.464987: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.464990: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.465068: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.465072: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.465075: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.465082: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.465188: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.465192: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.465280: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.465284: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.465362: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.465431: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.465435: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.465500: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.465503: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.465504: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.465574: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.465579: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.465582: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.465640: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.465644: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.465737: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.465800: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.465804: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.465893: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.465897: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.465987: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.465991: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.466053: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.466070: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.466143: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.466148: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.466152: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.466156: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.466254: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.466257: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.466353: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.466357: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.466449: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.466453: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.466557: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.466562: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.466624: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.466627: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.466694: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.466701: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.466704: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.466712: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.466816: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.466819: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.466903: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.466907: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.466998: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.467002: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.467133: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.467138: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.467201: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.467204: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.467269: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.467274: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.467277: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.467287: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.467395: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.467399: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.467475: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.467558: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.467562: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.467649: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.467652: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.467731: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.467734: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.467735: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.467815: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.467819: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.467822: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.467874: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.467878: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.467968: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.468033: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.468037: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.468135: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.468138: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.468244: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.468248: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.468313: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.468316: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.468383: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.468386: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.468390: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.468400: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.468499: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.468503: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.468603: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.468607: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.468693: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.468697: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.468794: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.468798: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.468893: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.468897: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.468975: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.468979: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.468982: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.468990: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.469094: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.469097: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.469187: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.469251: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.469255: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.469341: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.469346: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.469431: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.469433: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.469434: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.469510: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.469514: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.469517: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.469569: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.469573: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.469657: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.469660: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.469744: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.469748: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.469847: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.469851: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.469936: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.469938: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.469939: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.470034: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.470039: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.470043: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.470051: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.470156: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.470159: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.470244: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.470248: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.470334: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.470411: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.470415: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.470433: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.470533: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.470536: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.470603: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.470608: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.470614: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.470677: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.470681: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.470758: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.470839: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.470843: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.470919: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.470985: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.470989: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.471055: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.471057: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.471058: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.471124: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.471129: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.471134: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.471204: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.471208: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.471286: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.471350: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.471354: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.471441: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.471445: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.471529: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.471534: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.471596: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.471599: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.471691: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.471697: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.471701: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.471712: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.471814: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.471818: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.471896: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.471976: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.471980: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.472057: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.472124: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.472129: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.472192: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.472195: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.472261: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.472265: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.472268: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.472278: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.472372: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.472375: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.472472: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.472476: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.472563: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.472567: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.472674: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.472679: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.472741: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.472744: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.472814: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.472818: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.472821: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.472829: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.472933: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.472937: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.473034: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.473038: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.473123: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.473127: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.473215: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.473219: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.473282: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.473285: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.473351: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.473354: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.473358: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.473368: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.473472: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.473476: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.473572: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.473576: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.473664: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.473667: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.473786: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.473790: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.473792: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.473794: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.473866: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.473873: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.473877: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.473939: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.473942: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.474007: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.474070: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.474074: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.474157: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.474223: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.474227: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.474293: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.474295: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.474296: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.474364: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.474370: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.474374: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.474426: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.474443: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.474507: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.474572: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.474576: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.474665: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.474669: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.474775: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.474780: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.474843: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.474846: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.474911: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.474915: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.474918: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.474929: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.475035: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.475039: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.475127: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.475130: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.475218: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.475221: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.475319: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.475323: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.475407: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.475410: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.475491: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.475496: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.475500: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.475504: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.475609: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.475614: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.475696: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.475700: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.475790: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.475856: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.475860: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.475939: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.475941: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.475942: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.476004: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.476009: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.476013: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.476080: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.476084: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.476169: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.476245: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.476248: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.476329: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.476333: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.476431: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.476435: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.476517: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.476520: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.476599: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.476602: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.476606: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.476613: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.476717: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.476721: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.476804: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.476807: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.476886: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.476971: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.476975: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.477062: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.477064: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.477066: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.477152: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.477155: fsg_main_thread: finish_reply -> 0
     irq/17-dwc3-2522  [002] d...    43.477156: bulk_in_complete: 0, 16384/16384
    file-storage-2521  [001] ....    43.477164: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.477268: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.477272: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.477355: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.477359: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.477446: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.477450: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.477536: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.477540: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.477654: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.477657: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.477721: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.477727: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.477730: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.477741: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.477833: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.477837: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.477934: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.477938: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.478027: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.478031: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.478108: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.478174: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.478177: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.478178: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.478248: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.478253: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.478255: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.478323: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.478326: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.478409: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.478412: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.478498: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.478502: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.478589: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.478593: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.478658: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.478660: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.478661: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.478737: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.478741: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.478746: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.478815: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.478819: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.478906: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.478910: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.479009: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.479013: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.479101: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.479105: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.479209: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.479272: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.479274: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.479339: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.479345: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.479349: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.479359: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.479451: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.479455: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.479539: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.479543: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.479631: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.479635: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.479719: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.479723: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.479786: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.479789: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.479855: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.479862: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.479865: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.479873: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.479964: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.479968: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.480063: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.480067: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.480166: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.480170: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.480280: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.480285: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.480287: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.480288: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.480363: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.480370: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.480389: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.480434: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.480437: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.480527: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.480608: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.480612: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.480698: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.480701: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.480797: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.480801: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.480864: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.480867: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.480933: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.480937: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.480940: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.480950: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.481052: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.481055: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.481150: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.481154: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.481233: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.481297: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.481301: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.481367: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.481369: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.481371: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.481438: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.481442: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.481447: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.481507: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.481524: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.481589: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.481654: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.481658: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.481750: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.481814: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.481818: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.481883: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.481886: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.481887: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.481967: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.481974: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.481979: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.482043: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.482047: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.482130: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.482134: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.482235: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.482239: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.482324: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.482328: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.482394: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.482396: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.482397: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.482467: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.482473: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.482475: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.482531: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.482535: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.482617: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.482621: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.482708: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.482712: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.482802: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.482805: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.482910: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.482912: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.482913: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.482972: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.482977: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.482982: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.483040: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.483044: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.483128: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.483132: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.483216: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.483220: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.483309: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.483313: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.483378: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.483381: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.483382: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.483452: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.483457: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.483463: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.483515: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.483519: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.483603: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.483606: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.483690: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.483694: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.483784: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.483787: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.483853: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.483855: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.483856: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.483926: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.483932: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.483934: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.483989: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.483993: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.484077: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.484081: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.484168: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.484172: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.484259: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.484262: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.484328: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.484330: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.484332: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.484402: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.484406: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.484412: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.484463: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.484467: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.484558: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.484562: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.484651: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.484654: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.484742: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.484746: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.484812: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.484814: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.484815: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.484882: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.484886: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.484891: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.484949: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.484953: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.485035: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.485039: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.485124: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.485127: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.485216: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.485219: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.485287: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.485289: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.485290: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.485354: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.485361: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.485365: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.485416: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.485420: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.485506: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.485510: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.485594: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.485598: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.485684: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.485688: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.485771: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.485774: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.485775: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.485852: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.485856: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.485861: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.485919: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.485923: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.486010: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.486013: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.486102: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.486105: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.486196: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.486200: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.486267: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.486270: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.486272: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.486338: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.486342: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.486345: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.486406: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.486410: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.486497: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.486501: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.486588: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.486592: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.486677: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.486681: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.486748: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.486751: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.486752: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.486813: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.486816: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.486821: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.486881: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.486885: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.486967: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.486971: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.487059: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.487063: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.487148: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.487152: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.487220: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.487223: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.487223: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.487278: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.487282: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.487286: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.487296: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.487393: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.487397: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.487482: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.487486: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.487584: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.487588: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.487680: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.487683: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.487770: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.487773: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.487850: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.487853: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.487857: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.487868: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.487966: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.488042: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.488046: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.488133: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.488137: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.488221: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.488225: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.488323: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.488325: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.488326: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.488406: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.488413: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.488416: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.488477: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.488481: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.488582: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.488661: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.488664: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.488750: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.488754: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.488868: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.488876: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.488961: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.488964: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.489042: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.489046: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.489049: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.489059: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.489151: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.489154: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.489254: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.489258: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.489344: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.489348: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.489453: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.489458: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.489461: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.489464: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.489525: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.489529: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.489532: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.489537: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.489633: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.489637: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.489731: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.489735: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.489831: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.489835: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.489937: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.489942: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.489944: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.489947: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.490008: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.490014: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.490017: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.490021: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.490116: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.490120: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.490231: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.490235: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.490333: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.490336: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.490447: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.490452: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.490517: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.490520: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.490616: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.490620: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.490623: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.490627: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.490741: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.490745: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.490854: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.490857: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.490970: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.490974: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.491100: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.491105: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.491107: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.491107: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.491174: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.491178: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.491180: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.491184: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.491285: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.491288: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.491365: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.491437: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.491441: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.491528: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.491532: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.491609: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.491720: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.491723: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.491800: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.491804: fsg_main_thread: finish_reply -> 0
     irq/17-dwc3-2522  [002] d...    43.491805: bulk_in_complete: 0, 16384/16384
    file-storage-2521  [001] ....    43.491808: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.491912: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.491915: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.492007: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.492010: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.492095: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.492099: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.492188: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.492193: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.492290: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.492292: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.492359: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.492364: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.492368: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.492426: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.492430: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.492521: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.492525: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.492615: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.492619: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.492703: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.492707: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.492787: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.492881: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.492884: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.492946: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.492949: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.492952: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.492956: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.493068: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.493072: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.493159: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.493162: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.493252: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.493318: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.493322: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.493417: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.493510: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.493513: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.493571: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.493575: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.493579: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.493644: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.493648: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.493738: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.493741: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.493828: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.493832: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.493927: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.493931: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.494010: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.494105: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.494108: fsg_main_thread: get_next_command -> 0
     irq/17-dwc3-2522  [002] d...    43.494205: bulk_in_complete: 0, 16384/16384
    file-storage-2521  [001] ....    43.494205: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.494213: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.494216: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.494306: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.494323: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.494387: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.494454: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.494458: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.494544: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.494548: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.494624: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.494737: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.494740: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.494803: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.494807: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.494809: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.494813: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.494920: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.494923: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.495015: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.495017: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.495104: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.495107: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.495215: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.495224: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.495323: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.495325: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.495401: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.495405: fsg_main_thread: finish_reply -> 0
     irq/17-dwc3-2522  [002] d...    43.495406: bulk_in_complete: 0, 16384/16384
    file-storage-2521  [001] ....    43.495409: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.495485: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.495564: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.495567: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.495659: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.495662: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.495754: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.495821: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.495826: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.495927: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.495930: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.496009: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.496012: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.496016: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.496024: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.496128: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.496132: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.496231: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.496235: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.496322: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.496325: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.496412: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.496499: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.496622: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.496625: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.496708: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.496714: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.496718: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.496722: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.496822: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.496825: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.496914: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.496918: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.497003: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.497006: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.497091: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.497160: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.497268: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.497271: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.497337: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.497340: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.497344: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.497354: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.497445: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.497449: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.497533: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.497537: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.497648: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.497651: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.497779: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.497891: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.498016: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.498019: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.498084: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.498088: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.498091: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.498101: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.498194: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.498197: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.498283: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.498287: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.498375: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.498379: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.498504: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.498573: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.498660: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.498663: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.498760: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.498768: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.498770: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.498774: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.498867: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.498870: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.498957: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.498961: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.499047: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.499051: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.499177: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.499182: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.499185: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.499187: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.499273: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.499276: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.499279: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.499341: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.499346: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.499429: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.499433: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.499524: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.499528: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.499611: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.499615: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.499682: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.499684: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.499685: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.499750: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.499755: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.499760: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.499817: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.499821: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.499902: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.499906: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.499997: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.500001: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.500085: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.500089: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.500155: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.500157: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.500160: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.500224: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.500230: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.500234: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.500238: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.500329: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.500332: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.500419: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.500423: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.500509: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.500513: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.500603: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.500609: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.500687: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.500690: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.500747: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.500753: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.500757: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.500761: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.500868: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.500871: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.500963: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.500966: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.501051: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.501055: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.501168: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.501286: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.501289: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.501292: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.501352: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.501358: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.501360: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.501430: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.501434: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.501524: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.501529: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.501635: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.501639: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.501726: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.501730: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.501829: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.501832: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.501832: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.501894: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.501900: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.501906: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.501958: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.501962: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.502051: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.502128: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.502132: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.502227: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.502230: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.502323: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.502415: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.502417: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.502418: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.502486: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.502492: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.502496: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.502503: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.502597: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.502601: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.502691: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.502768: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.502772: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.502871: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.502875: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.502951: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.503055: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.503058: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.503133: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.503138: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.503142: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.503214: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.503217: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.503283: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.503359: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.503364: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.503440: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.503443: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.503537: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.503542: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.503620: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.503623: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.503682: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.503685: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.503689: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.503693: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.503792: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.503795: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.503893: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.503898: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.503984: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.503988: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.504096: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.504101: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.504199: bulk_out_complete: 0, 31/31
    file-storage-2521  [001] ....    43.504202: fsg_main_thread: get_next_command -> 0
    file-storage-2521  [001] ....    43.504288: fsg_main_thread: do_scsi_command -> 0
    file-storage-2521  [001] ....    43.504295: fsg_main_thread: finish_reply -> 0
    file-storage-2521  [001] ....    43.504298: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2522  [002] d...    43.504347: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.504351: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.504434: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.504438: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.504535: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.504539: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.504618: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.504703: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2522  [002] d...    43.504794: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2522  [002] d...    43.504797: bulk_out_complete: 0, 31/31

-- 
balbi

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 800 bytes --]

Re: Memory barrier needed with wake_up_process()?

[Felipe Balbi]

[-- Attachment #1.1: Type: text/plain, Size: 16299 bytes --]


Hi,

Felipe Balbi <felipe.balbi@linux.intel.com> writes:
> [ Unknown signature status ]
>
> Hi,
>
> Alan Stern <stern@rowland.harvard.edu> writes:
>
> [...]
>
>>> $ grep -RnH "raise_exception\|fsg_main_thread" /tmp/trace.txt 
>>> /tmp/trace.txt:111743:     irq/17-dwc3-3527  [003] d..1    64.833078: raise_exception: raise_exception 4
>>> /tmp/trace.txt:111745:    file-storage-3526  [002] ....    64.833139: fsg_main_thread: get_next_command -> -4
>>> /tmp/trace.txt:111746:    file-storage-3526  [002] ....    64.833140: fsg_main_thread: handling exception
>>> /tmp/trace.txt:112950:     irq/17-dwc3-3527  [003] d..1    64.951349: raise_exception: raise_exception 4
>>> /tmp/trace.txt:112956:    file-storage-3526  [002] ....    64.951401: fsg_main_thread: handling exception
>>> 
>>> Any ideas?
>>
>> I'm afraid not.  The only thing I can think of to try next is complete 
>> tracing of fsg_main_thread, as in the patch below.  It will generate 
>> continuous output as long as the gadget is doing something, but there 
>> doesn't seem to be any way to avoid this.  At least when everything 
>> stops, it should be able to tell us exactly where and why.
>
> tried with your changes plus a trace_printk() added to both
> bulk_out_complete and bulk_in_complete. Here's the diff:

another version of diff and logs. Any ideas of what could be going on?
(full trace compressed and attached)

diff --git a/drivers/usb/gadget/function/f_mass_storage.c b/drivers/usb/gadget/function/f_mass_storage.c
index 8f3659b65f53..de0a9ebe7f61 100644
--- a/drivers/usb/gadget/function/f_mass_storage.c
+++ b/drivers/usb/gadget/function/f_mass_storage.c
@@ -395,11 +395,24 @@ static int fsg_set_halt(struct fsg_dev *fsg, struct usb_ep *ep)
 /* Caller must hold fsg->lock */
 static void wakeup_thread(struct fsg_common *common)
 {
+	struct task_struct *p = common->thread_task;
+
 	smp_wmb();	/* ensure the write of bh->state is complete */
 	/* Tell the main thread that something has happened */
 	common->thread_wakeup_needed = 1;
-	if (common->thread_task)
-		wake_up_process(common->thread_task);
+
+	trace_printk("wkup needed %d task %p [comm=%s pid=%d prio=%d target_cpu=%03d on_rq %d state %lx]\n",
+			common->thread_wakeup_needed, p, p->comm, p->pid,
+			p->prio, task_cpu(p), p->on_rq, p->state);
+
+	if (common->thread_task) {
+		int rc;
+		rc = wake_up_process(common->thread_task);
+		if (rc)
+			trace_printk("thread_task woken up\n");
+		else
+			trace_printk("NOT woken up\n");
+	}
 }
 
 static void raise_exception(struct fsg_common *common, enum fsg_state new_state)
@@ -411,6 +424,7 @@ static void raise_exception(struct fsg_common *common, enum fsg_state new_state)
 	 * If a lower-or-equal priority exception is in progress, preempt it
 	 * and notify the main thread by sending it a signal.
 	 */
+	trace_printk("raise_exception %d\n", new_state);
 	spin_lock_irqsave(&common->lock, flags);
 	if (common->state <= new_state) {
 		common->exception_req_tag = common->ep0_req_tag;
@@ -449,6 +463,8 @@ static void bulk_in_complete(struct usb_ep *ep, struct usb_request *req)
 	struct fsg_common	*common = ep->driver_data;
 	struct fsg_buffhd	*bh = req->context;
 
+	trace_printk("%d, %u/%u\n", req->status, req->actual, req->length);
+
 	if (req->status || req->actual != req->length)
 		DBG(common, "%s --> %d, %u/%u\n", __func__,
 		    req->status, req->actual, req->length);
@@ -470,6 +486,8 @@ static void bulk_out_complete(struct usb_ep *ep, struct usb_request *req)
 	struct fsg_buffhd	*bh = req->context;
 
 	dump_msg(common, "bulk-out", req->buf, req->actual);
+
+	trace_printk("%d, %u/%u\n", req->status, req->actual, bh->bulk_out_intended_length);
 	if (req->status || req->actual != bh->bulk_out_intended_length)
 		DBG(common, "%s --> %d, %u/%u\n", __func__,
 		    req->status, req->actual, bh->bulk_out_intended_length);
@@ -573,6 +591,10 @@ static void start_transfer(struct fsg_dev *fsg, struct usb_ep *ep,
 	spin_unlock_irq(&fsg->common->lock);
 
 	rc = usb_ep_queue(ep, req, GFP_KERNEL);
+
+	trace_printk("%d: %s: req %u bytes state %d ---> %d\n", __LINE__, ep->name,
+			req->length, *state, rc);
+
 	if (rc == 0)
 		return;  /* All good, we're done */
 
@@ -2496,6 +2518,7 @@ static void handle_exception(struct fsg_common *common)
 static int fsg_main_thread(void *common_)
 {
 	struct fsg_common	*common = common_;
+	int			rc;
 
 	/*
 	 * Allow the thread to be killed by a signal, but set the signal mask
@@ -2519,6 +2542,7 @@ static int fsg_main_thread(void *common_)
 	/* The main loop */
 	while (common->state != FSG_STATE_TERMINATED) {
 		if (exception_in_progress(common) || signal_pending(current)) {
+			trace_printk("handling exception\n");
 			handle_exception(common);
 			continue;
 		}
@@ -2528,7 +2552,9 @@ static int fsg_main_thread(void *common_)
 			continue;
 		}
 
-		if (get_next_command(common))
+		rc = get_next_command(common);
+		trace_printk("get_next_command -> %d\n", rc);
+		if (rc)
 			continue;
 
 		spin_lock_irq(&common->lock);
@@ -2536,16 +2562,24 @@ static int fsg_main_thread(void *common_)
 			common->state = FSG_STATE_DATA_PHASE;
 		spin_unlock_irq(&common->lock);
 
-		if (do_scsi_command(common) || finish_reply(common))
+		rc = do_scsi_command(common);
+		trace_printk("do_scsi_command -> %d\n", rc);
+		if (rc)
 			continue;
+		rc = finish_reply(common);
+		trace_printk("finish_reply -> %d\n", rc);
+		if (rc)
+		       continue;
 
 		spin_lock_irq(&common->lock);
 		if (!exception_in_progress(common))
 			common->state = FSG_STATE_STATUS_PHASE;
 		spin_unlock_irq(&common->lock);
 
-		if (send_status(common))
-			continue;
+		rc = send_status(common);
+		trace_printk("send_status -> %d\n", rc);
+		if (rc)
+		       continue;
 
 		spin_lock_irq(&common->lock);
 		if (!exception_in_progress(common))





     irq/17-dwc3-2533  [002] d...    29.576760: bulk_out_complete: 0, 31/31
     irq/17-dwc3-2533  [002] d..1    29.576760: wakeup_thread.isra.15: wkup needed 1 task ffff88016e3dd100 [comm=file-storage pid=2532 prio=120 target_cpu=003 on_rq 0 state 1]
     irq/17-dwc3-2533  [002] d..1    29.576761: wakeup_thread.isra.15: thread_task woken up
    file-storage-2532  [003] ....    29.576763: fsg_main_thread: get_next_command -> 0
    file-storage-2532  [003] ....    29.576794: start_transfer.isra.14: 596: ep1in: req 16384 bytes state 2 ---> 0
    file-storage-2532  [003] ....    29.576802: start_transfer.isra.14: 596: ep1in: req 16384 bytes state 2 ---> 0
    file-storage-2532  [003] ....    29.576812: start_transfer.isra.14: 596: ep1in: req 16384 bytes state 2 ---> 0
    file-storage-2532  [003] ....    29.576834: start_transfer.isra.14: 596: ep1in: req 16384 bytes state 2 ---> 0
    file-storage-2532  [003] ....    29.576845: start_transfer.isra.14: 596: ep1in: req 16384 bytes state 2 ---> 0
    file-storage-2532  [003] ....    29.576852: start_transfer.isra.14: 596: ep1in: req 16384 bytes state 2 ---> 0
    file-storage-2532  [003] ....    29.576862: start_transfer.isra.14: 596: ep1in: req 16384 bytes state 2 ---> 0
    file-storage-2532  [003] ....    29.576867: fsg_main_thread: do_scsi_command -> 0
    file-storage-2532  [003] ....    29.576873: start_transfer.isra.14: 596: ep1in: req 16384 bytes state 2 ---> 0
    file-storage-2532  [003] ....    29.576873: fsg_main_thread: finish_reply -> 0
    file-storage-2532  [003] ....    29.576876: start_transfer.isra.14: 596: ep1in: req 13 bytes state 2 ---> 0
    file-storage-2532  [003] ....    29.576877: fsg_main_thread: send_status -> 0
     irq/17-dwc3-2533  [002] d...    29.576877: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2533  [002] d..1    29.576878: wakeup_thread.isra.15: wkup needed 1 task ffff88016e3dd100 [comm=file-storage pid=2532 prio=120 target_cpu=003 on_rq 1 state 0]
     irq/17-dwc3-2533  [002] d..1    29.576878: wakeup_thread.isra.15: NOT woken up
    file-storage-2532  [003] ....    29.576880: start_transfer.isra.14: 596: ep1out: req 1024 bytes state 2 ---> 0
     irq/17-dwc3-2533  [002] d...    29.576971: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2533  [002] d..1    29.576971: wakeup_thread.isra.15: wkup needed 1 task ffff88016e3dd100 [comm=file-storage pid=2532 prio=120 target_cpu=003 on_rq 0 state 1]
     irq/17-dwc3-2533  [002] d..1    29.576972: wakeup_thread.isra.15: thread_task woken up
     irq/17-dwc3-2533  [002] d...    29.576975: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2533  [002] d..1    29.576976: wakeup_thread.isra.15: wkup needed 1 task ffff88016e3dd100 [comm=file-storage pid=2532 prio=120 target_cpu=003 on_rq 0 state 1]
     irq/17-dwc3-2533  [002] d..1    29.576976: wakeup_thread.isra.15: thread_task woken up
     irq/17-dwc3-2533  [002] d...    29.577060: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2533  [002] d..1    29.577061: wakeup_thread.isra.15: wkup needed 1 task ffff88016e3dd100 [comm=file-storage pid=2532 prio=120 target_cpu=003 on_rq 0 state 1]
     irq/17-dwc3-2533  [002] d..1    29.577062: wakeup_thread.isra.15: thread_task woken up
     irq/17-dwc3-2533  [002] d...    29.577065: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2533  [002] d..1    29.577065: wakeup_thread.isra.15: wkup needed 1 task ffff88016e3dd100 [comm=file-storage pid=2532 prio=120 target_cpu=003 on_rq 0 state 1]
     irq/17-dwc3-2533  [002] d..1    29.577066: wakeup_thread.isra.15: thread_task woken up
     irq/17-dwc3-2533  [002] d...    29.577149: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2533  [002] d..1    29.577150: wakeup_thread.isra.15: wkup needed 1 task ffff88016e3dd100 [comm=file-storage pid=2532 prio=120 target_cpu=003 on_rq 0 state 1]
     irq/17-dwc3-2533  [002] d..1    29.577150: wakeup_thread.isra.15: thread_task woken up
     irq/17-dwc3-2533  [002] d...    29.577153: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2533  [002] d..1    29.577154: wakeup_thread.isra.15: wkup needed 1 task ffff88016e3dd100 [comm=file-storage pid=2532 prio=120 target_cpu=003 on_rq 0 state 1]
     irq/17-dwc3-2533  [002] d..1    29.577155: wakeup_thread.isra.15: thread_task woken up
     irq/17-dwc3-2533  [002] d...    29.577241: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2533  [002] d..1    29.577242: wakeup_thread.isra.15: wkup needed 1 task ffff88016e3dd100 [comm=file-storage pid=2532 prio=120 target_cpu=003 on_rq 0 state 1]
     irq/17-dwc3-2533  [002] d..1    29.577242: wakeup_thread.isra.15: thread_task woken up
     irq/17-dwc3-2533  [002] d...    29.577246: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2533  [002] d..1    29.577246: wakeup_thread.isra.15: wkup needed 1 task ffff88016e3dd100 [comm=file-storage pid=2532 prio=120 target_cpu=003 on_rq 0 state 1]
     irq/17-dwc3-2533  [002] d..1    29.577247: wakeup_thread.isra.15: thread_task woken up
     irq/17-dwc3-2533  [002] d...    29.577309: bulk_out_complete: 0, 31/31
     irq/17-dwc3-2533  [002] d..1    29.577309: wakeup_thread.isra.15: wkup needed 1 task ffff88016e3dd100 [comm=file-storage pid=2532 prio=120 target_cpu=003 on_rq 0 state 1]
     irq/17-dwc3-2533  [002] d..1    29.577310: wakeup_thread.isra.15: thread_task woken up
    file-storage-2532  [003] ....    29.577312: fsg_main_thread: get_next_command -> 0
    file-storage-2532  [003] ....    29.577340: start_transfer.isra.14: 596: ep1in: req 16384 bytes state 2 ---> 0
    file-storage-2532  [003] ....    29.577361: start_transfer.isra.14: 596: ep1in: req 16384 bytes state 2 ---> 0
    file-storage-2532  [003] ....    29.577369: start_transfer.isra.14: 596: ep1in: req 16384 bytes state 2 ---> 0
    file-storage-2532  [003] ....    29.577379: start_transfer.isra.14: 596: ep1in: req 16384 bytes state 2 ---> 0
    file-storage-2532  [003] ....    29.577387: start_transfer.isra.14: 596: ep1in: req 16384 bytes state 2 ---> 0
    file-storage-2532  [003] ....    29.577394: start_transfer.isra.14: 596: ep1in: req 16384 bytes state 2 ---> 0
    file-storage-2532  [003] ....    29.577404: start_transfer.isra.14: 596: ep1in: req 16384 bytes state 2 ---> 0
    file-storage-2532  [003] ....    29.577407: fsg_main_thread: do_scsi_command -> 0
    file-storage-2532  [003] ....    29.577410: start_transfer.isra.14: 596: ep1in: req 16384 bytes state 2 ---> 0
    file-storage-2532  [003] ....    29.577411: fsg_main_thread: finish_reply -> 0
    file-storage-2532  [003] ....    29.577414: start_transfer.isra.14: 596: ep1in: req 13 bytes state 2 ---> 0
    file-storage-2532  [003] ....    29.577414: fsg_main_thread: send_status -> 0
    file-storage-2532  [003] ....    29.577418: start_transfer.isra.14: 596: ep1out: req 1024 bytes state 2 ---> 0
     irq/17-dwc3-2533  [002] d...    29.577419: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2533  [002] d..1    29.577419: wakeup_thread.isra.15: wkup needed 1 task ffff88016e3dd100 [comm=file-storage pid=2532 prio=120 target_cpu=003 on_rq 0 state 1]
     irq/17-dwc3-2533  [002] d..1    29.577420: wakeup_thread.isra.15: thread_task woken up
     irq/17-dwc3-2533  [002] d...    29.577512: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2533  [002] d..1    29.577513: wakeup_thread.isra.15: wkup needed 1 task ffff88016e3dd100 [comm=file-storage pid=2532 prio=120 target_cpu=003 on_rq 0 state 1]
     irq/17-dwc3-2533  [002] d..1    29.577513: wakeup_thread.isra.15: thread_task woken up
     irq/17-dwc3-2533  [002] d...    29.577516: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2533  [002] d..1    29.577517: wakeup_thread.isra.15: wkup needed 1 task ffff88016e3dd100 [comm=file-storage pid=2532 prio=120 target_cpu=003 on_rq 0 state 1]
     irq/17-dwc3-2533  [002] d..1    29.577517: wakeup_thread.isra.15: thread_task woken up
     irq/17-dwc3-2533  [002] d...    29.577603: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2533  [002] d..1    29.577603: wakeup_thread.isra.15: wkup needed 1 task ffff88016e3dd100 [comm=file-storage pid=2532 prio=120 target_cpu=003 on_rq 0 state 1]
     irq/17-dwc3-2533  [002] d..1    29.577604: wakeup_thread.isra.15: thread_task woken up
     irq/17-dwc3-2533  [002] d...    29.577607: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2533  [002] d..1    29.577608: wakeup_thread.isra.15: wkup needed 1 task ffff88016e3dd100 [comm=file-storage pid=2532 prio=120 target_cpu=003 on_rq 0 state 1]
     irq/17-dwc3-2533  [002] d..1    29.577608: wakeup_thread.isra.15: thread_task woken up
     irq/17-dwc3-2533  [002] d...    29.577695: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2533  [002] d..1    29.577695: wakeup_thread.isra.15: wkup needed 1 task ffff88016e3dd100 [comm=file-storage pid=2532 prio=120 target_cpu=003 on_rq 0 state 1]
     irq/17-dwc3-2533  [002] d..1    29.577696: wakeup_thread.isra.15: thread_task woken up
     irq/17-dwc3-2533  [002] d...    29.577699: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2533  [002] d..1    29.577700: wakeup_thread.isra.15: wkup needed 1 task ffff88016e3dd100 [comm=file-storage pid=2532 prio=120 target_cpu=003 on_rq 0 state 1]
     irq/17-dwc3-2533  [002] d..1    29.577701: wakeup_thread.isra.15: thread_task woken up
     irq/17-dwc3-2533  [002] d...    29.577835: bulk_in_complete: 0, 16384/16384
     irq/17-dwc3-2533  [002] d..1    29.577836: wakeup_thread.isra.15: wkup needed 1 task ffff88016e3dd100 [comm=file-storage pid=2532 prio=120 target_cpu=003 on_rq 0 state 1]
     irq/17-dwc3-2533  [002] d..1    29.577837: wakeup_thread.isra.15: thread_task woken up
     irq/17-dwc3-2533  [002] d...    29.577840: bulk_in_complete: 0, 13/13
     irq/17-dwc3-2533  [002] d..1    29.577840: wakeup_thread.isra.15: wkup needed 1 task ffff88016e3dd100 [comm=file-storage pid=2532 prio=120 target_cpu=003 on_rq 0 state 1]
     irq/17-dwc3-2533  [002] d..1    29.577841: wakeup_thread.isra.15: thread_task woken up
     irq/17-dwc3-2533  [002] d...    29.577842: bulk_out_complete: 0, 31/31
     irq/17-dwc3-2533  [002] d..1    29.577843: wakeup_thread.isra.15: wkup needed 1 task ffff88016e3dd100 [comm=file-storage pid=2532 prio=120 target_cpu=003 on_rq 1 state 0]
     irq/17-dwc3-2533  [002] d..1    29.577844: wakeup_thread.isra.15: thread_task woken up


[-- Attachment #1.2: trace.txt.xz --]
[-- Type: application/x-xz, Size: 44564 bytes --]

[-- Attachment #1.3: Type: text/plain, Size: 16 bytes --]



-- 
balbi

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 800 bytes --]

Re: Memory barrier needed with wake_up_process()?

[Alan Stern]

On Tue, 20 Sep 2016, Felipe Balbi wrote:

> And here's trace output (complete, scroll to bottom). It seems to me
> like the thread didn't wake up at all. It didn't even try to
> execute. I'll add some more traces and try to get better information
> about what's going on.
> 
> 
> 
> # tracer: nop
> #
> # entries-in-buffer/entries-written: 2865/2865   #P:4
> #
> #                              _-----=> irqs-off
> #                             / _----=> need-resched
> #                            | / _---=> hardirq/softirq
> #                            || / _--=> preempt-depth
> #                            ||| /     delay
> #           TASK-PID   CPU#  ||||    TIMESTAMP  FUNCTION
> #              | |       |   ||||       |         |

Skipping to the end...

>      irq/17-dwc3-2522  [002] d...    43.504199: bulk_out_complete: 0, 31/31
>     file-storage-2521  [001] ....    43.504202: fsg_main_thread: get_next_command -> 0
>     file-storage-2521  [001] ....    43.504288: fsg_main_thread: do_scsi_command -> 0
>     file-storage-2521  [001] ....    43.504295: fsg_main_thread: finish_reply -> 0
>     file-storage-2521  [001] ....    43.504298: fsg_main_thread: send_status -> 0
>      irq/17-dwc3-2522  [002] d...    43.504347: bulk_in_complete: 0, 16384/16384
>      irq/17-dwc3-2522  [002] d...    43.504351: bulk_in_complete: 0, 16384/16384
>      irq/17-dwc3-2522  [002] d...    43.504434: bulk_in_complete: 0, 16384/16384
>      irq/17-dwc3-2522  [002] d...    43.504438: bulk_in_complete: 0, 16384/16384
>      irq/17-dwc3-2522  [002] d...    43.504535: bulk_in_complete: 0, 16384/16384
>      irq/17-dwc3-2522  [002] d...    43.504539: bulk_in_complete: 0, 16384/16384
>      irq/17-dwc3-2522  [002] d...    43.504618: bulk_in_complete: 0, 16384/16384
>      irq/17-dwc3-2522  [002] d...    43.504703: bulk_in_complete: 0, 16384/16384
>      irq/17-dwc3-2522  [002] d...    43.504794: bulk_in_complete: 0, 13/13
>      irq/17-dwc3-2522  [002] d...    43.504797: bulk_out_complete: 0, 31/31

Like you say, it appears that the thread didn't get woken up at all.
But this is inconsistent with your earlier results.  On Sep. 9, you 
posted a message that ended with these lines:

>      irq/17-dwc3-3579  [003] d..1 21167.729666: bulk_out_complete: compl: bh ffff880111e6aac0 state 1
>     file-storage-3578  [002] .... 21167.729670: fsg_main_thread: next: bh ffff880111e6aac0 state 1

This indicates that in the earlier test, the thread did start running 
and get_next_command should have returned.

The trace you posted after this one doesn't seem to show anything new, 
as far as I can tell.

So I still can't tell what's happening.  Maybe the patch below will 
help.  It concentrates on the critical area.

Alan Stern



Index: usb-4.x/drivers/usb/gadget/function/f_mass_storage.c
===================================================================
--- usb-4.x.orig/drivers/usb/gadget/function/f_mass_storage.c
+++ usb-4.x/drivers/usb/gadget/function/f_mass_storage.c
@@ -402,8 +402,12 @@ static void wakeup_thread(struct fsg_com
 	smp_wmb();	/* ensure the write of bh->state is complete */
 	/* Tell the main thread that something has happened */
 	common->thread_wakeup_needed = 1;
-	if (common->thread_task)
-		wake_up_process(common->thread_task);
+	if (common->thread_task) {
+		if (wake_up_process(common->thread_task))
+			trace_printk("wakeup_thread 1\n");
+		else
+			trace_printk("wakeup_thread 0\n");
+	}
 }
 
 static void raise_exception(struct fsg_common *common, enum fsg_state new_state)
@@ -479,6 +483,8 @@ static void bulk_out_complete(struct usb
 		    req->status, req->actual, bh->bulk_out_intended_length);
 	if (req->status == -ECONNRESET)		/* Request was cancelled */
 		usb_ep_fifo_flush(ep);
+	trace_printk("bulk_out: %d, %u/%u\n", req->status, req->actual,
+			bh->bulk_out_intended_length);
 
 	/* Hold the lock while we update the request and buffer states */
 	smp_wmb();
@@ -2204,13 +2210,17 @@ static int get_next_command(struct fsg_c
 
 	/* Wait for the CBW to arrive */
 	while (bh->state != BUF_STATE_FULL) {
+		trace_printk("get_next: sleep\n");
 		rc = sleep_thread(common, true);
+		trace_printk("get_next: sleep -> %d loop %d\n",
+				rc, bh->state != BUF_STATE_FULL);
 		if (rc)
 			return rc;
 	}
 	smp_rmb();
 	rc = fsg_is_set(common) ? received_cbw(common->fsg, bh) : -EIO;
 	bh->state = BUF_STATE_EMPTY;
+	trace_printk("get_next: return %d\n", rc);
 
 	return rc;
 }

Re: Memory barrier needed with wake_up_process()?

[Felipe Balbi]

[-- Attachment #1: Type: text/plain, Size: 3752 bytes --]


Hi,

Alan Stern <stern@rowland.harvard.edu> writes:
> On Tue, 20 Sep 2016, Felipe Balbi wrote:
>
>> And here's trace output (complete, scroll to bottom). It seems to me
>> like the thread didn't wake up at all. It didn't even try to
>> execute. I'll add some more traces and try to get better information
>> about what's going on.
>> 
>> 
>> 
>> # tracer: nop
>> #
>> # entries-in-buffer/entries-written: 2865/2865   #P:4
>> #
>> #                              _-----=> irqs-off
>> #                             / _----=> need-resched
>> #                            | / _---=> hardirq/softirq
>> #                            || / _--=> preempt-depth
>> #                            ||| /     delay
>> #           TASK-PID   CPU#  ||||    TIMESTAMP  FUNCTION
>> #              | |       |   ||||       |         |
>
> Skipping to the end...
>
>>      irq/17-dwc3-2522  [002] d...    43.504199: bulk_out_complete: 0, 31/31
>>     file-storage-2521  [001] ....    43.504202: fsg_main_thread: get_next_command -> 0
>>     file-storage-2521  [001] ....    43.504288: fsg_main_thread: do_scsi_command -> 0
>>     file-storage-2521  [001] ....    43.504295: fsg_main_thread: finish_reply -> 0
>>     file-storage-2521  [001] ....    43.504298: fsg_main_thread: send_status -> 0
>>      irq/17-dwc3-2522  [002] d...    43.504347: bulk_in_complete: 0, 16384/16384
>>      irq/17-dwc3-2522  [002] d...    43.504351: bulk_in_complete: 0, 16384/16384
>>      irq/17-dwc3-2522  [002] d...    43.504434: bulk_in_complete: 0, 16384/16384
>>      irq/17-dwc3-2522  [002] d...    43.504438: bulk_in_complete: 0, 16384/16384
>>      irq/17-dwc3-2522  [002] d...    43.504535: bulk_in_complete: 0, 16384/16384
>>      irq/17-dwc3-2522  [002] d...    43.504539: bulk_in_complete: 0, 16384/16384
>>      irq/17-dwc3-2522  [002] d...    43.504618: bulk_in_complete: 0, 16384/16384
>>      irq/17-dwc3-2522  [002] d...    43.504703: bulk_in_complete: 0, 16384/16384
>>      irq/17-dwc3-2522  [002] d...    43.504794: bulk_in_complete: 0, 13/13
>>      irq/17-dwc3-2522  [002] d...    43.504797: bulk_out_complete: 0, 31/31
>
> Like you say, it appears that the thread didn't get woken up at all.
> But this is inconsistent with your earlier results.  On Sep. 9, you 
> posted a message that ended with these lines:
>
>>      irq/17-dwc3-3579  [003] d..1 21167.729666: bulk_out_complete: compl: bh ffff880111e6aac0 state 1
>>     file-storage-3578  [002] .... 21167.729670: fsg_main_thread: next: bh ffff880111e6aac0 state 1
>
> This indicates that in the earlier test, the thread did start running 
> and get_next_command should have returned.
>
> The trace you posted after this one doesn't seem to show anything new, 
> as far as I can tell.
>
> So I still can't tell what's happening.  Maybe the patch below will 
> help.  It concentrates on the critical area.

Sorry for the long delay, I finally have more information on this. All
this time I was doing something that I never considered to matter: I've
been running host and peripheral on the same machine. Now that I have
tracepoints on xHCI as well, I could see that these 30 seconds of
"nothing" is actuall full of xHCI activity and I can see that for the
duration of these 30 seconds preempt depth on the CPU that (eventually)
queues a request on dwc3, is always > 1 (sometimes 2, most of the time
1). My conclusion from that is that xHCI (or usbcore ?!?) locks the CPU
and g_mass_storage is spinning for over 30 seconds at which point
storage.ko (host side class driver) dequeues the request.

I'll see if I can capture a fresh trace with both xHCI and dwc3 with
this happening, but probably not today (testing stuff for -rc).

-- 
balbi

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: Memory barrier needed with wake_up_process()?

[Alan Stern]

On Mon, 16 Jan 2017, Felipe Balbi wrote:

> Sorry for the long delay, I finally have more information on this. All
> this time I was doing something that I never considered to matter: I've
> been running host and peripheral on the same machine. Now that I have
> tracepoints on xHCI as well, I could see that these 30 seconds of
> "nothing" is actuall full of xHCI activity and I can see that for the
> duration of these 30 seconds preempt depth on the CPU that (eventually)
> queues a request on dwc3, is always > 1 (sometimes 2, most of the time
> 1). My conclusion from that is that xHCI (or usbcore ?!?) locks the CPU
> and g_mass_storage is spinning for over 30 seconds at which point
> storage.ko (host side class driver) dequeues the request.
> 
> I'll see if I can capture a fresh trace with both xHCI and dwc3 with
> this happening, but probably not today (testing stuff for -rc).

Does anything change if the host and peripheral are separate machines?

Alan Stern

Re: Memory barrier needed with wake_up_process()?

[Felipe Balbi]

Hi,

Alan Stern <stern@rowland.harvard.edu> writes:
> On Mon, 16 Jan 2017, Felipe Balbi wrote:
>
>> Sorry for the long delay, I finally have more information on this. All
>> this time I was doing something that I never considered to matter: I've
>> been running host and peripheral on the same machine. Now that I have
>> tracepoints on xHCI as well, I could see that these 30 seconds of
>> "nothing" is actuall full of xHCI activity and I can see that for the
>> duration of these 30 seconds preempt depth on the CPU that (eventually)
>> queues a request on dwc3, is always > 1 (sometimes 2, most of the time
>> 1). My conclusion from that is that xHCI (or usbcore ?!?) locks the CPU
>> and g_mass_storage is spinning for over 30 seconds at which point
>> storage.ko (host side class driver) dequeues the request.
>> 
>> I'll see if I can capture a fresh trace with both xHCI and dwc3 with
>> this happening, but probably not today (testing stuff for -rc).
>
> Does anything change if the host and peripheral are separate machines?

couldn't reproduce the problem yet ;-)

-- 
balbi

Re: Memory barrier needed with wake_up_process()?

[Felipe Balbi]

Hi,

Felipe Balbi <felipe.balbi@linux.intel.com> writes:
> Alan Stern <stern@rowland.harvard.edu> writes:
>> On Mon, 16 Jan 2017, Felipe Balbi wrote:
>>
>>> Sorry for the long delay, I finally have more information on this. All
>>> this time I was doing something that I never considered to matter: I've
>>> been running host and peripheral on the same machine. Now that I have
>>> tracepoints on xHCI as well, I could see that these 30 seconds of
>>> "nothing" is actuall full of xHCI activity and I can see that for the
>>> duration of these 30 seconds preempt depth on the CPU that (eventually)
>>> queues a request on dwc3, is always > 1 (sometimes 2, most of the time
>>> 1). My conclusion from that is that xHCI (or usbcore ?!?) locks the CPU
>>> and g_mass_storage is spinning for over 30 seconds at which point
>>> storage.ko (host side class driver) dequeues the request.
>>> 
>>> I'll see if I can capture a fresh trace with both xHCI and dwc3 with
>>> this happening, but probably not today (testing stuff for -rc).
>>
>> Does anything change if the host and peripheral are separate machines?
>
> couldn't reproduce the problem yet ;-)

*yet* is a keyword here. I wouldn't call this problem "done" until I
 successfully run my test case for at least a week.

-- 
balbi

Re: Memory barrier needed with wake_up_process()?

[Peter Zijlstra]

On Fri, Sep 02, 2016 at 02:10:13PM -0400, Alan Stern wrote:
> Paul, Peter, and Ingo:
> 
> This must have come up before, but I don't know what was decided.
> 
> Isn't it often true that a memory barrier is needed before a call to 
> wake_up_process()?  A typical scenario might look like this:
> 
> 	CPU 0
> 	-----
> 	for (;;) {
> 		set_current_state(TASK_INTERRUPTIBLE);
> 		if (signal_pending(current))
> 			break;
> 		if (wakeup_flag)
> 			break;
> 		schedule();
> 	}
> 	__set_current_state(TASK_RUNNING);
> 	wakeup_flag = 0;
> 
> 
> 	CPU 1
> 	-----
> 	wakeup_flag = 1;
> 	wake_up_process(my_task);
> 
> The underlying pattern is:
> 
> 	CPU 0				CPU 1
> 	-----				-----
> 	write current->state		write wakeup_flag
> 	smp_mb();
> 	read wakeup_flag		read my_task->state
> 
> where set_current_state() does the write to current->state and 
> automatically adds the smp_mb(), and wake_up_process() reads 
> my_task->state to see whether the task needs to be woken up.
> 
> The kerneldoc for wake_up_process() says that it has no implied memory
> barrier if it doesn't actually wake anything up.  And even when it
> does, the implied barrier is only smp_wmb, not smp_mb.
> 
> This is the so-called SB (Store Buffer) pattern, which is well known to
> require a full smp_mb on both sides.  Since wake_up_process() doesn't
> include smp_mb(), isn't it correct that the caller must add it
> explicitly?
> 
> In other words, shouldn't the code for CPU 1 really be:
> 
> 	wakeup_flag = 1;
> 	smp_mb();
> 	wake_up_process(task);
> 

No, it doesn't need to do that. try_to_wake_up() does the right thing.

It does:

	smp_mb__before_spinlock();
	raw_spin_lock_irqsave(&p->pi_lock);

Now, smp_mb__before_spinlock() is a bit of an odd duck, if you look at
its comment it says:

/*
 * Despite its name it doesn't necessarily has to be a full barrier.
 * It should only guarantee that a STORE before the critical section
 * can not be reordered with LOADs and STOREs inside this section.
 * spin_lock() is the one-way barrier, this LOAD can not escape out
 * of the region. So the default implementation simply ensures that
 * a STORE can not move into the critical section, smp_wmb() should
 * serialize it with another STORE done by spin_lock().
 */
#ifndef smp_mb__before_spinlock
#define smp_mb__before_spinlock()	smp_wmb()
#endif


So per default it ends up being:

	WMB
	LOCK

Which is sufficient to order the prior store vs the later load as is
required. Note that a spinlock acquire _must_ imply a store (we need to
mark the lock as taken), therefore the prior store is ordered against
the lock store per the wmb, and since the lock must imply an ACQUIRE
that limits the load.


Now, PowerPC defines smp_mb__before_spinlock as smp_mb(), and this is
because PowerPC ACQUIRE is a bit of an exception, if you want more
details I'm sure I or Paul can dredge them up :-)

Re: Memory barrier needed with wake_up_process()?

[Alan Stern]

On Fri, 2 Sep 2016, Peter Zijlstra wrote:

> On Fri, Sep 02, 2016 at 02:10:13PM -0400, Alan Stern wrote:
> > Paul, Peter, and Ingo:
> > 
> > This must have come up before, but I don't know what was decided.
> > 
> > Isn't it often true that a memory barrier is needed before a call to 
> > wake_up_process()?  A typical scenario might look like this:
> > 
> > 	CPU 0
> > 	-----
> > 	for (;;) {
> > 		set_current_state(TASK_INTERRUPTIBLE);
> > 		if (signal_pending(current))
> > 			break;
> > 		if (wakeup_flag)
> > 			break;
> > 		schedule();
> > 	}
> > 	__set_current_state(TASK_RUNNING);
> > 	wakeup_flag = 0;
> > 
> > 
> > 	CPU 1
> > 	-----
> > 	wakeup_flag = 1;
> > 	wake_up_process(my_task);
> > 
> > The underlying pattern is:
> > 
> > 	CPU 0				CPU 1
> > 	-----				-----
> > 	write current->state		write wakeup_flag
> > 	smp_mb();
> > 	read wakeup_flag		read my_task->state
> > 
> > where set_current_state() does the write to current->state and 
> > automatically adds the smp_mb(), and wake_up_process() reads 
> > my_task->state to see whether the task needs to be woken up.
> > 
> > The kerneldoc for wake_up_process() says that it has no implied memory
> > barrier if it doesn't actually wake anything up.  And even when it
> > does, the implied barrier is only smp_wmb, not smp_mb.
> > 
> > This is the so-called SB (Store Buffer) pattern, which is well known to
> > require a full smp_mb on both sides.  Since wake_up_process() doesn't
> > include smp_mb(), isn't it correct that the caller must add it
> > explicitly?
> > 
> > In other words, shouldn't the code for CPU 1 really be:
> > 
> > 	wakeup_flag = 1;
> > 	smp_mb();
> > 	wake_up_process(task);
> > 
> 
> No, it doesn't need to do that. try_to_wake_up() does the right thing.
> 
> It does:
> 
> 	smp_mb__before_spinlock();
> 	raw_spin_lock_irqsave(&p->pi_lock);
> 
> Now, smp_mb__before_spinlock() is a bit of an odd duck, if you look at
> its comment it says:
> 
> /*
>  * Despite its name it doesn't necessarily has to be a full barrier.
>  * It should only guarantee that a STORE before the critical section
>  * can not be reordered with LOADs and STOREs inside this section.
>  * spin_lock() is the one-way barrier, this LOAD can not escape out
>  * of the region. So the default implementation simply ensures that
>  * a STORE can not move into the critical section, smp_wmb() should
>  * serialize it with another STORE done by spin_lock().
>  */
> #ifndef smp_mb__before_spinlock
> #define smp_mb__before_spinlock()	smp_wmb()
> #endif

I see.  So the kerneldoc for wake_up_process() is misleading at best.

> So per default it ends up being:
> 
> 	WMB
> 	LOCK
> 
> Which is sufficient to order the prior store vs the later load as is
> required. Note that a spinlock acquire _must_ imply a store (we need to
> mark the lock as taken), therefore the prior store is ordered against
> the lock store per the wmb, and since the lock must imply an ACQUIRE
> that limits the load.

Actually, that's not entirely true (although presumably it works okay
for most architectures).  In theory, the first write and the WMB can
move down after the lock's ACQUIRE.  Then the later read could move
earlier, before the lock's store, the WMB, and the first write, but
still after the ACQUIRE.  Thus the later read could be reordered with
the first write.

In other words, these actions:

	store wakeup_flag
	WMB
	load-ACQUIRE lock
	store lock
	read task->state

can be reordered to:

	load-ACQUIRE lock
	store wakeup_flag
	WMB
	store lock
	read task->state

and then to:

	load-ACQUIRE lock
	read task->state
	store wakeup_flag
	WMB
	store lock

without violating any ordering rules.

> Now, PowerPC defines smp_mb__before_spinlock as smp_mb(), and this is
> because PowerPC ACQUIRE is a bit of an exception, if you want more
> details I'm sure I or Paul can dredge them up :-)

PPC can't do the reordering described above, but it does have other
weaknesses.  Without smp_mb(), the write to wakeup_flag doesn't have to
propagate from one CPU to the other before the second CPU tries to read
it.

Felipe, your tests will show whether my guess was totally off-base.  
For the new people, Felipe is tracking down a problem that involves
exactly the code sequence listed at the top of the email, where we know
that the wakeup routine runs but nevertheless the task sleeps.  At
least, that's what it looks like at the moment.

Alan Stern

Re: Memory barrier needed with wake_up_process()?

[Peter Zijlstra]

On Fri, Sep 02, 2016 at 04:16:54PM -0400, Alan Stern wrote:
> 
> Actually, that's not entirely true (although presumably it works okay
> for most architectures).

Yeah, all load-store archs (with exception of PowerPC and ARM64 and
possibly MIPS) implement ACQUIRE with a general fence (after the ll/sc).

( and MIPS doesn't use their fancy barriers in Linux )

PowerPC does the full fence for smp_mb__before_spinlock, which leaves
ARM64, I'm not sure its correct, but I'm way too tired to think about
that now.

The TSO archs imply full barriers with all atomic RmW ops and are
therefore also good.

Re: Memory barrier needed with wake_up_process()?

[Peter Zijlstra]

On Sat, Sep 03, 2016 at 12:14:13AM +0200, Peter Zijlstra wrote:
> On Fri, Sep 02, 2016 at 04:16:54PM -0400, Alan Stern wrote:
> > 
> > Actually, that's not entirely true (although presumably it works okay
> > for most architectures).
> 
> Yeah, all load-store archs (with exception of PowerPC and ARM64 and
> possibly MIPS) implement ACQUIRE with a general fence (after the ll/sc).
> 
> ( and MIPS doesn't use their fancy barriers in Linux )
> 
> PowerPC does the full fence for smp_mb__before_spinlock, which leaves
> ARM64, I'm not sure its correct, but I'm way too tired to think about
> that now.
> 
> The TSO archs imply full barriers with all atomic RmW ops and are
> therefore also good.
> 

Forgot to Cc Will. Will, does ARM64 need to make smp_mb__before_spinlock
smp_mb() too?

Re: Memory barrier needed with wake_up_process()?

[Will Deacon]

On Sat, Sep 03, 2016 at 12:16:29AM +0200, Peter Zijlstra wrote:
> On Sat, Sep 03, 2016 at 12:14:13AM +0200, Peter Zijlstra wrote:
> > On Fri, Sep 02, 2016 at 04:16:54PM -0400, Alan Stern wrote:
> > > 
> > > Actually, that's not entirely true (although presumably it works okay
> > > for most architectures).
> > 
> > Yeah, all load-store archs (with exception of PowerPC and ARM64 and
> > possibly MIPS) implement ACQUIRE with a general fence (after the ll/sc).
> > 
> > ( and MIPS doesn't use their fancy barriers in Linux )
> > 
> > PowerPC does the full fence for smp_mb__before_spinlock, which leaves
> > ARM64, I'm not sure its correct, but I'm way too tired to think about
> > that now.
> > 
> > The TSO archs imply full barriers with all atomic RmW ops and are
> > therefore also good.
> > 
> 
> Forgot to Cc Will. Will, does ARM64 need to make smp_mb__before_spinlock
> smp_mb() too?

Yes, probably. Just to confirm, the test is something like:


CPU0
----

Wx=1
smp_mb__before_spinlock()
LOCK(y)
Rz=0

CPU1
----

Wz=1
smp_mb()
Rx=0


and that should be forbidden?

Will

Re: Memory barrier needed with wake_up_process()?

[Peter Zijlstra]

On Mon, Sep 05, 2016 at 10:43:11AM +0100, Will Deacon wrote:
> On Sat, Sep 03, 2016 at 12:16:29AM +0200, Peter Zijlstra wrote:

> > Forgot to Cc Will. Will, does ARM64 need to make smp_mb__before_spinlock
> > smp_mb() too?
> 
> Yes, probably. Just to confirm, the test is something like:
> 
> 
> CPU0
> ----
> 
> Wx=1
> smp_mb__before_spinlock()
> LOCK(y)
> Rz=0
> 
> CPU1
> ----
> 
> Wz=1
> smp_mb()
> Rx=0
> 
> 
> and that should be forbidden?

Indeed.

Re: Memory barrier needed with wake_up_process()?

[Peter Zijlstra]

On Fri, Sep 02, 2016 at 04:16:54PM -0400, Alan Stern wrote:
> Felipe, your tests will show whether my guess was totally off-base.  
> For the new people, Felipe is tracking down a problem that involves
> exactly the code sequence listed at the top of the email, where we know
> that the wakeup routine runs but nevertheless the task sleeps.  At
> least, that's what it looks like at the moment.

What arch are you seeing this on?

Re: Memory barrier needed with wake_up_process()?

[Felipe Balbi]

hi,

Peter Zijlstra <peterz@infradead.org> writes:
> On Fri, Sep 02, 2016 at 04:16:54PM -0400, Alan Stern wrote:
>> Felipe, your tests will show whether my guess was totally off-base.  
>> For the new people, Felipe is tracking down a problem that involves
>> exactly the code sequence listed at the top of the email, where we know
>> that the wakeup routine runs but nevertheless the task sleeps.  At
>> least, that's what it looks like at the moment.
>
> What arch are you seeing this on?

x86. Skylake to be exact.

The following change survived through the night:

diff --git a/drivers/usb/gadget/function/f_mass_storage.c b/drivers/usb/gadget/function/f_mass_storage.c
index 8f3659b65f53..d31581dd5ce5 100644
--- a/drivers/usb/gadget/function/f_mass_storage.c
+++ b/drivers/usb/gadget/function/f_mass_storage.c
@@ -395,7 +395,7 @@ static int fsg_set_halt(struct fsg_dev *fsg, struct usb_ep *ep)
 /* Caller must hold fsg->lock */
 static void wakeup_thread(struct fsg_common *common)
 {
-	smp_wmb();	/* ensure the write of bh->state is complete */
+	smp_mb();	/* ensure the write of bh->state is complete */
 	/* Tell the main thread that something has happened */
 	common->thread_wakeup_needed = 1;
 	if (common->thread_task)
@@ -626,7 +626,7 @@ static int sleep_thread(struct fsg_common *common, bool can_freeze)
 	}
 	__set_current_state(TASK_RUNNING);
 	common->thread_wakeup_needed = 0;
-	smp_rmb();	/* ensure the latest bh->state is visible */
+	smp_mb();	/* ensure the latest bh->state is visible */
 	return rc;
 }
 

I'll keep it running until Monday at least

-- 
balbi

Re: Memory barrier needed with wake_up_process()?

[Peter Zijlstra]

On Sat, Sep 03, 2016 at 09:58:09AM +0300, Felipe Balbi wrote:

> > What arch are you seeing this on?
> 
> x86. Skylake to be exact.

So it _cannot_ be the thing Alan mentioned. By the simple fact that
spin_lock() is a full barrier on x86 (every LOCK prefixed instruction
is).

> The following change survived through the night:
> 
> diff --git a/drivers/usb/gadget/function/f_mass_storage.c b/drivers/usb/gadget/function/f_mass_storage.c
> index 8f3659b65f53..d31581dd5ce5 100644
> --- a/drivers/usb/gadget/function/f_mass_storage.c
> +++ b/drivers/usb/gadget/function/f_mass_storage.c
> @@ -395,7 +395,7 @@ static int fsg_set_halt(struct fsg_dev *fsg, struct usb_ep *ep)
>  /* Caller must hold fsg->lock */
>  static void wakeup_thread(struct fsg_common *common)
>  {
> -	smp_wmb();	/* ensure the write of bh->state is complete */
> +	smp_mb();	/* ensure the write of bh->state is complete */
>  	/* Tell the main thread that something has happened */
>  	common->thread_wakeup_needed = 1;
>  	if (common->thread_task)
> @@ -626,7 +626,7 @@ static int sleep_thread(struct fsg_common *common, bool can_freeze)
>  	}
>  	__set_current_state(TASK_RUNNING);
>  	common->thread_wakeup_needed = 0;
> -	smp_rmb();	/* ensure the latest bh->state is visible */
> +	smp_mb();	/* ensure the latest bh->state is visible */
>  	return rc;
>  }

Sorry, but that is horrible code. A barrier cannot ensure writes are
'complete', at best they can ensure order between writes (or reads
etc..).

Also, looking at that thing, that common->thread_wakeup_needed variable
is 100% redundant. All sleep_thread() invocations are inside a loop of
sorts and basically wait for other conditions to become true.

For example:

	while (bh->state != BUF_STATE_EMPTY) {
		rc = sleep_thread(common, false);
		if (rc)
			return rc;
	}

All you care about there is bh->state, _not_
common->thread_wakeup_needed.

That said, I cannot spot an obvious fail, but the code can certainly use
help.

Re: Memory barrier needed with wake_up_process()?

[Felipe Balbi]

Hi,

Peter Zijlstra <peterz@infradead.org> writes:
> On Sat, Sep 03, 2016 at 09:58:09AM +0300, Felipe Balbi wrote:
>
>> > What arch are you seeing this on?
>> 
>> x86. Skylake to be exact.
>
> So it _cannot_ be the thing Alan mentioned. By the simple fact that
> spin_lock() is a full barrier on x86 (every LOCK prefixed instruction
> is).

I still have this working even after 15 hours of runtime on a test case
that was failing consistently within few minutes. At a minimum smp_mb()
has some side effect which is hiding the actual problem.

>> The following change survived through the night:
>> 
>> diff --git a/drivers/usb/gadget/function/f_mass_storage.c b/drivers/usb/gadget/function/f_mass_storage.c
>> index 8f3659b65f53..d31581dd5ce5 100644
>> --- a/drivers/usb/gadget/function/f_mass_storage.c
>> +++ b/drivers/usb/gadget/function/f_mass_storage.c
>> @@ -395,7 +395,7 @@ static int fsg_set_halt(struct fsg_dev *fsg, struct usb_ep *ep)
>>  /* Caller must hold fsg->lock */
>>  static void wakeup_thread(struct fsg_common *common)
>>  {
>> -	smp_wmb();	/* ensure the write of bh->state is complete */
>> +	smp_mb();	/* ensure the write of bh->state is complete */
>>  	/* Tell the main thread that something has happened */
>>  	common->thread_wakeup_needed = 1;
>>  	if (common->thread_task)
>> @@ -626,7 +626,7 @@ static int sleep_thread(struct fsg_common *common, bool can_freeze)
>>  	}
>>  	__set_current_state(TASK_RUNNING);
>>  	common->thread_wakeup_needed = 0;
>> -	smp_rmb();	/* ensure the latest bh->state is visible */
>> +	smp_mb();	/* ensure the latest bh->state is visible */
>>  	return rc;
>>  }
>
> Sorry, but that is horrible code. A barrier cannot ensure writes are
> 'complete', at best they can ensure order between writes (or reads
> etc..).

not arguing ;-)

> Also, looking at that thing, that common->thread_wakeup_needed variable
> is 100% redundant. All sleep_thread() invocations are inside a loop of
> sorts and basically wait for other conditions to become true.
>
> For example:
>
> 	while (bh->state != BUF_STATE_EMPTY) {
> 		rc = sleep_thread(common, false);
> 		if (rc)
> 			return rc;
> 	}

right

> All you care about there is bh->state, _not_
> common->thread_wakeup_needed.
>
> That said, I cannot spot an obvious fail,

okay, but a fail does exist. Any hints on what extra information I could
capture to help figuring this one out?

> but the code can certainly use help.

Sure, that can be done for v4.9 (if I have time) or v4.10 merge
window. Meanwhile, we're trying to find a minimal fix for the -rc which
can also be backported to stable, right?

-- 
balbi

Re: Memory barrier needed with wake_up_process()?

[Peter Zijlstra]

On Sat, Sep 03, 2016 at 04:51:07PM +0300, Felipe Balbi wrote:
> > That said, I cannot spot an obvious fail,
> 
> okay, but a fail does exist. Any hints on what extra information I could
> capture to help figuring this one out?

More information on which sleep is not waking woudl help I suppose. That
greatly limits the amount of code one has to stare at.

Also, a better picture of all threads involved in the wakeup. Alan seems
to suggest there's multiple CPUs involved in the wakeup.

Re: Memory barrier needed with wake_up_process()?

[Alan Stern]

On Sat, 3 Sep 2016, Peter Zijlstra wrote:

> On Sat, Sep 03, 2016 at 09:58:09AM +0300, Felipe Balbi wrote:
> 
> > > What arch are you seeing this on?
> > 
> > x86. Skylake to be exact.
> 
> So it _cannot_ be the thing Alan mentioned. By the simple fact that
> spin_lock() is a full barrier on x86 (every LOCK prefixed instruction
> is).

True, my guess was wrong.

> > The following change survived through the night:
> > 
> > diff --git a/drivers/usb/gadget/function/f_mass_storage.c b/drivers/usb/gadget/function/f_mass_storage.c
> > index 8f3659b65f53..d31581dd5ce5 100644
> > --- a/drivers/usb/gadget/function/f_mass_storage.c
> > +++ b/drivers/usb/gadget/function/f_mass_storage.c
> > @@ -395,7 +395,7 @@ static int fsg_set_halt(struct fsg_dev *fsg, struct usb_ep *ep)
> >  /* Caller must hold fsg->lock */
> >  static void wakeup_thread(struct fsg_common *common)
> >  {
> > -	smp_wmb();	/* ensure the write of bh->state is complete */
> > +	smp_mb();	/* ensure the write of bh->state is complete */
> >  	/* Tell the main thread that something has happened */
> >  	common->thread_wakeup_needed = 1;
> >  	if (common->thread_task)
> > @@ -626,7 +626,7 @@ static int sleep_thread(struct fsg_common *common, bool can_freeze)
> >  	}
> >  	__set_current_state(TASK_RUNNING);
> >  	common->thread_wakeup_needed = 0;
> > -	smp_rmb();	/* ensure the latest bh->state is visible */
> > +	smp_mb();	/* ensure the latest bh->state is visible */
> >  	return rc;
> >  }
> 
> Sorry, but that is horrible code. A barrier cannot ensure writes are
> 'complete', at best they can ensure order between writes (or reads
> etc..).

The code is better than the comment.  What I really meant was that the 
write of bh->state needs to be visible to the thread after it wakes up 
(or after it checks the wakeup condition and skips going to sleep).

> Also, looking at that thing, that common->thread_wakeup_needed variable
> is 100% redundant. All sleep_thread() invocations are inside a loop of
> sorts and basically wait for other conditions to become true.
> 
> For example:
> 
> 	while (bh->state != BUF_STATE_EMPTY) {
> 		rc = sleep_thread(common, false);
> 		if (rc)
> 			return rc;
> 	}
> 
> All you care about there is bh->state, _not_
> common->thread_wakeup_needed.

You know, I never went through and verified that _all_ the invocations 
of sleep_thread() are like that.  In fact, I wrote the sleep/wakeup 
routines _before_ the rest of the code, and I didn't know in advance 
exactly how they were going to be called.

> That said, I cannot spot an obvious fail, but the code can certainly use
> help.

The problem may be that when the thread wakes up (or skips going to 
sleep), it needs to see more than just bh->state.  Those other values 
it needs are not written by the same CPU that calls wakeup_thread(), 
and so to ensure that they are visible that smp_wmb() really ought to 
be smp_mb() (and correspondingly in the thread.  That's what Felipe has 
been testing.

Alan Stern

Re: Memory barrier needed with wake_up_process()?

[Peter Zijlstra]

On Sat, Sep 03, 2016 at 10:16:31AM -0400, Alan Stern wrote:

> > Sorry, but that is horrible code. A barrier cannot ensure writes are
> > 'complete', at best they can ensure order between writes (or reads
> > etc..).
> 
> The code is better than the comment.  What I really meant was that the 
> write of bh->state needs to be visible to the thread after it wakes up 
> (or after it checks the wakeup condition and skips going to sleep).

Yeah, I got that.

> > Also, looking at that thing, that common->thread_wakeup_needed variable
> > is 100% redundant. All sleep_thread() invocations are inside a loop of
> > sorts and basically wait for other conditions to become true.
> > 
> > For example:
> > 
> > 	while (bh->state != BUF_STATE_EMPTY) {
> > 		rc = sleep_thread(common, false);
> > 		if (rc)
> > 			return rc;
> > 	}
> > 
> > All you care about there is bh->state, _not_
> > common->thread_wakeup_needed.
> 
> You know, I never went through and verified that _all_ the invocations 
> of sleep_thread() are like that. 

Well, thing is, they're all inside a loop which checks other conditions
for forward progress. Therefore the loop inside sleep_thread() is
pointless. Even if you were to return early, you'd simply loop in the
outer loop and go back to sleep again.

> In fact, I wrote the sleep/wakeup 
> routines _before_ the rest of the code, and I didn't know in advance 
> exactly how they were going to be called.

Still seems strange to me, why not use wait-queues for the first cut?

Only if you find a performance issue with wait-queues, which cannot be
fixed in the wait-queue proper, then do you do custom thingies.

Starting with a custom sleeper, just doesn't make sense to me.

> > That said, I cannot spot an obvious fail, but the code can certainly use
> > help.
> 
> The problem may be that when the thread wakes up (or skips going to 
> sleep), it needs to see more than just bh->state.  Those other values 
> it needs are not written by the same CPU that calls wakeup_thread(), 
> and so to ensure that they are visible that smp_wmb() really ought to 
> be smp_mb() (and correspondingly in the thread.  That's what Felipe has 
> been testing.

So you're saying something like:


	CPU0		CPU1		CPU2

	X = 1				sleep_thread()
			wakeup_thread()
					r = X

But how does CPU1 know to do the wakeup? That is, how are CPU0 and CPU1
coupled.

Re: Memory barrier needed with wake_up_process()?

[Alan Stern]

On Mon, 5 Sep 2016, Peter Zijlstra wrote:

> > You know, I never went through and verified that _all_ the invocations 
> > of sleep_thread() are like that. 
> 
> Well, thing is, they're all inside a loop which checks other conditions
> for forward progress. Therefore the loop inside sleep_thread() is
> pointless. Even if you were to return early, you'd simply loop in the
> outer loop and go back to sleep again.
> 
> > In fact, I wrote the sleep/wakeup 
> > routines _before_ the rest of the code, and I didn't know in advance 
> > exactly how they were going to be called.
> 
> Still seems strange to me, why not use wait-queues for the first cut?
> 
> Only if you find a performance issue with wait-queues, which cannot be
> fixed in the wait-queue proper, then do you do custom thingies.
> 
> Starting with a custom sleeper, just doesn't make sense to me.

I really don't remember.  Felipe says that the ancient history shows
the initial implementation did use a wait-queue, and then it was
changed.  Perhaps I was imitating the structure of
scsi_error_handler().

> > The problem may be that when the thread wakes up (or skips going to 
> > sleep), it needs to see more than just bh->state.  Those other values 
> > it needs are not written by the same CPU that calls wakeup_thread(), 
> > and so to ensure that they are visible that smp_wmb() really ought to 
> > be smp_mb() (and correspondingly in the thread.  That's what Felipe has 
> > been testing.
> 
> So you're saying something like:
> 
> 
> 	CPU0		CPU1		CPU2
> 
> 	X = 1				sleep_thread()
> 			wakeup_thread()
> 					r = X
> 
> But how does CPU1 know to do the wakeup? That is, how are CPU0 and CPU1
> coupled.

As mentioned later on, "CPU0" is actually a DMA master, not another 
CPU.

Alan Stern