Re: [PATCH] usb: dwc3: gadget: Skip resizing EP's TX FIFO if already resized

[Thinh Nguyen <Thinh.Nguyen@synopsys.com>]

Thinh Nguyen wrote:
> Jack Pham wrote:
>> On Fri, Oct 08, 2021 at 12:07:20AM +0000, Thinh Nguyen wrote:
>>> Wesley Cheng wrote:
>>>>
>>>>
>>>> On 9/10/2021 8:08 PM, Thinh Nguyen wrote:
>>>>> Wesley Cheng wrote:
>>>>>>
>>>>>>
>>>>>> On 9/9/2021 6:15 PM, Thinh Nguyen wrote:
>>>>>>> Jack Pham wrote:
>>>>>>>> Some functions may dynamically enable and disable their endpoints
>>>>>>>> regularly throughout their operation, particularly when Set Interface
>>>>>>>> is employed to switch between Alternate Settings.  For instance the
>>>>>>>> UAC2 function has its respective endpoints for playback & capture
>>>>>>>> associated with AltSetting 1, in which case those endpoints would not
>>>>>>>> get enabled until the host activates the AltSetting.  And they
>>>>>>>> conversely become disabled when the interfaces' AltSetting 0 is
>>>>>>>> chosen.
>>>>>>>>
>>>>>>>> With the DWC3 FIFO resizing algorithm recently added, every
>>>>>>>> usb_ep_enable() call results in a call to resize that EP's TXFIFO,
>>>>>>>> but if the same endpoint is enabled again and again, this incorrectly
>>>>>>>> leads to FIFO RAM allocation exhaustion as the mechanism did not
>>>>>>>> account for the possibility that endpoints can be re-enabled many
>>>>>>>> times.
>>>>>>>>
>>>>>>>> Example log splat:
>>>>>>>>
>>>>>>>> 	dwc3 a600000.dwc3: Fifosize(3717) > RAM size(3462) ep3in depth:217973127
>>>>>>>> 	configfs-gadget gadget: u_audio_start_capture:521 Error!
>>>>>>>> 	dwc3 a600000.dwc3: request 000000000be13e18 was not queued to ep3in
>>>>>>>>
>>>>>>>> This is easily fixed by bailing out of dwc3_gadget_resize_tx_fifos()
>>>>>>>> if an endpoint is already resized, avoiding the calculation error
>>>>>>>> resulting from accumulating the EP's FIFO depth repeatedly.
>>>>>>>>
>>>>>>>> Fixes: 9f607a309fbe9 ("usb: dwc3: Resize TX FIFOs to meet EP bursting requirements")
>>>>>>>> Signed-off-by: Jack Pham <jackp@codeaurora.org>
>>>>>>>> ---
>>>>>>>>  drivers/usb/dwc3/gadget.c | 4 ++++
>>>>>>>>  1 file changed, 4 insertions(+)
>>>>>>>>
>>>>>>>> diff --git a/drivers/usb/dwc3/gadget.c b/drivers/usb/dwc3/gadget.c
>>>>>>>> index 804b50548163..c647c76d7361 100644
>>>>>>>> --- a/drivers/usb/dwc3/gadget.c
>>>>>>>> +++ b/drivers/usb/dwc3/gadget.c
>>>>>>>> @@ -747,6 +747,10 @@ static int dwc3_gadget_resize_tx_fifos(struct dwc3_ep *dep)
>>>>>>>>  	if (!usb_endpoint_dir_in(dep->endpoint.desc) || dep->number <= 1)
>>>>>>>>  		return 0;
>>>>>>>>  
>>>>>>>> +	/* bail if already resized */
>>>>>>>> +	if (dwc3_readl(dwc->regs, DWC3_GTXFIFOSIZ(dep->number >> 1)))
>>>>>>>> +		return 0;
>>>>>>>> +
>>>>>>>>  	ram1_depth = DWC3_RAM1_DEPTH(dwc->hwparams.hwparams7);
>>>>>>>>  
>>>>>>>>  	if ((dep->endpoint.maxburst > 1 &&
>>>>>>>>
>>>>>>
>>>>>> Hi Thinh,
>>>>>>
>>>>>>>
>>>>>>> This seems like a workaround more than a fix. As previously pointed out,
>>>>>>> there will be problems when there are multiple alternate setting
>>>>>>> interface [2]. If we're doing this way, are we properly allocating the
>>>>>>> fifo size for the alternate setting that requires the most fifo size and
>>>>>>> not just the first alt-setting 0? Also different alt-setting can have
>>>>>>
>>>>>> Each alt interface will call usb_ep_autoconfig() which should be
>>>>>> assigned different endpoint numbers.  This would mean that if alt intf#0
>>>>>> gets selected, and EP is enabled, then we will resize the TXFIFO and map
>>>>>> that FIFO to the physical EP.  Then when/if the host requests the other
>>>>>> alt intf#1, and that calls EP enable, then the logic will then attempt
>>>>>> to resize based on the parameters, and again map that FIFO to the
>>>>>> physical EP. (since we call autoconfig on all interfaces, they should be
>>>>>> assigned different endpoints)
>>>>
>>>> Hi Thinh,
>>>>
>>>>>
>>>>> That's not true. Different alt-settings of an interface can share
>>>>> endpoint numbers. This is often the case for UASP driver where
>>>>> alt-setting 0 is for BOT protocol and alt-setting 1 is UASP. When we
>>>>> switch alt-setting, we disable the current endpoints and enable the
>>>>> old/new ones.
>>>>>
>>>>
>>>> Thanks for pointing that use case out.  Maybe we can consider seeing if
>>>> we can walk through all alternate interfaces for a particular function,
>>>> and resize for the largest setting?  That might be a possible
>>>> improvement made to the check_config() function.  Let me start makign
>>>> the changes for this and verifying it.
>>>>
>>>
>>> Thanks!
>>>
>>> Currently the gadget configures early and informs the gadget driver of
>>> how many endpoints are available, which doesn't leave much room for the
>>> gadget to do optimization/reconfiguration.
>>>
>>> If there's an option for the composite layer to inform the controller
>>> driver of the entire configuration, then we can take advantage of more
>>> dwc3 controller capability/flexibility (not just resizing txfifo).
>>>
>>>>>>
>>>>>> I agree that there is currently a limitation because we are going to
>>>>>> reserve at minimum 1 FIFO for BOTH alt interfaces, even though there is
>>>>>> only 1 interface active at a time.  The missing logic that we might be
>>>>>> missing is seeing how we can re-purpose the FIFO that is being disabled.
>>>>>>  However, I think Jack's fix here would be applicable to the improvement
>>>>>> in logic to re-use/re-assign FIFO space allocated by disabled EPs also.
>>>>>>
>>>>>
>>>>> Improvement is always great. I just hope we don't just stop where we are
>>>>> now. Since you're working on this feature at the moment, it would be
>>>>> good to also resolve some of the outstanding issues as Jack's fix seems
>>>>> to be incomplete.
>>>>>
>>>>
>>>> If we implement the improvement mentioned above, I think Jack's fix will
>>>> be applicable there as well.  If we resize for the largest alternate
>>>> interface, then there would be no reason for us to resize again.
>>>>
>>> As long as you have the above as part of your roadmap, I don't mind
>>> Jack's fix for now.
>>
>> Thanks for your input as always Thinh.  Does the patch still look ok in
>> its current state?  Last time I had responded to Felipe that I would try
> 
> TX endpoints should have non-zero GTXFIFOSIZ. Using the register as a
> flag to check whether it's been resized is not ok. Also, what happened
> after resizing the txfifo? Do you restore its previous default value?
> 
>> to entertain adding an explicit flag to keep track of whether an EP had
>> been resized yet or not.  When trying to implement this as another
>> DWC3_EP_* bit for dep->flags we'd then need to retain this flag across
>> ep_disable/enable calls, so it looks a tiny bit cumbersome because
>> dep->flags isn't neatly 0 anymore :-P.
>>
>> So ep_disable() would need to look something like this:
>>
>>  static int __dwc3_gadget_ep_disable(struct dwc3_ep *dep)
>>  {
>>  ...
>> 	dep->stream_capable = false;
>> 	dep->type = 0;
>> -	dep->flags = 0;
>> +	dep->flags &= DWC3_EP_TXFIFO_RESIZED;
> 
> I think you mean this?
> dep->flags &= ~DWC3_EP_TXFIFO_RESIZED;
> 

nvm, ignore this comment. Your intention is
dep->flags &= DWC3_EP_TXFIFO_RESIZED;

Thinh