[PATCH] usb: dwc3: fix endpoint direction when inputs are more than outputs

[PATCH] usb: dwc3: fix endpoint direction when inputs are more than outputs

[Shantur Rathore]

In RK3399 as per documentation (
https://usermanual.wiki/Document/RockchipDeveloperGuidelinux44USB.31610806
), there are 7 Input Endpoints and 6 Output endpoints, in total 13
endpoints.

Currently dwc3/gadget.c driver uses the number of endpoints
available and starts setting them up with even endpoints as output
endpoints and odd numbered as even endpoints. This leads to 7 Output
endpoints and 6 input endpoints for RK3399.

If you try to create a composite gadget which uses all the input
endpoints, one can see the issue. You just need to create functions to
use up the last input ep and it would fail to create. No need to
connect it to the host.
This was confirmed when running a rockchip-linux bsp image.

[root@rockpro rock]# ls /sys/kernel/debug/usb/fe800000.usb/
ep0in  ep0out  ep1in  ep1out  ep2in  ep2out  ep3in  ep3out  ep4in
ep4out  ep5in  ep5out  ep6in  link_state  lsp_dump  mode  regdump
testmode

Currently in linux mainline it is

[root@rockpro rock]# ls /sys/kernel/debug/usb/fe800000.usb/
ep0in  ep0out  ep1in  ep1out  ep2in  ep2out  ep3in  ep3out  ep4in
ep4out  ep5in  ep5out  ep6out  link_state  lsp_dump  mode  regdump
testmode

ep6 being out instead of in as per the hardware spec.

Upon investigation of rockchip bsp kernel,
https://github.com/rockchip-linux/kernel/

The issue was clear, currently, dwc3/gadget driver doesn't take
DWC3_NUM_IN_EPS into consideration while enumerating them.

The patch below fixes the issue and ep6 is correctly enumerated as input.
---
 drivers/usb/dwc3/core.c   |  1 +
 drivers/usb/dwc3/core.h   |  1 +
 drivers/usb/dwc3/gadget.c | 40 ++++++++++++++++++++++++---------------
 3 files changed, 27 insertions(+), 15 deletions(-)

diff --git a/drivers/usb/dwc3/core.c b/drivers/usb/dwc3/core.c
index 01866dcb953b..279c9a97cb8c 100644
--- a/drivers/usb/dwc3/core.c
+++ b/drivers/usb/dwc3/core.c
@@ -555,6 +555,7 @@ static void dwc3_core_num_eps(struct dwc3 *dwc)
 	struct dwc3_hwparams	*parms = &dwc->hwparams;
 
 	dwc->num_eps = DWC3_NUM_EPS(parms);
+	dwc->num_in_eps = DWC3_NUM_IN_EPS(parms);
 }
 
 static void dwc3_cache_hwparams(struct dwc3 *dwc)
diff --git a/drivers/usb/dwc3/core.h b/drivers/usb/dwc3/core.h
index 5612bfdf37da..89a0998c618c 100644
--- a/drivers/usb/dwc3/core.h
+++ b/drivers/usb/dwc3/core.h
@@ -1227,6 +1227,7 @@ struct dwc3 {
 	u8			speed;
 
 	u8			num_eps;
+	u8			num_in_eps;
 
 	struct dwc3_hwparams	hwparams;
 	struct debugfs_regset32	*regset;
diff --git a/drivers/usb/dwc3/gadget.c b/drivers/usb/dwc3/gadget.c
index 804b50548163..d9d19dc0a29f 100644
--- a/drivers/usb/dwc3/gadget.c
+++ b/drivers/usb/dwc3/gadget.c
@@ -693,9 +693,11 @@ void dwc3_gadget_clear_tx_fifos(struct dwc3 *dwc)
 
 	dwc->last_fifo_depth = fifo_depth;
 	/* Clear existing TXFIFO for all IN eps except ep0 */
-	for (num = 3; num < min_t(int, dwc->num_eps, DWC3_ENDPOINTS_NUM);
-	     num += 2) {
+	for (num = 3; num < DWC3_ENDPOINTS_NUM; num += 2) {
 		dep = dwc->eps[num];
+
+		if(!dep)
+			continue;
 		/* Don't change TXFRAMNUM on usb31 version */
 		size = DWC3_IP_IS(DWC3) ? 0 :
 			dwc3_readl(dwc->regs, DWC3_GTXFIFOSIZ(num >> 1)) &
@@ -2257,7 +2259,7 @@ static void dwc3_stop_active_transfers(struct dwc3 *dwc)
 {
 	u32 epnum;
 
-	for (epnum = 2; epnum < dwc->num_eps; epnum++) {
+	for (epnum = 2; epnum < DWC3_ENDPOINTS_NUM; epnum++) {
 		struct dwc3_ep *dep;
 
 		dep = dwc->eps[epnum];
@@ -2960,10 +2962,9 @@ static int dwc3_gadget_init_out_endpoint(struct dwc3_ep *dep)
 	return dwc3_alloc_trb_pool(dep);
 }
 
-static int dwc3_gadget_init_endpoint(struct dwc3 *dwc, u8 epnum)
+static int dwc3_gadget_init_endpoint(struct dwc3 *dwc, u8 epnum, bool direction)
 {
 	struct dwc3_ep			*dep;
-	bool				direction = epnum & 1;
 	int				ret;
 	u8				num = epnum >> 1;
 
@@ -3011,21 +3012,30 @@ static int dwc3_gadget_init_endpoint(struct dwc3 *dwc, u8 epnum)
 	return 0;
 }
 
-static int dwc3_gadget_init_endpoints(struct dwc3 *dwc, u8 total)
+static int dwc3_gadget_init_endpoints(struct dwc3 *dwc, u8 total, u8 num_in_eps)
 {
-	u8				epnum;
+	u8				num;
+	int				ret;
 
 	INIT_LIST_HEAD(&dwc->gadget->ep_list);
 
-	for (epnum = 0; epnum < total; epnum++) {
-		int			ret;
+	/* init input endpoints as reported by hw */
+        for (num = 0; num < num_in_eps; num++) {
 
-		ret = dwc3_gadget_init_endpoint(dwc, epnum);
-		if (ret)
-			return ret;
-	}
+                ret = dwc3_gadget_init_endpoint(dwc, (num << 1) + 1, 1);
+                if (ret)
+                        return ret;
+        }
 
-	return 0;
+        /* init rest endpoints as output endpoints */
+        for (num = 0; num < total - num_in_eps; num++) {
+
+                ret = dwc3_gadget_init_endpoint(dwc, num << 1, 0);
+                if (ret)
+                        return ret;
+        }
+
+	return ret;
 }
 
 static void dwc3_gadget_free_endpoints(struct dwc3 *dwc)
@@ -4282,7 +4292,7 @@ int dwc3_gadget_init(struct dwc3 *dwc)
 	 * sure we're starting from a well known location.
 	 */
 
-	ret = dwc3_gadget_init_endpoints(dwc, dwc->num_eps);
+	ret = dwc3_gadget_init_endpoints(dwc, dwc->num_eps, dwc->num_in_eps);
 	if (ret)
 		goto err4;
 
-- 
2.33.0

Re: [PATCH] usb: dwc3: fix endpoint direction when inputs are more than outputs

[Thinh Nguyen]

Shantur Rathore wrote:
> In RK3399 as per documentation (
> https://urldefense.com/v3/__https://usermanual.wiki/Document/RockchipDeveloperGuidelinux44USB.31610806__;!!A4F2R9G_pg!JqYr6U87SL3rYylirF6W2vwNzC0Ft8YiZwTlMTwWl7bpaHGZuh-JMfOvUaG781GKZIBd$ 
> ), there are 7 Input Endpoints and 6 Output endpoints, in total 13
> endpoints.
> 
> Currently dwc3/gadget.c driver uses the number of endpoints
> available and starts setting them up with even endpoints as output
> endpoints and odd numbered as even endpoints. This leads to 7 Output
> endpoints and 6 input endpoints for RK3399.
> 
> If you try to create a composite gadget which uses all the input
> endpoints, one can see the issue. You just need to create functions to
> use up the last input ep and it would fail to create. No need to
> connect it to the host.
> This was confirmed when running a rockchip-linux bsp image.
> 
> [root@rockpro rock]# ls /sys/kernel/debug/usb/fe800000.usb/
> ep0in  ep0out  ep1in  ep1out  ep2in  ep2out  ep3in  ep3out  ep4in
> ep4out  ep5in  ep5out  ep6in  link_state  lsp_dump  mode  regdump
> testmode
> 
> Currently in linux mainline it is
> 
> [root@rockpro rock]# ls /sys/kernel/debug/usb/fe800000.usb/
> ep0in  ep0out  ep1in  ep1out  ep2in  ep2out  ep3in  ep3out  ep4in
> ep4out  ep5in  ep5out  ep6out  link_state  lsp_dump  mode  regdump
> testmode
> 
> ep6 being out instead of in as per the hardware spec.
> 
> Upon investigation of rockchip bsp kernel,
> https://urldefense.com/v3/__https://github.com/rockchip-linux/kernel/__;!!A4F2R9G_pg!JqYr6U87SL3rYylirF6W2vwNzC0Ft8YiZwTlMTwWl7bpaHGZuh-JMfOvUaG788DHJSUE$ 
> 
> The issue was clear, currently, dwc3/gadget driver doesn't take
> DWC3_NUM_IN_EPS into consideration while enumerating them.
> 
> The patch below fixes the issue and ep6 is correctly enumerated as input.

No Signed-of-by?

> ---
>  drivers/usb/dwc3/core.c   |  1 +
>  drivers/usb/dwc3/core.h   |  1 +
>  drivers/usb/dwc3/gadget.c | 40 ++++++++++++++++++++++++---------------
>  3 files changed, 27 insertions(+), 15 deletions(-)
> 
> diff --git a/drivers/usb/dwc3/core.c b/drivers/usb/dwc3/core.c
> index 01866dcb953b..279c9a97cb8c 100644
> --- a/drivers/usb/dwc3/core.c
> +++ b/drivers/usb/dwc3/core.c
> @@ -555,6 +555,7 @@ static void dwc3_core_num_eps(struct dwc3 *dwc)
>  	struct dwc3_hwparams	*parms = &dwc->hwparams;
>  
>  	dwc->num_eps = DWC3_NUM_EPS(parms);
> +	dwc->num_in_eps = DWC3_NUM_IN_EPS(parms);
>  }
>  
>  static void dwc3_cache_hwparams(struct dwc3 *dwc)
> diff --git a/drivers/usb/dwc3/core.h b/drivers/usb/dwc3/core.h
> index 5612bfdf37da..89a0998c618c 100644
> --- a/drivers/usb/dwc3/core.h
> +++ b/drivers/usb/dwc3/core.h
> @@ -1227,6 +1227,7 @@ struct dwc3 {
>  	u8			speed;
>  
>  	u8			num_eps;
> +	u8			num_in_eps;
>  
>  	struct dwc3_hwparams	hwparams;
>  	struct debugfs_regset32	*regset;
> diff --git a/drivers/usb/dwc3/gadget.c b/drivers/usb/dwc3/gadget.c
> index 804b50548163..d9d19dc0a29f 100644
> --- a/drivers/usb/dwc3/gadget.c
> +++ b/drivers/usb/dwc3/gadget.c
> @@ -693,9 +693,11 @@ void dwc3_gadget_clear_tx_fifos(struct dwc3 *dwc)
>  
>  	dwc->last_fifo_depth = fifo_depth;
>  	/* Clear existing TXFIFO for all IN eps except ep0 */
> -	for (num = 3; num < min_t(int, dwc->num_eps, DWC3_ENDPOINTS_NUM);
> -	     num += 2) {
> +	for (num = 3; num < DWC3_ENDPOINTS_NUM; num += 2) {
>  		dep = dwc->eps[num];
> +
> +		if(!dep)
> +			continue;
>  		/* Don't change TXFRAMNUM on usb31 version */
>  		size = DWC3_IP_IS(DWC3) ? 0 :
>  			dwc3_readl(dwc->regs, DWC3_GTXFIFOSIZ(num >> 1)) &
> @@ -2257,7 +2259,7 @@ static void dwc3_stop_active_transfers(struct dwc3 *dwc)
>  {
>  	u32 epnum;
>  
> -	for (epnum = 2; epnum < dwc->num_eps; epnum++) {
> +	for (epnum = 2; epnum < DWC3_ENDPOINTS_NUM; epnum++) {
>  		struct dwc3_ep *dep;
>  
>  		dep = dwc->eps[epnum];
> @@ -2960,10 +2962,9 @@ static int dwc3_gadget_init_out_endpoint(struct dwc3_ep *dep)
>  	return dwc3_alloc_trb_pool(dep);
>  }
>  
> -static int dwc3_gadget_init_endpoint(struct dwc3 *dwc, u8 epnum)
> +static int dwc3_gadget_init_endpoint(struct dwc3 *dwc, u8 epnum, bool direction)
>  {
>  	struct dwc3_ep			*dep;
> -	bool				direction = epnum & 1;
>  	int				ret;
>  	u8				num = epnum >> 1;
>  
> @@ -3011,21 +3012,30 @@ static int dwc3_gadget_init_endpoint(struct dwc3 *dwc, u8 epnum)
>  	return 0;
>  }
>  
> -static int dwc3_gadget_init_endpoints(struct dwc3 *dwc, u8 total)
> +static int dwc3_gadget_init_endpoints(struct dwc3 *dwc, u8 total, u8 num_in_eps)
>  {
> -	u8				epnum;
> +	u8				num;
> +	int				ret;
>  
>  	INIT_LIST_HEAD(&dwc->gadget->ep_list);
>  
> -	for (epnum = 0; epnum < total; epnum++) {
> -		int			ret;
> +	/* init input endpoints as reported by hw */
> +        for (num = 0; num < num_in_eps; num++) {
>  
> -		ret = dwc3_gadget_init_endpoint(dwc, epnum);
> -		if (ret)
> -			return ret;
> -	}
> +                ret = dwc3_gadget_init_endpoint(dwc, (num << 1) + 1, 1);
> +                if (ret)
> +                        return ret;
> +        }
>  
> -	return 0;
> +        /* init rest endpoints as output endpoints */
> +        for (num = 0; num < total - num_in_eps; num++) {
> +
> +                ret = dwc3_gadget_init_endpoint(dwc, num << 1, 0);
> +                if (ret)
> +                        return ret;
> +        }
> +
> +	return ret;
>  }
>  

* DWC3_NUM_EPS(parms) is the total number of endpoints configured in HW
* DWC3_NUM_IN_EPS(parms) is the max number of IN endpoints that SW may
configure

The number of OUT endpoints does not mean DWC3_NUM_EPS(parms) -
DWC3_NUM_IN_EPS(parms).

As long as physical endpoint 0 and 1 are dedicated for control endpoint,
other endpoints can be assigned as IN or OUT direction. So, you can have
as many as DWC3_NUM_EPS(parms) - 1 number of OUT endpoints.

Currently, dwc3 driver assumes that DWC3_NUM_IN_EPS(params) is at least
half of DWC3_NUM_EPS(parms). If that's not the case, we may see
problems. To cover most application setup, the driver tries to setup
number of OUT = IN.

For your case, is there an application that needs all
DWC3_NUM_IN_EPS(params)? If you're going to make a change, please keep
in mind of the info above to prevent regression.

Thanks,
Thinh

Re: [PATCH] usb: dwc3: fix endpoint direction when inputs are more than outputs

[Thinh Nguyen]

Shantur Rathore wrote:
> Hi Thinh,

Hi Shantur, please don't do top-post.

> 
> Thanks for replying.
> 
> As per the documentation for RK3399, it says
> 
> USB3.0 Device Features
> Up to 7 IN endpoints, including control endpoint 0
> Up to 6 OUT endpoints, including control endpoint 0
> 

The statements above are not accurate. You can have more OUT endpoints,
but if you assign all 7 endpoints as IN, then you have up to 6 endpoints
left for OUT. The dwc3 controller is flexible and you can have various
configurations.

> In this case, I think the hardware defines the endpoints being in / out.
> I am in no case a dwc3 expert or have DWC3 documentation so cannot say
> for certain.
> It is all based on RK3399 documentation.
> 
> I am trying to use a composite gadget with
> 
> - HID Keyboard - 1 IN
> - HID Mouse - 1 IN
> - NCM - 2 IN and 1 OUT
> - 2 Mass Storage Devices - 2 IN and 2 OUT

Why do you need 2 Mass Storage devices? Can you use 2 LUNs instead?

> 
> That including ep0 totals to 7 IN endpoints and 3 Out endpoints.
> 
>> Currently, dwc3 driver assumes that DWC3_NUM_IN_EPS(params) is at least
>> half of DWC3_NUM_EPS(parms). If that's not the case, we may see
>> problems. To cover most application setup, the driver tries to setup
>> number of OUT = IN.
> 
> I think I am seeing this issue when I try to use the last IN endpoint.
> 
> Can you please let me know the direction I should take?
> 

At the moment, I don't have a good/clean solution for this. dwc3 setups
half IN and half OUT from the total number of endpoints ahead of time.
Without the full context of the configuration from the composite layer,
it can't adjust easily.

Maybe others have some ideas.

BR,
Thinh

> 
> On Fri, Sep 10, 2021 at 12:43 AM Thinh Nguyen <Thinh.Nguyen@synopsys.com>
> wrote:
> 
>> Shantur Rathore wrote:
>>> In RK3399 as per documentation (
>>>
>> https://urldefense.com/v3/__https://usermanual.wiki/Document/RockchipDeveloperGuidelinux44USB.31610806__;!!A4F2R9G_pg!JqYr6U87SL3rYylirF6W2vwNzC0Ft8YiZwTlMTwWl7bpaHGZuh-JMfOvUaG781GKZIBd$
>>> ), there are 7 Input Endpoints and 6 Output endpoints, in total 13
>>> endpoints.
>>>
>>> Currently dwc3/gadget.c driver uses the number of endpoints
>>> available and starts setting them up with even endpoints as output
>>> endpoints and odd numbered as even endpoints. This leads to 7 Output
>>> endpoints and 6 input endpoints for RK3399.
>>>
>>> If you try to create a composite gadget which uses all the input
>>> endpoints, one can see the issue. You just need to create functions to
>>> use up the last input ep and it would fail to create. No need to
>>> connect it to the host.
>>> This was confirmed when running a rockchip-linux bsp image.
>>>
>>> [root@rockpro rock]# ls /sys/kernel/debug/usb/fe800000.usb/
>>> ep0in  ep0out  ep1in  ep1out  ep2in  ep2out  ep3in  ep3out  ep4in
>>> ep4out  ep5in  ep5out  ep6in  link_state  lsp_dump  mode  regdump
>>> testmode
>>>
>>> Currently in linux mainline it is
>>>
>>> [root@rockpro rock]# ls /sys/kernel/debug/usb/fe800000.usb/
>>> ep0in  ep0out  ep1in  ep1out  ep2in  ep2out  ep3in  ep3out  ep4in
>>> ep4out  ep5in  ep5out  ep6out  link_state  lsp_dump  mode  regdump
>>> testmode
>>>
>>> ep6 being out instead of in as per the hardware spec.
>>>
>>> Upon investigation of rockchip bsp kernel,
>>>
>> https://urldefense.com/v3/__https://github.com/rockchip-linux/kernel/__;!!A4F2R9G_pg!JqYr6U87SL3rYylirF6W2vwNzC0Ft8YiZwTlMTwWl7bpaHGZuh-JMfOvUaG788DHJSUE$
>>>
>>> The issue was clear, currently, dwc3/gadget driver doesn't take
>>> DWC3_NUM_IN_EPS into consideration while enumerating them.
>>>
>>> The patch below fixes the issue and ep6 is correctly enumerated as input.
>>
>> No Signed-of-by?
>>
>>> ---
>>>  drivers/usb/dwc3/core.c   |  1 +
>>>  drivers/usb/dwc3/core.h   |  1 +
>>>  drivers/usb/dwc3/gadget.c | 40 ++++++++++++++++++++++++---------------
>>>  3 files changed, 27 insertions(+), 15 deletions(-)
>>>
>>> diff --git a/drivers/usb/dwc3/core.c b/drivers/usb/dwc3/core.c
>>> index 01866dcb953b..279c9a97cb8c 100644
>>> --- a/drivers/usb/dwc3/core.c
>>> +++ b/drivers/usb/dwc3/core.c
>>> @@ -555,6 +555,7 @@ static void dwc3_core_num_eps(struct dwc3 *dwc)
>>>       struct dwc3_hwparams    *parms = &dwc->hwparams;
>>>
>>>       dwc->num_eps = DWC3_NUM_EPS(parms);
>>> +     dwc->num_in_eps = DWC3_NUM_IN_EPS(parms);
>>>  }
>>>
>>>  static void dwc3_cache_hwparams(struct dwc3 *dwc)
>>> diff --git a/drivers/usb/dwc3/core.h b/drivers/usb/dwc3/core.h
>>> index 5612bfdf37da..89a0998c618c 100644
>>> --- a/drivers/usb/dwc3/core.h
>>> +++ b/drivers/usb/dwc3/core.h
>>> @@ -1227,6 +1227,7 @@ struct dwc3 {
>>>       u8                      speed;
>>>
>>>       u8                      num_eps;
>>> +     u8                      num_in_eps;
>>>
>>>       struct dwc3_hwparams    hwparams;
>>>       struct debugfs_regset32 *regset;
>>> diff --git a/drivers/usb/dwc3/gadget.c b/drivers/usb/dwc3/gadget.c
>>> index 804b50548163..d9d19dc0a29f 100644
>>> --- a/drivers/usb/dwc3/gadget.c
>>> +++ b/drivers/usb/dwc3/gadget.c
>>> @@ -693,9 +693,11 @@ void dwc3_gadget_clear_tx_fifos(struct dwc3 *dwc)
>>>
>>>       dwc->last_fifo_depth = fifo_depth;
>>>       /* Clear existing TXFIFO for all IN eps except ep0 */
>>> -     for (num = 3; num < min_t(int, dwc->num_eps, DWC3_ENDPOINTS_NUM);
>>> -          num += 2) {
>>> +     for (num = 3; num < DWC3_ENDPOINTS_NUM; num += 2) {
>>>               dep = dwc->eps[num];
>>> +
>>> +             if(!dep)
>>> +                     continue;
>>>               /* Don't change TXFRAMNUM on usb31 version */
>>>               size = DWC3_IP_IS(DWC3) ? 0 :
>>>                       dwc3_readl(dwc->regs, DWC3_GTXFIFOSIZ(num >> 1)) &
>>> @@ -2257,7 +2259,7 @@ static void dwc3_stop_active_transfers(struct dwc3
>> *dwc)
>>>  {
>>>       u32 epnum;
>>>
>>> -     for (epnum = 2; epnum < dwc->num_eps; epnum++) {
>>> +     for (epnum = 2; epnum < DWC3_ENDPOINTS_NUM; epnum++) {
>>>               struct dwc3_ep *dep;
>>>
>>>               dep = dwc->eps[epnum];
>>> @@ -2960,10 +2962,9 @@ static int dwc3_gadget_init_out_endpoint(struct
>> dwc3_ep *dep)
>>>       return dwc3_alloc_trb_pool(dep);
>>>  }
>>>
>>> -static int dwc3_gadget_init_endpoint(struct dwc3 *dwc, u8 epnum)
>>> +static int dwc3_gadget_init_endpoint(struct dwc3 *dwc, u8 epnum, bool
>> direction)
>>>  {
>>>       struct dwc3_ep                  *dep;
>>> -     bool                            direction = epnum & 1;
>>>       int                             ret;
>>>       u8                              num = epnum >> 1;
>>>
>>> @@ -3011,21 +3012,30 @@ static int dwc3_gadget_init_endpoint(struct dwc3
>> *dwc, u8 epnum)
>>>       return 0;
>>>  }
>>>
>>> -static int dwc3_gadget_init_endpoints(struct dwc3 *dwc, u8 total)
>>> +static int dwc3_gadget_init_endpoints(struct dwc3 *dwc, u8 total, u8
>> num_in_eps)
>>>  {
>>> -     u8                              epnum;
>>> +     u8                              num;
>>> +     int                             ret;
>>>
>>>       INIT_LIST_HEAD(&dwc->gadget->ep_list);
>>>
>>> -     for (epnum = 0; epnum < total; epnum++) {
>>> -             int                     ret;
>>> +     /* init input endpoints as reported by hw */
>>> +        for (num = 0; num < num_in_eps; num++) {
>>>
>>> -             ret = dwc3_gadget_init_endpoint(dwc, epnum);
>>> -             if (ret)
>>> -                     return ret;
>>> -     }
>>> +                ret = dwc3_gadget_init_endpoint(dwc, (num << 1) + 1, 1);
>>> +                if (ret)
>>> +                        return ret;
>>> +        }
>>>
>>> -     return 0;
>>> +        /* init rest endpoints as output endpoints */
>>> +        for (num = 0; num < total - num_in_eps; num++) {
>>> +
>>> +                ret = dwc3_gadget_init_endpoint(dwc, num << 1, 0);
>>> +                if (ret)
>>> +                        return ret;
>>> +        }
>>> +
>>> +     return ret;
>>>  }
>>>
>>
>> * DWC3_NUM_EPS(parms) is the total number of endpoints configured in HW
>> * DWC3_NUM_IN_EPS(parms) is the max number of IN endpoints that SW may
>> configure
>>
>> The number of OUT endpoints does not mean DWC3_NUM_EPS(parms) -
>> DWC3_NUM_IN_EPS(parms).
>>
>> As long as physical endpoint 0 and 1 are dedicated for control endpoint,
>> other endpoints can be assigned as IN or OUT direction. So, you can have
>> as many as DWC3_NUM_EPS(parms) - 1 number of OUT endpoints.
>>
>> Currently, dwc3 driver assumes that DWC3_NUM_IN_EPS(params) is at least
>> half of DWC3_NUM_EPS(parms). If that's not the case, we may see
>> problems. To cover most application setup, the driver tries to setup
>> number of OUT = IN.
>>
>> For your case, is there an application that needs all
>> DWC3_NUM_IN_EPS(params)? If you're going to make a change, please keep
>> in mind of the info above to prevent regression.
>>
>> Thanks,
>> Thinh
>>
> 

Re: [PATCH] usb: dwc3: fix endpoint direction when inputs are more than outputs

[Greg Kroah-Hartman]

On Mon, Sep 06, 2021 at 01:08:34PM +0100, Shantur Rathore wrote:
> In RK3399 as per documentation (
> https://usermanual.wiki/Document/RockchipDeveloperGuidelinux44USB.31610806
> ), there are 7 Input Endpoints and 6 Output endpoints, in total 13
> endpoints.
> 
> Currently dwc3/gadget.c driver uses the number of endpoints
> available and starts setting them up with even endpoints as output
> endpoints and odd numbered as even endpoints. This leads to 7 Output
> endpoints and 6 input endpoints for RK3399.
> 
> If you try to create a composite gadget which uses all the input
> endpoints, one can see the issue. You just need to create functions to
> use up the last input ep and it would fail to create. No need to
> connect it to the host.
> This was confirmed when running a rockchip-linux bsp image.
> 
> [root@rockpro rock]# ls /sys/kernel/debug/usb/fe800000.usb/
> ep0in  ep0out  ep1in  ep1out  ep2in  ep2out  ep3in  ep3out  ep4in
> ep4out  ep5in  ep5out  ep6in  link_state  lsp_dump  mode  regdump
> testmode
> 
> Currently in linux mainline it is
> 
> [root@rockpro rock]# ls /sys/kernel/debug/usb/fe800000.usb/
> ep0in  ep0out  ep1in  ep1out  ep2in  ep2out  ep3in  ep3out  ep4in
> ep4out  ep5in  ep5out  ep6out  link_state  lsp_dump  mode  regdump
> testmode
> 
> ep6 being out instead of in as per the hardware spec.
> 
> Upon investigation of rockchip bsp kernel,
> https://github.com/rockchip-linux/kernel/
> 
> The issue was clear, currently, dwc3/gadget driver doesn't take
> DWC3_NUM_IN_EPS into consideration while enumerating them.
> 
> The patch below fixes the issue and ep6 is correctly enumerated as input.
> ---
>  drivers/usb/dwc3/core.c   |  1 +
>  drivers/usb/dwc3/core.h   |  1 +
>  drivers/usb/dwc3/gadget.c | 40 ++++++++++++++++++++++++---------------
>  3 files changed, 27 insertions(+), 15 deletions(-)
> 
> diff --git a/drivers/usb/dwc3/core.c b/drivers/usb/dwc3/core.c
> index 01866dcb953b..279c9a97cb8c 100644
> --- a/drivers/usb/dwc3/core.c
> +++ b/drivers/usb/dwc3/core.c
> @@ -555,6 +555,7 @@ static void dwc3_core_num_eps(struct dwc3 *dwc)
>  	struct dwc3_hwparams	*parms = &dwc->hwparams;
>  
>  	dwc->num_eps = DWC3_NUM_EPS(parms);
> +	dwc->num_in_eps = DWC3_NUM_IN_EPS(parms);
>  }
>  
>  static void dwc3_cache_hwparams(struct dwc3 *dwc)
> diff --git a/drivers/usb/dwc3/core.h b/drivers/usb/dwc3/core.h
> index 5612bfdf37da..89a0998c618c 100644
> --- a/drivers/usb/dwc3/core.h
> +++ b/drivers/usb/dwc3/core.h
> @@ -1227,6 +1227,7 @@ struct dwc3 {
>  	u8			speed;
>  
>  	u8			num_eps;
> +	u8			num_in_eps;
>  
>  	struct dwc3_hwparams	hwparams;
>  	struct debugfs_regset32	*regset;
> diff --git a/drivers/usb/dwc3/gadget.c b/drivers/usb/dwc3/gadget.c
> index 804b50548163..d9d19dc0a29f 100644
> --- a/drivers/usb/dwc3/gadget.c
> +++ b/drivers/usb/dwc3/gadget.c
> @@ -693,9 +693,11 @@ void dwc3_gadget_clear_tx_fifos(struct dwc3 *dwc)
>  
>  	dwc->last_fifo_depth = fifo_depth;
>  	/* Clear existing TXFIFO for all IN eps except ep0 */
> -	for (num = 3; num < min_t(int, dwc->num_eps, DWC3_ENDPOINTS_NUM);
> -	     num += 2) {
> +	for (num = 3; num < DWC3_ENDPOINTS_NUM; num += 2) {
>  		dep = dwc->eps[num];
> +
> +		if(!dep)
> +			continue;
>  		/* Don't change TXFRAMNUM on usb31 version */
>  		size = DWC3_IP_IS(DWC3) ? 0 :
>  			dwc3_readl(dwc->regs, DWC3_GTXFIFOSIZ(num >> 1)) &
> @@ -2257,7 +2259,7 @@ static void dwc3_stop_active_transfers(struct dwc3 *dwc)
>  {
>  	u32 epnum;
>  
> -	for (epnum = 2; epnum < dwc->num_eps; epnum++) {
> +	for (epnum = 2; epnum < DWC3_ENDPOINTS_NUM; epnum++) {
>  		struct dwc3_ep *dep;
>  
>  		dep = dwc->eps[epnum];
> @@ -2960,10 +2962,9 @@ static int dwc3_gadget_init_out_endpoint(struct dwc3_ep *dep)
>  	return dwc3_alloc_trb_pool(dep);
>  }
>  
> -static int dwc3_gadget_init_endpoint(struct dwc3 *dwc, u8 epnum)
> +static int dwc3_gadget_init_endpoint(struct dwc3 *dwc, u8 epnum, bool direction)
>  {
>  	struct dwc3_ep			*dep;
> -	bool				direction = epnum & 1;
>  	int				ret;
>  	u8				num = epnum >> 1;
>  
> @@ -3011,21 +3012,30 @@ static int dwc3_gadget_init_endpoint(struct dwc3 *dwc, u8 epnum)
>  	return 0;
>  }
>  
> -static int dwc3_gadget_init_endpoints(struct dwc3 *dwc, u8 total)
> +static int dwc3_gadget_init_endpoints(struct dwc3 *dwc, u8 total, u8 num_in_eps)
>  {
> -	u8				epnum;
> +	u8				num;
> +	int				ret;
>  
>  	INIT_LIST_HEAD(&dwc->gadget->ep_list);
>  
> -	for (epnum = 0; epnum < total; epnum++) {
> -		int			ret;
> +	/* init input endpoints as reported by hw */
> +        for (num = 0; num < num_in_eps; num++) {
>  
> -		ret = dwc3_gadget_init_endpoint(dwc, epnum);
> -		if (ret)
> -			return ret;
> -	}
> +                ret = dwc3_gadget_init_endpoint(dwc, (num << 1) + 1, 1);
> +                if (ret)
> +                        return ret;
> +        }
>  
> -	return 0;
> +        /* init rest endpoints as output endpoints */
> +        for (num = 0; num < total - num_in_eps; num++) {
> +
> +                ret = dwc3_gadget_init_endpoint(dwc, num << 1, 0);
> +                if (ret)
> +                        return ret;
> +        }
> +
> +	return ret;
>  }
>  
>  static void dwc3_gadget_free_endpoints(struct dwc3 *dwc)
> @@ -4282,7 +4292,7 @@ int dwc3_gadget_init(struct dwc3 *dwc)
>  	 * sure we're starting from a well known location.
>  	 */
>  
> -	ret = dwc3_gadget_init_endpoints(dwc, dwc->num_eps);
> +	ret = dwc3_gadget_init_endpoints(dwc, dwc->num_eps, dwc->num_in_eps);
>  	if (ret)
>  		goto err4;
>  
> -- 
> 2.33.0
> 

Hi,

This is the friendly patch-bot of Greg Kroah-Hartman.  You have sent him
a patch that has triggered this response.  He used to manually respond
to these common problems, but in order to save his sanity (he kept
writing the same thing over and over, yet to different people), I was
created.  Hopefully you will not take offence and will fix the problem
in your patch and resubmit it so that it can be accepted into the Linux
kernel tree.

You are receiving this message because of the following common error(s)
as indicated below:

- Your patch does not have a Signed-off-by: line.  Please read the
  kernel file, Documentation/SubmittingPatches and resend it after
  adding that line.  Note, the line needs to be in the body of the
  email, before the patch, not at the bottom of the patch or in the
  email signature.

If you wish to discuss this problem further, or you have questions about
how to resolve this issue, please feel free to respond to this email and
Greg will reply once he has dug out from the pending patches received
from other developers.

thanks,

greg k-h's patch email bot

Re: [PATCH] usb: dwc3: fix endpoint direction when inputs are more than outputs

[Thinh Nguyen]

Shantur Rathore wrote:
> On Tue, Sep 14, 2021 at 1:52 AM Thinh Nguyen <Thinh.Nguyen@synopsys.com>
> wrote:
> 
>> Shantur Rathore wrote:
>>> Hi Thinh,
>>
>> Hi Shantur, please don't do top-post.
>>
>>>
>>> Thanks for replying.
>>>
>>> As per the documentation for RK3399, it says
>>>
>>> USB3.0 Device Features
>>> Up to 7 IN endpoints, including control endpoint 0
>>> Up to 6 OUT endpoints, including control endpoint 0
>>>
>>
>> The statements above are not accurate. You can have more OUT endpoints,
>> but if you assign all 7 endpoints as IN, then you have up to 6 endpoints
>> left for OUT. The dwc3 controller is flexible and you can have various
>> configurations.
>>
>>> In this case, I think the hardware defines the endpoints being in / out.
>>> I am in no case a dwc3 expert or have DWC3 documentation so cannot say
>>> for certain.
>>> It is all based on RK3399 documentation.
>>>
>>> I am trying to use a composite gadget with
>>>
>>> - HID Keyboard - 1 IN
>>> - HID Mouse - 1 IN
>>> - NCM - 2 IN and 1 OUT
>>> - 2 Mass Storage Devices - 2 IN and 2 OUT
>>
>> Why do you need 2 Mass Storage devices? Can you use 2 LUNs instead?
>>
>> Multi lun mass storage devices don't work when you boot from them.
> So, I need to use dual Mass storage devices
> 

Ok.

> 
>>>
>>> That including ep0 totals to 7 IN endpoints and 3 Out endpoints.
>>>
>>>> Currently, dwc3 driver assumes that DWC3_NUM_IN_EPS(params) is at least
>>>> half of DWC3_NUM_EPS(parms). If that's not the case, we may see
>>>> problems. To cover most application setup, the driver tries to setup
>>>> number of OUT = IN.
>>>
>>> I think I am seeing this issue when I try to use the last IN endpoint.
>>>
>>> Can you please let me know the direction I should take?
>>>
>>
>> At the moment, I don't have a good/clean solution for this. dwc3 setups
>> half IN and half OUT from the total number of endpoints ahead of time.
>> Without the full context of the configuration from the composite layer,
>> it can't adjust easily.
>>
>> Maybe others have some ideas.
>>
> 
> It would be best if endpoints could be set up as IN or OUT as when
> allocated rather than preallocating it.
> Looking forward to other ideas.
> 

One way is if the controller driver can get the entire configuration
from the composite layer to check if it's possible to setup. So it knows
how many and what direction to setup the endpoints instead of the other
way around.

Maybe this is something that Wesley can also benefit for txfifo resizing
by enhancing the "check_config"?

BR,
Thinh