[PATCH 1/2] mac80211: add tx dequeue function for process context

[PATCH 1/2] mac80211: add tx dequeue function for process context

[Erik Stromdahl]

Since ieee80211_tx_dequeue() must not be called with softirqs enabled
(i.e. from process context without proper disable of bottom halves),
we add a wrapper that disables bottom halves before calling
ieee80211_tx_dequeue()

The new function is named ieee80211_tx_dequeue_ni() just as all other
from-process-context versions found in mac80211.

The documentation of ieee80211_tx_dequeue() is also updated so it
mentions that the function should not be called from process context.

Signed-off-by: Erik Stromdahl <erik.stromdahl@gmail.com>
---
 include/net/mac80211.h | 26 ++++++++++++++++++++++++++
 net/mac80211/tx.c      |  2 ++
 2 files changed, 28 insertions(+)

diff --git a/include/net/mac80211.h b/include/net/mac80211.h
index 72080d9d617e..c47990d8db79 100644
--- a/include/net/mac80211.h
+++ b/include/net/mac80211.h
@@ -6251,10 +6251,36 @@ void ieee80211_unreserve_tid(struct ieee80211_sta *sta, u8 tid);
  * but for the duration of the frame handling.
  * However, also note that while in the wake_tx_queue() method,
  * rcu_read_lock() is already held.
+ *
+ * softirqs must also be disabled when this function is called.
+ * In process context, use ieee80211_tx_dequeue_ni() instead.
  */
 struct sk_buff *ieee80211_tx_dequeue(struct ieee80211_hw *hw,
 				     struct ieee80211_txq *txq);
 
+/**
+ * ieee80211_tx_dequeue_ni - dequeue a packet from a software tx queue
+ * (in process context)
+ *
+ * Like ieee80211_tx_dequeue() but can be called in process context
+ * (internally disables bottom halves).
+ *
+ * @hw: pointer as obtained from ieee80211_alloc_hw()
+ * @txq: pointer obtained from station or virtual interface, or from
+ *	ieee80211_next_txq()
+ */
+static inline struct sk_buff *ieee80211_tx_dequeue_ni(struct ieee80211_hw *hw,
+						      struct ieee80211_txq *txq)
+{
+	struct sk_buff *skb;
+
+	local_bh_disable();
+	skb = ieee80211_tx_dequeue(hw, txq);
+	local_bh_enable();
+
+	return skb;
+}
+
 /**
  * ieee80211_next_txq - get next tx queue to pull packets from
  *
diff --git a/net/mac80211/tx.c b/net/mac80211/tx.c
index dd220b977025..4bd0066ea7cd 100644
--- a/net/mac80211/tx.c
+++ b/net/mac80211/tx.c
@@ -3550,6 +3550,8 @@ struct sk_buff *ieee80211_tx_dequeue(struct ieee80211_hw *hw,
 	ieee80211_tx_result r;
 	struct ieee80211_vif *vif = txq->vif;
 
+	WARN_ON_ONCE(softirq_count() == 0);
+
 begin:
 	spin_lock_bh(&fq->lock);
 
-- 
2.22.0

[PATCH 2/2] ath10k: switch to ieee80211_tx_dequeue_ni

[Erik Stromdahl]

Since ath10k_mac_tx_push_txq() can be called from process context, we
must explicitly disable softirqs before the call into mac80211.

By calling ieee80211_tx_dequeue_ni() instead of ieee80211_tx_dequeue()
we make sure softirqs are always disabled even in the case when
ath10k_mac_tx_push_txq() is called from process context.

Calling ieee80211_tx_dequeue_ni() with softirq's already disabled
(e.g., from softirq context) should be safe as the local_bh_disable()
and local_bh_enable() functions (called from ieee80211_tx_dequeue_ni)
are fully reentrant.

Signed-off-by: Erik Stromdahl <erik.stromdahl@gmail.com>
---
 drivers/net/wireless/ath/ath10k/mac.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/net/wireless/ath/ath10k/mac.c b/drivers/net/wireless/ath/ath10k/mac.c
index 2d503d6cdcd2..bbed9f1b1778 100644
--- a/drivers/net/wireless/ath/ath10k/mac.c
+++ b/drivers/net/wireless/ath/ath10k/mac.c
@@ -4033,7 +4033,7 @@ int ath10k_mac_tx_push_txq(struct ieee80211_hw *hw,
 	if (ret)
 		return ret;
 
-	skb = ieee80211_tx_dequeue(hw, txq);
+	skb = ieee80211_tx_dequeue_ni(hw, txq);
 	if (!skb) {
 		spin_lock_bh(&ar->htt.tx_lock);
 		ath10k_htt_tx_dec_pending(htt);
-- 
2.22.0

Re: [PATCH 2/2] ath10k: switch to ieee80211_tx_dequeue_ni

[Kalle Valo]

Erik Stromdahl <erik.stromdahl@gmail.com> wrote:

> Since ath10k_mac_tx_push_txq() can be called from process context, we
> must explicitly disable softirqs before the call into mac80211.
> 
> By calling ieee80211_tx_dequeue_ni() instead of ieee80211_tx_dequeue()
> we make sure softirqs are always disabled even in the case when
> ath10k_mac_tx_push_txq() is called from process context.
> 
> Calling ieee80211_tx_dequeue_ni() with softirq's already disabled
> (e.g., from softirq context) should be safe as the local_bh_disable()
> and local_bh_enable() functions (called from ieee80211_tx_dequeue_ni)
> are fully reentrant.
> 
> Signed-off-by: Erik Stromdahl <erik.stromdahl@gmail.com>
> Signed-off-by: Kalle Valo <kvalo@codeaurora.org>

Patch applied to ath-next branch of ath.git, thanks.

306547608c84 ath10k: switch to ieee80211_tx_dequeue_ni

-- 
https://patchwork.kernel.org/patch/11000363/

https://wireless.wiki.kernel.org/en/developers/documentation/submittingpatches

Re: [PATCH 2/2] ath10k: switch to ieee80211_tx_dequeue_ni

[Kalle Valo]

Erik Stromdahl <erik.stromdahl@gmail.com> writes:

> Since ath10k_mac_tx_push_txq() can be called from process context, we
> must explicitly disable softirqs before the call into mac80211.
>
> By calling ieee80211_tx_dequeue_ni() instead of ieee80211_tx_dequeue()
> we make sure softirqs are always disabled even in the case when
> ath10k_mac_tx_push_txq() is called from process context.
>
> Calling ieee80211_tx_dequeue_ni() with softirq's already disabled
> (e.g., from softirq context) should be safe as the local_bh_disable()
> and local_bh_enable() functions (called from ieee80211_tx_dequeue_ni)
> are fully reentrant.
>
> Signed-off-by: Erik Stromdahl <erik.stromdahl@gmail.com>

I already applied this, but I still want to check _why_ you are changing
this? Is it that you want to call ath10k_mac_tx_push_pending() from a
workqueue in sdio.c in a future patch, or what? Because at the moment me
and Johannes were not able to find where this is called in process
context.

-- 
https://wireless.wiki.kernel.org/en/developers/documentation/submittingpatches

Re: [PATCH 2/2] ath10k: switch to ieee80211_tx_dequeue_ni

[Peter Oh]

On 10/1/19 4:48 AM, Kalle Valo wrote:
> Erik Stromdahl <erik.stromdahl@gmail.com> writes:
>
>> Since ath10k_mac_tx_push_txq() can be called from process context, we
>> must explicitly disable softirqs before the call into mac80211.
>>
>> By calling ieee80211_tx_dequeue_ni() instead of ieee80211_tx_dequeue()
>> we make sure softirqs are always disabled even in the case when
>> ath10k_mac_tx_push_txq() is called from process context.
>>
>> Calling ieee80211_tx_dequeue_ni() with softirq's already disabled
>> (e.g., from softirq context) should be safe as the local_bh_disable()
>> and local_bh_enable() functions (called from ieee80211_tx_dequeue_ni)
>> are fully reentrant.
>>
>> Signed-off-by: Erik Stromdahl <erik.stromdahl@gmail.com>
> I already applied this, but I still want to check _why_ you are changing
> this? Is it that you want to call ath10k_mac_tx_push_pending() from a
> workqueue in sdio.c in a future patch, or what? Because at the moment me
> and Johannes were not able to find where this is called in process
> context.
>
It seems Johannes wants to fix it in mac80211.

[PATCH v2] mac80211: keep BHs disabled while calling drv_tx_wake_queue()

Drivers typically expect this, as it's the case for almost all cases
where this is called (i.e. from the TX path). Also, the code in mac80211
itself (if the driver calls ieee80211_tx_dequeue()) expects this as it
uses this_cpu_ptr() without additional protection.

Re: [PATCH 2/2] ath10k: switch to ieee80211_tx_dequeue_ni

[Erik Stromdahl]

On 10/1/19 7:13 PM, Peter Oh wrote:
> 
> On 10/1/19 4:48 AM, Kalle Valo wrote:
>> Erik Stromdahl <erik.stromdahl@gmail.com> writes:
>>
>>> Since ath10k_mac_tx_push_txq() can be called from process context, we
>>> must explicitly disable softirqs before the call into mac80211.
>>>
>>> By calling ieee80211_tx_dequeue_ni() instead of ieee80211_tx_dequeue()
>>> we make sure softirqs are always disabled even in the case when
>>> ath10k_mac_tx_push_txq() is called from process context.
>>>
>>> Calling ieee80211_tx_dequeue_ni() with softirq's already disabled
>>> (e.g., from softirq context) should be safe as the local_bh_disable()
>>> and local_bh_enable() functions (called from ieee80211_tx_dequeue_ni)
>>> are fully reentrant.
>>>
>>> Signed-off-by: Erik Stromdahl <erik.stromdahl@gmail.com>
>> I already applied this, but I still want to check _why_ you are changing
>> this? Is it that you want to call ath10k_mac_tx_push_pending() from a
>> workqueue in sdio.c in a future patch, or what? Because at the moment me
>> and Johannes were not able to find where this is called in process
>> context.
>>
SDIO irqs are threaded irqs (at least on my iMX6 board) and hence process context.
I will see if I can find a trace that shows the call chain more exactly.


> It seems Johannes wants to fix it in mac80211.
> 
> [PATCH v2] mac80211: keep BHs disabled while calling drv_tx_wake_queue()
> 
> Drivers typically expect this, as it's the case for almost all cases
> where this is called (i.e. from the TX path). Also, the code in mac80211
> itself (if the driver calls ieee80211_tx_dequeue()) expects this as it
> uses this_cpu_ptr() without additional protection.
> 

Re: [PATCH 2/2] ath10k: switch to ieee80211_tx_dequeue_ni

[Erik Stromdahl]

On 10/9/19 9:23 PM, Erik Stromdahl wrote:
> 
> 
> On 10/1/19 7:13 PM, Peter Oh wrote:
>>
>> On 10/1/19 4:48 AM, Kalle Valo wrote:
>>> Erik Stromdahl <erik.stromdahl@gmail.com> writes:
>>>
>>>> Since ath10k_mac_tx_push_txq() can be called from process context, we
>>>> must explicitly disable softirqs before the call into mac80211.
>>>>
>>>> By calling ieee80211_tx_dequeue_ni() instead of ieee80211_tx_dequeue()
>>>> we make sure softirqs are always disabled even in the case when
>>>> ath10k_mac_tx_push_txq() is called from process context.
>>>>
>>>> Calling ieee80211_tx_dequeue_ni() with softirq's already disabled
>>>> (e.g., from softirq context) should be safe as the local_bh_disable()
>>>> and local_bh_enable() functions (called from ieee80211_tx_dequeue_ni)
>>>> are fully reentrant.
>>>>
>>>> Signed-off-by: Erik Stromdahl <erik.stromdahl@gmail.com>
>>> I already applied this, but I still want to check _why_ you are changing
>>> this? Is it that you want to call ath10k_mac_tx_push_pending() from a
>>> workqueue in sdio.c in a future patch, or what? Because at the moment me
>>> and Johannes were not able to find where this is called in process
>>> context.
>>>
> SDIO irqs are threaded irqs (at least on my iMX6 board) and hence process context.
> I will see if I can find a trace that shows the call chain more exactly.
> 

I found this backtrace in a log file:
(it does not show the call to ieee80211_tx_dequeue_ni(), but it shows that
ath10k_sdio_irq_handler() is called from process context)

          irq/62-mmc1-65    [000] ....   785.261081: ath10k_mac_op_wake_tx_queue <-ieee80211_queue_skb
          irq/62-mmc1-65    [000] ....   785.261090: <stack trace>
      => ieee80211_queue_skb
      => __ieee80211_subif_start_xmit
      => ieee80211_subif_start_xmit
      => dev_hard_start_xmit
      => __dev_queue_xmit
      => dev_queue_xmit
      => ip_finish_output2
      => ip_finish_output
      => ip_output
      => ip_local_out
      => ip_queue_xmit
      => tcp_transmit_skb
      => tcp_write_xmit
      => __tcp_push_pending_frames
      => tcp_rcv_established
      => tcp_v4_do_rcv
      => tcp_v4_rcv
      => ip_local_deliver_finish
      => ip_local_deliver
      => ip_rcv_finish
      => ip_rcv
      => __netif_receive_skb_core
      => __netif_receive_skb
      => netif_receive_skb_internal
      => netif_receive_skb
      => ieee80211_deliver_skb
      => ieee80211_rx_handlers
      => ieee80211_prepare_and_rx_handle
      => ieee80211_rx_napi
      => ath10k_htt_t2h_msg_handler
      => ath10k_htt_htc_t2h_msg_handler
      => ath10k_sdio_mbox_rxmsg_pending_handler
      => ath10k_sdio_irq_handler                        <- ath10k_mac_tx_push_pending() is called from here
      => process_sdio_pending_irqs
      => sdio_run_irqs
      => sdhci_thread_irq
      => irq_thread_fn
      => irq_thread
      => kthread
      => ret_from_fork
      => 0

 From ath10k_sdio_irq_handler(), the call chain down to ieee80211_tx_dequeue_ni()
looks like this:

ath10k_sdio_irq_handler() =>
   ath10k_mac_tx_push_pending() =>
     ath10k_mac_schedule_txq() =>
       ath10k_mac_tx_push_txq() =>
         ieee80211_tx_dequeue_ni()

> 
>> It seems Johannes wants to fix it in mac80211.
>>
>> [PATCH v2] mac80211: keep BHs disabled while calling drv_tx_wake_queue()
>>
>> Drivers typically expect this, as it's the case for almost all cases
>> where this is called (i.e. from the TX path). Also, the code in mac80211
>> itself (if the driver calls ieee80211_tx_dequeue()) expects this as it
>> uses this_cpu_ptr() without additional protection.
>>