[PATCH net v3] sch_htb: Avoid grafting on htb_destroy_class_offload when destroying htb

[PATCH net v3] sch_htb: Avoid grafting on htb_destroy_class_offload when destroying htb

[Rahul Rameshbabu]

Peek at old qdisc and graft only when deleting a leaf class in the htb,
rather than when deleting the htb itself. Do not peek at the qdisc of the
netdev queue when destroying the htb. The caller may already have grafted a
new qdisc that is not part of the htb structure being destroyed.

This fix resolves two use cases.

  1. Using tc to destroy the htb.
    - Netdev was being prematurely activated before the htb was fully
      destroyed.
  2. Using tc to replace the htb with another qdisc (which also leads to
     the htb being destroyed).
    - Premature netdev activation like previous case. Newly grafted qdisc
      was also getting accidentally overwritten when destroying the htb.

Fixes: d03b195b5aa0 ("sch_htb: Hierarchical QoS hardware offload")
Signed-off-by: Rahul Rameshbabu <rrameshbabu@nvidia.com>
Reviewed-by: Saeed Mahameed <saeedm@nvidia.com>
Cc: Eric Dumazet <edumazet@google.com>
Cc: Maxim Mikityanskiy <maxtram95@gmail.com>
---

Notes:
    Changelog
    
    v2->v3:
      - Removed NULL assignment to struct Qdisc *old to catch uninitialized usage
      - Improved code comments based on previous review feedback
      - Removed WARN_ON(err && destroying) to avoid being triggered when the
        TC_HTB_LEAF_DEL_LAST_FORCE offload operation is used
      - Described underlying design errors that caused issues for use cases
        mentioned in commit message
    v1->v2:
      - Added use cases that were triggering relevant issues in commit message

 net/sched/sch_htb.c | 27 ++++++++++++++++-----------
 1 file changed, 16 insertions(+), 11 deletions(-)

diff --git a/net/sched/sch_htb.c b/net/sched/sch_htb.c
index 2238edece1a4..f46643850df8 100644
--- a/net/sched/sch_htb.c
+++ b/net/sched/sch_htb.c
@@ -1549,7 +1549,7 @@ static int htb_destroy_class_offload(struct Qdisc *sch, struct htb_class *cl,
 	struct tc_htb_qopt_offload offload_opt;
 	struct netdev_queue *dev_queue;
 	struct Qdisc *q = cl->leaf.q;
-	struct Qdisc *old = NULL;
+	struct Qdisc *old;
 	int err;
 
 	if (cl->level)
@@ -1557,14 +1557,17 @@ static int htb_destroy_class_offload(struct Qdisc *sch, struct htb_class *cl,
 
 	WARN_ON(!q);
 	dev_queue = htb_offload_get_queue(cl);
-	old = htb_graft_helper(dev_queue, NULL);
-	if (destroying)
-		/* Before HTB is destroyed, the kernel grafts noop_qdisc to
-		 * all queues.
+	/* When destroying, caller qdisc_graft grafts the new qdisc and invokes
+	 * qdisc_put for the qdisc being destroyed. htb_destroy_class_offload
+	 * does not need to graft or qdisc_put the qdisc being destroyed.
+	 */
+	if (!destroying) {
+		old = htb_graft_helper(dev_queue, NULL);
+		/* Last qdisc grafted should be the same as cl->leaf.q when
+		 * calling htb_delete.
 		 */
-		WARN_ON(!(old->flags & TCQ_F_BUILTIN));
-	else
 		WARN_ON(old != q);
+	}
 
 	if (cl->parent) {
 		_bstats_update(&cl->parent->bstats_bias,
@@ -1581,10 +1584,12 @@ static int htb_destroy_class_offload(struct Qdisc *sch, struct htb_class *cl,
 	};
 	err = htb_offload(qdisc_dev(sch), &offload_opt);
 
-	if (!err || destroying)
-		qdisc_put(old);
-	else
-		htb_graft_helper(dev_queue, old);
+	if (!destroying) {
+		if (!err)
+			qdisc_put(old);
+		else
+			htb_graft_helper(dev_queue, old);
+	}
 
 	if (last_child)
 		return err;
-- 
2.36.2

Previous related discussions

[1] https://lore.kernel.org/netdev/20230111203732.51363-1-rrameshbabu@nvidia.com/
[2] https://lore.kernel.org/netdev/20230110202003.25452-1-rrameshbabu@nvidia.com/
[3] https://lore.kernel.org/netdev/20230104174744.22280-1-rrameshbabu@nvidia.com/
[4] https://lore.kernel.org/all/CANn89iJSsFPBp5dYm3y6Jbbpuwbb9P+X3gmqk6zow0VWgx1Q-A@mail.gmail.com/

Re: [PATCH net v3] sch_htb: Avoid grafting on htb_destroy_class_offload when destroying htb

[Maxim Mikityanskiy]

On Thu, Jan 12, 2023 at 04:55:29PM -0800, Rahul Rameshbabu wrote:
> Peek at old qdisc and graft only when deleting a leaf class in the htb,
> rather than when deleting the htb itself. Do not peek at the qdisc of the
> netdev queue when destroying the htb. The caller may already have grafted a
> new qdisc that is not part of the htb structure being destroyed.
> 
> This fix resolves two use cases.
> 
>   1. Using tc to destroy the htb.
>     - Netdev was being prematurely activated before the htb was fully
>       destroyed.
>   2. Using tc to replace the htb with another qdisc (which also leads to
>      the htb being destroyed).
>     - Premature netdev activation like previous case. Newly grafted qdisc
>       was also getting accidentally overwritten when destroying the htb.
> 
> Fixes: d03b195b5aa0 ("sch_htb: Hierarchical QoS hardware offload")
> Signed-off-by: Rahul Rameshbabu <rrameshbabu@nvidia.com>
> Reviewed-by: Saeed Mahameed <saeedm@nvidia.com>
> Cc: Eric Dumazet <edumazet@google.com>
> Cc: Maxim Mikityanskiy <maxtram95@gmail.com>
> ---

Thanks, looks good to me!

Reviewed-by: Maxim Mikityanskiy <maxtram95@gmail.com>

Re: [PATCH net v3] sch_htb: Avoid grafting on htb_destroy_class_offload when destroying htb

[Jiri Pirko]

Fri, Jan 13, 2023 at 01:55:29AM CET, rrameshbabu@nvidia.com wrote:
>Peek at old qdisc and graft only when deleting a leaf class in the htb,
>rather than when deleting the htb itself. Do not peek at the qdisc of the
>netdev queue when destroying the htb. The caller may already have grafted a
>new qdisc that is not part of the htb structure being destroyed.
>
>This fix resolves two use cases.
>
>  1. Using tc to destroy the htb.
>    - Netdev was being prematurely activated before the htb was fully
>      destroyed.
>  2. Using tc to replace the htb with another qdisc (which also leads to
>     the htb being destroyed).
>    - Premature netdev activation like previous case. Newly grafted qdisc
>      was also getting accidentally overwritten when destroying the htb.
>
>Fixes: d03b195b5aa0 ("sch_htb: Hierarchical QoS hardware offload")
>Signed-off-by: Rahul Rameshbabu <rrameshbabu@nvidia.com>
>Reviewed-by: Saeed Mahameed <saeedm@nvidia.com>
>Cc: Eric Dumazet <edumazet@google.com>
>Cc: Maxim Mikityanskiy <maxtram95@gmail.com>

Reviewed-by: Jiri Pirko <jiri@nvidia.com>

Re: [PATCH net v3] sch_htb: Avoid grafting on htb_destroy_class_offload when destroying htb

[patchwork-bot+netdevbpf]

Hello:

This patch was applied to netdev/net.git (master)
by Jakub Kicinski <kuba@kernel.org>:

On Thu, 12 Jan 2023 16:55:29 -0800 you wrote:
> Peek at old qdisc and graft only when deleting a leaf class in the htb,
> rather than when deleting the htb itself. Do not peek at the qdisc of the
> netdev queue when destroying the htb. The caller may already have grafted a
> new qdisc that is not part of the htb structure being destroyed.
> 
> This fix resolves two use cases.
> 
> [...]

Here is the summary with links:
  - [net,v3] sch_htb: Avoid grafting on htb_destroy_class_offload when destroying htb
    https://git.kernel.org/netdev/net/c/a22b7388d658

You are awesome, thank you!
-- 
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html