All of lore.kernel.org
 help / color / mirror / Atom feed
From: Kan Yan <kyan@google.com>
To: Johannes Berg <johannes@sipsolutions.net>
Cc: "Toke Høiland-Jørgensen" <toke@redhat.com>,
	linux-wireless@vger.kernel.org,
	Make-Wifi-fast <make-wifi-fast@lists.bufferbloat.net>,
	ath10k@lists.infradead.org, "John Crispin" <john@phrozen.org>,
	"Lorenzo Bianconi" <lorenzo@kernel.org>,
	"Felix Fietkau" <nbd@nbd.name>,
	"Rajkumar Manoharan" <rmanohar@codeaurora.org>,
	"Kevin Hayes" <kevinhayes@google.com>
Subject: Re: [PATCH v6 3/4] mac80211: Implement Airtime-based Queue Limit (AQL)
Date: Fri, 8 Nov 2019 17:22:38 -0800	[thread overview]
Message-ID: <CA+iem5saT0ntxnSV1ZatFuRRcaBzivkQ_MU66ZdV1kfb1dC6Qg@mail.gmail.com> (raw)
In-Reply-To: <300bf0146db6c0d5890699b3911d35174d28c9c0.camel@sipsolutions.net>

It is most likely just insufficient locking. active_txq_lock is per
AC, can't protect local->aql_total_pending_airtime against racing
conditions:
void ieee80211_sta_update_pending_airtime(...)
{
        spin_lock_bh(&local->active_txq_lock[ac]);
        ...
        local->aql_total_pending_airtime -= tx_airtime;
        ...
        spin_unlock_bh(&local->active_txq_lock[ac]);
}

After changing it to atomic_t, no more aql_total_pending_airtime
underflow so far :). Using atomic operation should also help reduce
CPU overhead due to lock contention, as
ieee80211_sta_update_pending_airtime() is often called from the tx
completion routine triggered by interrupts, often in a different core
than where __ieee80211_schedule_txq() is running.

I will post a new version a bit later if the test goes well.

Regards,
Kan


On Fri, Nov 8, 2019 at 3:17 AM Johannes Berg <johannes@sipsolutions.net> wrote:
>
> On Fri, 2019-11-08 at 12:10 +0100, Toke Høiland-Jørgensen wrote:
>
> > Right, bugger. I was thinking maybe there's a case where skbs can be
> > cloned (and retain the tx_time_est field) and then released twice?
>
> They could be cloned, but I don't see how that'd be while *inside* the
> stack and then they get reported twice - unless the driver did something
> like that?
>
> I mean, TCP surely does that for example, but it's before we even get to
> mac80211.
>
> > Or
> > maybe somewhere that steps on the skb->cb field in some other way?
> > Couldn't find anything obvious on a first perusal of the TX path code,
> > but maybe you could think of something?
>
> No, sorry. But I also didn't actually look at the driver at all.
>
> > Otherwise I guess we'll be forced to go and do some actual,
> > old-fashioned debugging ;)
>
> :)
>
> johannes
>

WARNING: multiple messages have this Message-ID (diff)
From: Kan Yan <kyan@google.com>
To: Johannes Berg <johannes@sipsolutions.net>
Cc: "Rajkumar Manoharan" <rmanohar@codeaurora.org>,
	"Kevin Hayes" <kevinhayes@google.com>,
	Make-Wifi-fast <make-wifi-fast@lists.bufferbloat.net>,
	"Toke Høiland-Jørgensen" <toke@redhat.com>,
	linux-wireless@vger.kernel.org, ath10k@lists.infradead.org,
	"John Crispin" <john@phrozen.org>,
	"Lorenzo Bianconi" <lorenzo@kernel.org>,
	"Felix Fietkau" <nbd@nbd.name>
Subject: Re: [PATCH v6 3/4] mac80211: Implement Airtime-based Queue Limit (AQL)
Date: Fri, 8 Nov 2019 17:22:38 -0800	[thread overview]
Message-ID: <CA+iem5saT0ntxnSV1ZatFuRRcaBzivkQ_MU66ZdV1kfb1dC6Qg@mail.gmail.com> (raw)
In-Reply-To: <300bf0146db6c0d5890699b3911d35174d28c9c0.camel@sipsolutions.net>

It is most likely just insufficient locking. active_txq_lock is per
AC, can't protect local->aql_total_pending_airtime against racing
conditions:
void ieee80211_sta_update_pending_airtime(...)
{
        spin_lock_bh(&local->active_txq_lock[ac]);
        ...
        local->aql_total_pending_airtime -= tx_airtime;
        ...
        spin_unlock_bh(&local->active_txq_lock[ac]);
}

After changing it to atomic_t, no more aql_total_pending_airtime
underflow so far :). Using atomic operation should also help reduce
CPU overhead due to lock contention, as
ieee80211_sta_update_pending_airtime() is often called from the tx
completion routine triggered by interrupts, often in a different core
than where __ieee80211_schedule_txq() is running.

I will post a new version a bit later if the test goes well.

Regards,
Kan


On Fri, Nov 8, 2019 at 3:17 AM Johannes Berg <johannes@sipsolutions.net> wrote:
>
> On Fri, 2019-11-08 at 12:10 +0100, Toke Høiland-Jørgensen wrote:
>
> > Right, bugger. I was thinking maybe there's a case where skbs can be
> > cloned (and retain the tx_time_est field) and then released twice?
>
> They could be cloned, but I don't see how that'd be while *inside* the
> stack and then they get reported twice - unless the driver did something
> like that?
>
> I mean, TCP surely does that for example, but it's before we even get to
> mac80211.
>
> > Or
> > maybe somewhere that steps on the skb->cb field in some other way?
> > Couldn't find anything obvious on a first perusal of the TX path code,
> > but maybe you could think of something?
>
> No, sorry. But I also didn't actually look at the driver at all.
>
> > Otherwise I guess we'll be forced to go and do some actual,
> > old-fashioned debugging ;)
>
> :)
>
> johannes
>

_______________________________________________
ath10k mailing list
ath10k@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/ath10k

  reply	other threads:[~2019-11-09  1:22 UTC|newest]

Thread overview: 43+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-10-23  9:58 [PATCH v6 0/4] Add Airtime Queue Limits (AQL) to mac80211 Toke Høiland-Jørgensen
2019-10-23  9:58 ` Toke Høiland-Jørgensen
2019-10-23  9:59 ` [PATCH v6 1/4] mac80211: Shrink the size of ack_frame_id to make room for tx_time_est Toke Høiland-Jørgensen
2019-10-23  9:59   ` Toke Høiland-Jørgensen
2019-11-08 10:03   ` Johannes Berg
2019-11-08 10:03     ` Johannes Berg
2019-10-23  9:59 ` [PATCH v6 2/4] mac80211: Import airtime calculation code from mt76 Toke Høiland-Jørgensen
2019-10-23  9:59   ` Toke Høiland-Jørgensen
2019-11-08 10:07   ` Johannes Berg
2019-11-08 10:07     ` Johannes Berg
2019-11-08 10:55     ` Toke Høiland-Jørgensen
2019-11-08 10:55       ` Toke Høiland-Jørgensen
2019-11-08 10:57       ` Johannes Berg
2019-11-08 10:57         ` Johannes Berg
2019-10-23  9:59 ` [PATCH v6 3/4] mac80211: Implement Airtime-based Queue Limit (AQL) Toke Høiland-Jørgensen
2019-10-23  9:59   ` Toke Høiland-Jørgensen
2019-11-08 10:09   ` Johannes Berg
2019-11-08 10:09     ` Johannes Berg
2019-11-08 10:56     ` Toke Høiland-Jørgensen
2019-11-08 10:56       ` Toke Høiland-Jørgensen
2019-11-08 10:59       ` Johannes Berg
2019-11-08 10:59         ` Johannes Berg
2019-11-08 11:10         ` Toke Høiland-Jørgensen
2019-11-08 11:10           ` Toke Høiland-Jørgensen
2019-11-08 11:17           ` Johannes Berg
2019-11-08 11:17             ` Johannes Berg
2019-11-09  1:22             ` Kan Yan [this message]
2019-11-09  1:22               ` Kan Yan
2019-11-09 11:22               ` Toke Høiland-Jørgensen
2019-11-09 11:22                 ` Toke Høiland-Jørgensen
2019-10-23  9:59 ` [PATCH v6 4/4] mac80211: Use Airtime-based Queue Limits (AQL) on packet dequeue Toke Høiland-Jørgensen
2019-10-23  9:59   ` Toke Høiland-Jørgensen
2019-11-08 10:17   ` Johannes Berg
2019-11-08 10:17     ` Johannes Berg
2019-11-08 11:01     ` Toke Høiland-Jørgensen
2019-11-08 11:01       ` Toke Høiland-Jørgensen
2019-11-08 11:05       ` Johannes Berg
2019-11-08 11:05         ` Johannes Berg
2019-11-07  6:14 ` [PATCH v6 0/4] Add Airtime Queue Limits (AQL) to mac80211 Kan Yan
2019-11-07  6:14   ` Kan Yan
2019-11-07 10:55   ` Toke Høiland-Jørgensen
2019-11-07 10:55     ` Toke Høiland-Jørgensen
2019-11-07 21:24   ` [Make-wifi-fast] " Dave Taht

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=CA+iem5saT0ntxnSV1ZatFuRRcaBzivkQ_MU66ZdV1kfb1dC6Qg@mail.gmail.com \
    --to=kyan@google.com \
    --cc=ath10k@lists.infradead.org \
    --cc=johannes@sipsolutions.net \
    --cc=john@phrozen.org \
    --cc=kevinhayes@google.com \
    --cc=linux-wireless@vger.kernel.org \
    --cc=lorenzo@kernel.org \
    --cc=make-wifi-fast@lists.bufferbloat.net \
    --cc=nbd@nbd.name \
    --cc=rmanohar@codeaurora.org \
    --cc=toke@redhat.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.