From: Dave Taht <dave@taht.net> To: "Toke Høiland-Jørgensen" <toke@toke.dk> Cc: Felix Fietkau <nbd@nbd.name>, Rajkumar Manoharan <rmanohar@codeaurora.org>, linux-wireless@vger.kernel.org, ath10k@lists.infradead.org, make-wifi-fast@lists.bufferbloat.net Subject: Re: [Make-wifi-fast] [PATCH v3 3/6] mac80211: Add airtime accounting and scheduling to TXQs Date: Mon, 19 Nov 2018 09:55:16 -0800 [thread overview] Message-ID: <878t1p2bqz.fsf@taht.net> (raw) In-Reply-To: <87sh02tfsp.fsf@toke.dk> ("Toke \=\?utf-8\?Q\?H\=C3\=B8iland-J\?\= \=\?utf-8\?Q\?\=C3\=B8rgensen\=22's\?\= message of "Thu, 15 Nov 2018 09:24:22 -0800") Toke Høiland-Jørgensen <toke@toke.dk> writes: > Felix Fietkau <nbd@nbd.name> writes: > >> On 2018-11-14 18:40, Toke Høiland-Jørgensen wrote: >>>> This part doesn't really make much sense to me, but maybe I'm >>>> misunderstanding how the code works. >>>> Let's assume we have a driver like ath9k or mt76, which tries to keep a >>>> number of aggregates in the hardware queue, and the hardware queue is >>>> currently empty. >>>> If the current txq entry is kept at the head of the schedule list, >>>> wouldn't the code just pull from that one over and over again, until >>>> enough packets are transmitted by the hardware and their tx status >>>> processed? >>>> It seems to me that while fairness is still preserved in the long run, >>>> this could lead to rather bursty scheduling, which may not be >>>> particularly latency friendly. >>> >>> Yes, it'll be a bit more bursty when the hardware queue is completely >>> empty. However, when a TX completion comes back, that will adjust the >>> deficit of that sta and cause it to be rotated on the next dequeue. This >>> obviously relies on the fact that the lower-level hardware queue is >>> sufficiently shallow to not add a lot of latency. But we want that to be >>> the case anyway. In practice, it works quite well for ath9k, but not so >>> well for ath10k because it has a large buffer in firmware. >>> >>> If we requeue the TXQ at the end of the list, a station that is taking >>> up too much airtime will fail to be throttled properly, so the >>> queue-at-head is kinda needed to ensure fairness... >> Thanks for the explanation, that makes sense to me. I have an idea on >> how to mitigate the burstiness within the driver. I'll write it down in >> pseudocode, please let me know if you think that'll work. > > I don't think it will, unfortunately. For example, consider the case > where there are two stations queued; one with a large negative deficit > (say, -10ms), and one with a positive deficit. Perhaps a flag for one way or the other? if(driver->has_absurd_hardware_queue_depth) doitthisway(); else doitabetterway(); > > In this case, we really need to throttle the station with a negative > deficit. But if the driver loops and caches txqs, we'll get something > like the following: > > - First driver loop iteration: returns TXQ with positive deficit. > - Second driver loop iteration: Only the negative-deficit TXQ is in the > mac80211 list, so it will loop until that TXQ's deficit turns positive > and return it. > > Because of this, the negative-deficit station won't be throttled, and we > won't get fairness. > > How many frames will mt76 queue up below the driver point? I.e., how > much burstiness are you expecting this will introduce on that driver? > > Taking a step back, it's clear that it would be good to be able to > dequeue packets to multiple STAs at once (we need that for MU-MIMO on > ath10k as well). However, I don't think we can do that with the > round-robin fairness scheduler; so we are going to need a different > algorithm. I *think* it may be possible to do this with a virtual-time > scheduler, but I haven't sat down and worked out the details yet... The answer to which did not fit on the margins of your thesis. :) I too have been trying to come up with a better means of gang scheduling... for about 2 years now. In terms of bitmaps it looks a bit like QFQ, but honestly... Is there going to be some point where whatever we have here is significantly better than what we had? Or not significantly worse? Or handwavy enough to fix the rest once enlightenment arrives? The perfect is the enemy of the good. I'd rather like the intel folk to be weighing in on this stuff, too, trying to get an API right requires use cases. > > -Toke > _______________________________________________ > Make-wifi-fast mailing list > Make-wifi-fast@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/make-wifi-fast
WARNING: multiple messages have this Message-ID (diff)
From: Dave Taht <dave@taht.net> To: "Toke Høiland-Jørgensen" <toke@toke.dk> Cc: make-wifi-fast@lists.bufferbloat.net, linux-wireless@vger.kernel.org, ath10k@lists.infradead.org, Rajkumar Manoharan <rmanohar@codeaurora.org>, Felix Fietkau <nbd@nbd.name> Subject: Re: [Make-wifi-fast] [PATCH v3 3/6] mac80211: Add airtime accounting and scheduling to TXQs Date: Mon, 19 Nov 2018 09:55:16 -0800 [thread overview] Message-ID: <878t1p2bqz.fsf@taht.net> (raw) In-Reply-To: <87sh02tfsp.fsf@toke.dk> ("Toke \=\?utf-8\?Q\?H\=C3\=B8iland-J\?\= \=\?utf-8\?Q\?\=C3\=B8rgensen\=22's\?\= message of "Thu, 15 Nov 2018 09:24:22 -0800") Toke Høiland-Jørgensen <toke@toke.dk> writes: > Felix Fietkau <nbd@nbd.name> writes: > >> On 2018-11-14 18:40, Toke Høiland-Jørgensen wrote: >>>> This part doesn't really make much sense to me, but maybe I'm >>>> misunderstanding how the code works. >>>> Let's assume we have a driver like ath9k or mt76, which tries to keep a >>>> number of aggregates in the hardware queue, and the hardware queue is >>>> currently empty. >>>> If the current txq entry is kept at the head of the schedule list, >>>> wouldn't the code just pull from that one over and over again, until >>>> enough packets are transmitted by the hardware and their tx status >>>> processed? >>>> It seems to me that while fairness is still preserved in the long run, >>>> this could lead to rather bursty scheduling, which may not be >>>> particularly latency friendly. >>> >>> Yes, it'll be a bit more bursty when the hardware queue is completely >>> empty. However, when a TX completion comes back, that will adjust the >>> deficit of that sta and cause it to be rotated on the next dequeue. This >>> obviously relies on the fact that the lower-level hardware queue is >>> sufficiently shallow to not add a lot of latency. But we want that to be >>> the case anyway. In practice, it works quite well for ath9k, but not so >>> well for ath10k because it has a large buffer in firmware. >>> >>> If we requeue the TXQ at the end of the list, a station that is taking >>> up too much airtime will fail to be throttled properly, so the >>> queue-at-head is kinda needed to ensure fairness... >> Thanks for the explanation, that makes sense to me. I have an idea on >> how to mitigate the burstiness within the driver. I'll write it down in >> pseudocode, please let me know if you think that'll work. > > I don't think it will, unfortunately. For example, consider the case > where there are two stations queued; one with a large negative deficit > (say, -10ms), and one with a positive deficit. Perhaps a flag for one way or the other? if(driver->has_absurd_hardware_queue_depth) doitthisway(); else doitabetterway(); > > In this case, we really need to throttle the station with a negative > deficit. But if the driver loops and caches txqs, we'll get something > like the following: > > - First driver loop iteration: returns TXQ with positive deficit. > - Second driver loop iteration: Only the negative-deficit TXQ is in the > mac80211 list, so it will loop until that TXQ's deficit turns positive > and return it. > > Because of this, the negative-deficit station won't be throttled, and we > won't get fairness. > > How many frames will mt76 queue up below the driver point? I.e., how > much burstiness are you expecting this will introduce on that driver? > > Taking a step back, it's clear that it would be good to be able to > dequeue packets to multiple STAs at once (we need that for MU-MIMO on > ath10k as well). However, I don't think we can do that with the > round-robin fairness scheduler; so we are going to need a different > algorithm. I *think* it may be possible to do this with a virtual-time > scheduler, but I haven't sat down and worked out the details yet... The answer to which did not fit on the margins of your thesis. :) I too have been trying to come up with a better means of gang scheduling... for about 2 years now. In terms of bitmaps it looks a bit like QFQ, but honestly... Is there going to be some point where whatever we have here is significantly better than what we had? Or not significantly worse? Or handwavy enough to fix the rest once enlightenment arrives? The perfect is the enemy of the good. I'd rather like the intel folk to be weighing in on this stuff, too, trying to get an API right requires use cases. > > -Toke > _______________________________________________ > Make-wifi-fast mailing list > Make-wifi-fast@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/make-wifi-fast _______________________________________________ ath10k mailing list ath10k@lists.infradead.org http://lists.infradead.org/mailman/listinfo/ath10k
next prev parent reply other threads:[~2018-11-19 18:01 UTC|newest] Thread overview: 56+ messages / expand[flat|nested] mbox.gz Atom feed top 2018-11-12 22:51 [PATCH v3 0/6] Move TXQ scheduling and airtime fairness into mac80211 Rajkumar Manoharan 2018-11-12 22:51 ` Rajkumar Manoharan 2018-11-12 22:51 ` [PATCH v3 1/6] mac80211: Add TXQ scheduling API Rajkumar Manoharan 2018-11-12 22:51 ` Rajkumar Manoharan 2018-11-12 22:51 ` [PATCH v3 2/6] cfg80211: Add airtime statistics and settings Rajkumar Manoharan 2018-11-12 22:51 ` Rajkumar Manoharan 2018-11-12 22:51 ` [PATCH v3 3/6] mac80211: Add airtime accounting and scheduling to TXQs Rajkumar Manoharan 2018-11-12 22:51 ` Rajkumar Manoharan 2018-11-14 10:57 ` Felix Fietkau 2018-11-14 10:57 ` Felix Fietkau 2018-11-14 17:40 ` Toke Høiland-Jørgensen 2018-11-14 17:40 ` Toke Høiland-Jørgensen 2018-11-15 11:09 ` Felix Fietkau 2018-11-15 11:09 ` Felix Fietkau 2018-11-15 17:24 ` Toke Høiland-Jørgensen 2018-11-15 17:24 ` Toke Høiland-Jørgensen 2018-11-19 17:55 ` Dave Taht [this message] 2018-11-19 17:55 ` [Make-wifi-fast] " Dave Taht 2018-11-19 22:44 ` Toke Høiland-Jørgensen 2018-11-19 22:44 ` Toke Høiland-Jørgensen 2018-11-19 23:30 ` Dave Taht 2018-11-19 23:30 ` Dave Taht [not found] ` <4DD985B6-7DBE-42F8-AC87-D6B40CEAE553@superduper.net> 2018-11-19 23:47 ` Dave Taht 2018-11-19 23:47 ` Dave Taht 2018-11-19 23:56 ` Ben Greear 2018-11-19 23:56 ` Ben Greear 2018-11-20 0:13 ` Dave Taht 2018-11-20 0:13 ` Dave Taht 2018-11-20 0:20 ` Ben Greear 2018-11-20 0:20 ` Ben Greear 2018-11-20 0:37 ` Dave Taht 2018-11-20 0:37 ` Dave Taht 2018-11-20 2:12 ` David Lang 2018-11-20 2:12 ` David Lang [not found] ` <46F43681-DF84-4E08-9426-328BA7AE1CED@superduper.net> 2018-11-20 1:04 ` Dave Taht 2018-11-20 1:04 ` Dave Taht 2018-11-19 23:02 ` Toke Høiland-Jørgensen 2018-11-19 23:02 ` Toke Høiland-Jørgensen 2018-12-04 14:55 ` Toke Høiland-Jørgensen 2018-12-04 14:55 ` Toke Høiland-Jørgensen 2018-11-15 8:18 ` [Make-wifi-fast] " Louie Lu 2018-11-15 8:18 ` Louie Lu 2018-11-15 17:10 ` Toke Høiland-Jørgensen 2018-11-15 17:10 ` Toke Høiland-Jørgensen 2018-12-18 12:11 ` Johannes Berg 2018-12-18 12:11 ` Johannes Berg 2018-12-18 14:08 ` Dave Taht 2018-12-18 14:08 ` Dave Taht 2018-12-18 19:19 ` Toke Høiland-Jørgensen 2018-12-18 19:19 ` Toke Høiland-Jørgensen 2018-11-12 22:51 ` [PATCH v3 4/6] ath9k: Switch to mac80211 TXQ scheduling and airtime APIs Rajkumar Manoharan 2018-11-12 22:51 ` Rajkumar Manoharan 2018-11-12 22:51 ` [PATCH v3 5/6] ath10k: migrate to mac80211 txq scheduling Rajkumar Manoharan 2018-11-12 22:51 ` Rajkumar Manoharan 2018-11-12 22:51 ` [PATCH v3 6/6] ath10k: reporting estimated tx airtime for fairness Rajkumar Manoharan 2018-11-12 22:51 ` Rajkumar Manoharan
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=878t1p2bqz.fsf@taht.net \ --to=dave@taht.net \ --cc=ath10k@lists.infradead.org \ --cc=linux-wireless@vger.kernel.org \ --cc=make-wifi-fast@lists.bufferbloat.net \ --cc=nbd@nbd.name \ --cc=rmanohar@codeaurora.org \ --cc=toke@toke.dk \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.