From mboxrd@z Thu Jan 1 00:00:00 1970 Return-path: Received: from mail-bk0-f46.google.com ([209.85.214.46]:59191 "EHLO mail-bk0-f46.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751896Ab3H1NQa convert rfc822-to-8bit (ORCPT ); Wed, 28 Aug 2013 09:16:30 -0400 Received: by mail-bk0-f46.google.com with SMTP id 6so2127551bkj.5 for ; Wed, 28 Aug 2013 06:16:29 -0700 (PDT) MIME-Version: 1.0 In-Reply-To: <877gf6lay7.fsf@kamboji.qca.qualcomm.com> References: <1377066854-13981-1-git-send-email-michal.kazior@tieto.com> <1377507205-5386-1-git-send-email-michal.kazior@tieto.com> <1377507205-5386-5-git-send-email-michal.kazior@tieto.com> <87ioyrmx3u.fsf@kamboji.qca.qualcomm.com> <877gf6lay7.fsf@kamboji.qca.qualcomm.com> Date: Wed, 28 Aug 2013 15:16:29 +0200 Message-ID: (sfid-20130828_151634_173095_7C519CF6) Subject: Re: [PATCH v2 4/4] ath10k: fix issues on non-preemptible systems From: Michal Kazior To: Kalle Valo Cc: ath10k@lists.infradead.org, linux-wireless Content-Type: text/plain; charset=UTF-8 Sender: linux-wireless-owner@vger.kernel.org List-ID: On 28 August 2013 06:02, Kalle Valo wrote: > Michal Kazior writes: >> There's another solution that I had in mind. Instead of: >> >> for (;;) { dequeue(z); process; } >> >> I did: >> >> q = dequeue_all(z); for (;;) { dequeue(q); process; } >> >> I.e. move all queued stuff at the worker entry and move it out of the >> global queue, that can, and will be, having more stuff queued while >> the worker does its job). >> >> This way workers would exit/restart more often, but from what I tested >> it didn't really solve the problem. Given enough traffic HTC worker >> responsible for HTT TX gets overwhelmed eventually. You could try >> limit how many frames a worker can process during one execution, but >> how do you guess that? This starvation depends on how fast your CPU >> is. > > I think we should come up with better ways to handle this. To have a > quota (like you mention above) would be one option. Other option would > be to have multiple queues and some sort of priorisation or fair > queueing. Having quota will not help here in any way. You can re-queue a worker after each single frame and avoid WMI starvation, however you can still starve the rest of the system (and that can lead to system reset via watchdog). I'm also unsure about the overhead queueing a work may have (on a uP system without preemption in might be negligible, but what about other systems?), so you'd have to guess the quota size or else you'd get increased latency/overhead and perhaps slower performance. I believe cond_resched is a solution, not a workaround. Slow systems without preemption need this. I wonder how other drivers got around it? Or perhaps none of the other drivers had to deal with really insufficient number of CPU cycles versus lots of work. We could perhaps move workers out of HTC and have a single TX worker in core.c for both WMI and HTT that would prioritize WMI, before trying HTT. This could help guarantee that all beacons (which go through WMI) are sent in a timely fashion in response to SWBA event. But that won't fix the overall system starvation. > And most importantly of all, we should minimise the lenght of queue we > have inside ath10k. I'm worried that we queue way too many packets > within ath10k right now. Felix pointed that out quite some time ago. I would agree but I'm affraid you'll hurt performance if you decrease the queue depth. There seems to be some kind of latency thing going on (either on host, or on firmware/hardware, or both combined). I tried decreasing HTT TX ring buffer from 2048 to 256. In result on AP135 UDP TX got trimmed at ~330mbps max. Stuffing more throughput even left some idle CPU cycles. If you consider 3x3 devices that are supposed to get you 1.3gbps, then you apparently need that 2048 depth. Michał. From mboxrd@z Thu Jan 1 00:00:00 1970 Return-path: Received: from mail-bk0-x235.google.com ([2a00:1450:4008:c01::235]) by merlin.infradead.org with esmtps (Exim 4.80.1 #2 (Red Hat Linux)) id 1VEfcD-00055y-Nv for ath10k@lists.infradead.org; Wed, 28 Aug 2013 13:16:54 +0000 Received: by mail-bk0-f53.google.com with SMTP id d7so2125286bkh.40 for ; Wed, 28 Aug 2013 06:16:29 -0700 (PDT) MIME-Version: 1.0 In-Reply-To: <877gf6lay7.fsf@kamboji.qca.qualcomm.com> References: <1377066854-13981-1-git-send-email-michal.kazior@tieto.com> <1377507205-5386-1-git-send-email-michal.kazior@tieto.com> <1377507205-5386-5-git-send-email-michal.kazior@tieto.com> <87ioyrmx3u.fsf@kamboji.qca.qualcomm.com> <877gf6lay7.fsf@kamboji.qca.qualcomm.com> Date: Wed, 28 Aug 2013 15:16:29 +0200 Message-ID: Subject: Re: [PATCH v2 4/4] ath10k: fix issues on non-preemptible systems From: Michal Kazior List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Sender: "ath10k" Errors-To: ath10k-bounces+kvalo=adurom.com@lists.infradead.org To: Kalle Valo Cc: linux-wireless , ath10k@lists.infradead.org T24gMjggQXVndXN0IDIwMTMgMDY6MDIsIEthbGxlIFZhbG8gPGt2YWxvQHFjYS5xdWFsY29tbS5j b20+IHdyb3RlOgo+IE1pY2hhbCBLYXppb3IgPG1pY2hhbC5rYXppb3JAdGlldG8uY29tPiB3cml0 ZXM6Cj4+IFRoZXJlJ3MgYW5vdGhlciBzb2x1dGlvbiB0aGF0IEkgaGFkIGluIG1pbmQuIEluc3Rl YWQgb2Y6Cj4+Cj4+ICAgZm9yICg7OykgeyBkZXF1ZXVlKHopOyBwcm9jZXNzOyB9Cj4+Cj4+IEkg ZGlkOgo+Pgo+PiAgIHEgPSBkZXF1ZXVlX2FsbCh6KTsgZm9yICg7OykgeyBkZXF1ZXVlKHEpOyBw cm9jZXNzOyB9Cj4+Cj4+IEkuZS4gbW92ZSBhbGwgcXVldWVkIHN0dWZmIGF0IHRoZSB3b3JrZXIg ZW50cnkgYW5kIG1vdmUgaXQgb3V0IG9mIHRoZQo+PiBnbG9iYWwgcXVldWUsIHRoYXQgY2FuLCBh bmQgd2lsbCBiZSwgaGF2aW5nIG1vcmUgc3R1ZmYgcXVldWVkIHdoaWxlCj4+IHRoZSB3b3JrZXIg ZG9lcyBpdHMgam9iKS4KPj4KPj4gVGhpcyB3YXkgd29ya2VycyB3b3VsZCBleGl0L3Jlc3RhcnQg bW9yZSBvZnRlbiwgYnV0IGZyb20gd2hhdCBJIHRlc3RlZAo+PiBpdCBkaWRuJ3QgcmVhbGx5IHNv bHZlIHRoZSBwcm9ibGVtLiBHaXZlbiBlbm91Z2ggdHJhZmZpYyBIVEMgd29ya2VyCj4+IHJlc3Bv bnNpYmxlIGZvciBIVFQgVFggZ2V0cyBvdmVyd2hlbG1lZCBldmVudHVhbGx5LiBZb3UgY291bGQg dHJ5Cj4+IGxpbWl0IGhvdyBtYW55IGZyYW1lcyBhIHdvcmtlciBjYW4gcHJvY2VzcyBkdXJpbmcg b25lIGV4ZWN1dGlvbiwgYnV0Cj4+IGhvdyBkbyB5b3UgZ3Vlc3MgdGhhdD8gVGhpcyBzdGFydmF0 aW9uIGRlcGVuZHMgb24gaG93IGZhc3QgeW91ciBDUFUKPj4gaXMuCj4KPiBJIHRoaW5rIHdlIHNo b3VsZCBjb21lIHVwIHdpdGggYmV0dGVyIHdheXMgdG8gaGFuZGxlIHRoaXMuIFRvIGhhdmUgYQo+ IHF1b3RhIChsaWtlIHlvdSBtZW50aW9uIGFib3ZlKSB3b3VsZCBiZSBvbmUgb3B0aW9uLiBPdGhl ciBvcHRpb24gd291bGQKPiBiZSB0byBoYXZlIG11bHRpcGxlIHF1ZXVlcyBhbmQgc29tZSBzb3J0 IG9mIHByaW9yaXNhdGlvbiBvciBmYWlyCj4gcXVldWVpbmcuCgpIYXZpbmcgcXVvdGEgd2lsbCBu b3QgaGVscCBoZXJlIGluIGFueSB3YXkuIFlvdSBjYW4gcmUtcXVldWUgYSB3b3JrZXIKYWZ0ZXIg ZWFjaCBzaW5nbGUgZnJhbWUgYW5kIGF2b2lkIFdNSSBzdGFydmF0aW9uLCBob3dldmVyIHlvdSBj YW4Kc3RpbGwgc3RhcnZlIHRoZSByZXN0IG9mIHRoZSBzeXN0ZW0gKGFuZCB0aGF0IGNhbiBsZWFk IHRvIHN5c3RlbSByZXNldAp2aWEgd2F0Y2hkb2cpLiBJJ20gYWxzbyB1bnN1cmUgYWJvdXQgdGhl IG92ZXJoZWFkIHF1ZXVlaW5nIGEgd29yayBtYXkKaGF2ZSAob24gYSB1UCBzeXN0ZW0gd2l0aG91 dCBwcmVlbXB0aW9uIGluIG1pZ2h0IGJlIG5lZ2xpZ2libGUsIGJ1dAp3aGF0IGFib3V0IG90aGVy IHN5c3RlbXM/KSwgc28geW91J2QgaGF2ZSB0byBndWVzcyB0aGUgcXVvdGEgc2l6ZSBvcgplbHNl IHlvdSdkIGdldCBpbmNyZWFzZWQgbGF0ZW5jeS9vdmVyaGVhZCBhbmQgcGVyaGFwcyBzbG93ZXIK cGVyZm9ybWFuY2UuCgpJIGJlbGlldmUgY29uZF9yZXNjaGVkIGlzIGEgc29sdXRpb24sIG5vdCBh IHdvcmthcm91bmQuIFNsb3cgc3lzdGVtcwp3aXRob3V0IHByZWVtcHRpb24gbmVlZCB0aGlzLiBJ IHdvbmRlciBob3cgb3RoZXIgZHJpdmVycyBnb3QgYXJvdW5kCml0PyBPciBwZXJoYXBzIG5vbmUg b2YgdGhlIG90aGVyIGRyaXZlcnMgaGFkIHRvIGRlYWwgd2l0aCByZWFsbHkKaW5zdWZmaWNpZW50 IG51bWJlciBvZiBDUFUgY3ljbGVzIHZlcnN1cyBsb3RzIG9mIHdvcmsuCgpXZSBjb3VsZCBwZXJo YXBzIG1vdmUgd29ya2VycyBvdXQgb2YgSFRDIGFuZCBoYXZlIGEgc2luZ2xlIFRYIHdvcmtlcgpp biBjb3JlLmMgZm9yIGJvdGggV01JIGFuZCBIVFQgdGhhdCB3b3VsZCBwcmlvcml0aXplIFdNSSwg YmVmb3JlCnRyeWluZyBIVFQuIFRoaXMgY291bGQgaGVscCBndWFyYW50ZWUgdGhhdCBhbGwgYmVh Y29ucyAod2hpY2ggZ28KdGhyb3VnaCBXTUkpIGFyZSBzZW50IGluIGEgdGltZWx5IGZhc2hpb24g aW4gcmVzcG9uc2UgdG8gU1dCQSBldmVudC4KQnV0IHRoYXQgd29uJ3QgZml4IHRoZSBvdmVyYWxs IHN5c3RlbSBzdGFydmF0aW9uLgoKCj4gQW5kIG1vc3QgaW1wb3J0YW50bHkgb2YgYWxsLCB3ZSBz aG91bGQgbWluaW1pc2UgdGhlIGxlbmdodCBvZiBxdWV1ZSB3ZQo+IGhhdmUgaW5zaWRlIGF0aDEw ay4gSSdtIHdvcnJpZWQgdGhhdCB3ZSBxdWV1ZSB3YXkgdG9vIG1hbnkgcGFja2V0cwo+IHdpdGhp biBhdGgxMGsgcmlnaHQgbm93LgoKRmVsaXggcG9pbnRlZCB0aGF0IG91dCBxdWl0ZSBzb21lIHRp bWUgYWdvLiBJIHdvdWxkIGFncmVlIGJ1dCBJJ20KYWZmcmFpZCB5b3UnbGwgaHVydCBwZXJmb3Jt YW5jZSBpZiB5b3UgZGVjcmVhc2UgdGhlIHF1ZXVlIGRlcHRoLiBUaGVyZQpzZWVtcyB0byBiZSBz b21lIGtpbmQgb2YgbGF0ZW5jeSB0aGluZyBnb2luZyBvbiAoZWl0aGVyIG9uIGhvc3QsIG9yIG9u CmZpcm13YXJlL2hhcmR3YXJlLCBvciBib3RoIGNvbWJpbmVkKS4gSSB0cmllZCBkZWNyZWFzaW5n IEhUVCBUWCByaW5nCmJ1ZmZlciBmcm9tIDIwNDggdG8gMjU2LiBJbiByZXN1bHQgb24gQVAxMzUg VURQIFRYIGdvdCB0cmltbWVkIGF0Cn4zMzBtYnBzIG1heC4gU3R1ZmZpbmcgbW9yZSB0aHJvdWdo cHV0IGV2ZW4gbGVmdCBzb21lIGlkbGUgQ1BVIGN5Y2xlcy4KSWYgeW91IGNvbnNpZGVyIDN4MyBk ZXZpY2VzIHRoYXQgYXJlIHN1cHBvc2VkIHRvIGdldCB5b3UgMS4zZ2JwcywgdGhlbgp5b3UgYXBw YXJlbnRseSBuZWVkIHRoYXQgMjA0OCBkZXB0aC4KCgpNaWNoYcWCLgoKX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KYXRoMTBrIG1haWxpbmcgbGlzdAphdGgx MGtAbGlzdHMuaW5mcmFkZWFkLm9yZwpodHRwOi8vbGlzdHMuaW5mcmFkZWFkLm9yZy9tYWlsbWFu L2xpc3RpbmZvL2F0aDEwawo=