From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751337AbcGMNsZ (ORCPT ); Wed, 13 Jul 2016 09:48:25 -0400 Received: from mail-wm0-f50.google.com ([74.125.82.50]:36443 "EHLO mail-wm0-f50.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751083AbcGMNsT (ORCPT ); Wed, 13 Jul 2016 09:48:19 -0400 Date: Wed, 13 Jul 2016 15:47:17 +0200 From: Michal Hocko To: Matthias Dahl Cc: linux-raid@vger.kernel.org, linux-mm@kvack.org, dm-devel@redhat.com, linux-kernel@vger.kernel.org, Mike Snitzer Subject: Re: Page Allocation Failures/OOM with dm-crypt on software RAID10 (Intel Rapid Storage) Message-ID: <20160713134717.GL28723@dhcp22.suse.cz> References: <02580b0a303da26b669b4a9892624b13@mail.ud19.udmedia.de> <20160712095013.GA14591@dhcp22.suse.cz> <20160712114920.GF14586@dhcp22.suse.cz> <20160712140715.GL14586@dhcp22.suse.cz> <459d501038de4d25db6d140ac5ea5f8d@mail.ud19.udmedia.de> <20160713112126.GH28723@dhcp22.suse.cz> <20160713121828.GI28723@dhcp22.suse.cz> <74b9325c37948cf2b460bd759cff23dd@mail.ud19.udmedia.de> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <74b9325c37948cf2b460bd759cff23dd@mail.ud19.udmedia.de> User-Agent: Mutt/1.6.0 (2016-04-01) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Wed 13-07-16 15:18:11, Matthias Dahl wrote: > Hello Michal, > > many thanks for all your time and help on this issue. It is very much > appreciated and I hope we can track this down somehow. > > On 2016-07-13 14:18, Michal Hocko wrote: > > > So it seems we are accumulating bios and 256B objects. Buffer heads as > > well but so much. Having over 4G worth of bios sounds really suspicious. > > Note that they pin pages to be written so this might be consuming the > > rest of the unaccounted memory! So the main question is why those bios > > do not get dispatched or finished. > > Ok. It is the Block IOs that do not get completed. I do get it right > that those bio-3 are already the encrypted data that should be written > out but do not for some reason? Hard to tell. Maybe they are just allocated and waiting for encryption. But this is just a wild guessing. > I tried to figure this out myself but > couldn't find anything -- what does the number "-3" state? It is the > position in some chain or has it a different meaning? $ git grep "kmem_cache_create.*bio" block/bio-integrity.c: bip_slab = kmem_cache_create("bio_integrity_payload", so there doesn't seem to be any cache like that in the vanilla kernel. > Do you think a trace like you mentioned would help shed some more light > on this? Or would you recommend something else? Dunno. Seeing who is allocating those bios might be helpful but it won't tell much about what has happened to them after allocation. The tracing would be more helpful for a mem leak situation which doesn't seem to be the case here. This is getting out of my area of expertise so I am not sure I can help you much more, I am afraid. -- Michal Hocko SUSE Labs