From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932921AbcLICA2 (ORCPT ); Thu, 8 Dec 2016 21:00:28 -0500 Received: from zeniv.linux.org.uk ([195.92.253.2]:33568 "EHLO ZenIV.linux.org.uk" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754037AbcLICA0 (ORCPT ); Thu, 8 Dec 2016 21:00:26 -0500 Date: Fri, 9 Dec 2016 02:00:17 +0000 From: Al Viro To: Dave Chinner Cc: Michal Hocko , linux-mm@kvack.org, Andrew Morton , Vlastimil Babka , David Rientjes , Mel Gorman , Johannes Weiner , Anatoly Stepanov , LKML , Michal Hocko , Paolo Bonzini , Mike Snitzer , dm-devel@redhat.com, "Michael S. Tsirkin" , "Theodore Ts'o" , kvm@vger.kernel.org, linux-ext4@vger.kernel.org, linux-f2fs-devel@lists.sourceforge.net, linux-security-module@vger.kernel.org Subject: Re: [RFC PATCH] mm: introduce kv[mz]alloc helpers Message-ID: <20161209020016.GX1555@ZenIV.linux.org.uk> References: <20161208103300.23217-1-mhocko@kernel.org> <20161209014417.GN4326@dastard> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20161209014417.GN4326@dastard> User-Agent: Mutt/1.7.1 (2016-10-04) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Fri, Dec 09, 2016 at 12:44:17PM +1100, Dave Chinner wrote: > On Thu, Dec 08, 2016 at 11:33:00AM +0100, Michal Hocko wrote: > > From: Michal Hocko > > > > Using kmalloc with the vmalloc fallback for larger allocations is a > > common pattern in the kernel code. Yet we do not have any common helper > > for that and so users have invented their own helpers. Some of them are > > really creative when doing so. Let's just add kv[mz]alloc and make sure > > it is implemented properly. This implementation makes sure to not make > > a large memory pressure for > PAGE_SZE requests (__GFP_NORETRY) and also > > to not warn about allocation failures. This also rules out the OOM > > killer as the vmalloc is a more approapriate fallback than a disruptive > > user visible action. > > > > This patch also changes some existing users and removes helpers which > > are specific for them. In some cases this is not possible (e.g. > > ext4_kvmalloc, libcfs_kvzalloc, __aa_kvmalloc) because those seems to be > > broken and require GFP_NO{FS,IO} context which is not vmalloc compatible > > in general (note that the page table allocation is GFP_KERNEL). Those > > need to be fixed separately. > > See fs/xfs/kmem.c::kmem_zalloc_large(), which is XFS's version of > kvmalloc() that is GFP_NOFS/GFP_NOIO safe. Any generic API for this > functionality will have to play these memalloc_noio_save/ > memalloc_noio_restore games to ensure they are GFP_NOFS safe.... Easier to handle those in vmalloc() itself. The problem I have with these helpers is that different places have different cutoff thresholds for switch from kmalloc to vmalloc; has anyone done an analysis of those? From mboxrd@z Thu Jan 1 00:00:00 1970 From: Al Viro Subject: Re: [RFC PATCH] mm: introduce kv[mz]alloc helpers Date: Fri, 9 Dec 2016 02:00:17 +0000 Message-ID: <20161209020016.GX1555@ZenIV.linux.org.uk> References: <20161208103300.23217-1-mhocko@kernel.org> <20161209014417.GN4326@dastard> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: Michal Hocko , linux-mm@kvack.org, Andrew Morton , Vlastimil Babka , David Rientjes , Mel Gorman , Johannes Weiner , Anatoly Stepanov , LKML , Michal Hocko , Paolo Bonzini , Mike Snitzer , dm-devel@redhat.com, "Michael S. Tsirkin" , Theodore Ts'o , kvm@vger.kernel.org, linux-ext4@vger.kernel.org, linux-f2fs-devel@lists.sourceforge.net, linux-security-module@vger.kernel.org To: Dave Chinner Return-path: Content-Disposition: inline In-Reply-To: <20161209014417.GN4326@dastard> Sender: owner-linux-mm@kvack.org List-Id: linux-ext4.vger.kernel.org On Fri, Dec 09, 2016 at 12:44:17PM +1100, Dave Chinner wrote: > On Thu, Dec 08, 2016 at 11:33:00AM +0100, Michal Hocko wrote: > > From: Michal Hocko > > > > Using kmalloc with the vmalloc fallback for larger allocations is a > > common pattern in the kernel code. Yet we do not have any common helper > > for that and so users have invented their own helpers. Some of them are > > really creative when doing so. Let's just add kv[mz]alloc and make sure > > it is implemented properly. This implementation makes sure to not make > > a large memory pressure for > PAGE_SZE requests (__GFP_NORETRY) and also > > to not warn about allocation failures. This also rules out the OOM > > killer as the vmalloc is a more approapriate fallback than a disruptive > > user visible action. > > > > This patch also changes some existing users and removes helpers which > > are specific for them. In some cases this is not possible (e.g. > > ext4_kvmalloc, libcfs_kvzalloc, __aa_kvmalloc) because those seems to be > > broken and require GFP_NO{FS,IO} context which is not vmalloc compatible > > in general (note that the page table allocation is GFP_KERNEL). Those > > need to be fixed separately. > > See fs/xfs/kmem.c::kmem_zalloc_large(), which is XFS's version of > kvmalloc() that is GFP_NOFS/GFP_NOIO safe. Any generic API for this > functionality will have to play these memalloc_noio_save/ > memalloc_noio_restore games to ensure they are GFP_NOFS safe.... Easier to handle those in vmalloc() itself. The problem I have with these helpers is that different places have different cutoff thresholds for switch from kmalloc to vmalloc; has anyone done an analysis of those? -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org