From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753643AbbCPKth (ORCPT ); Mon, 16 Mar 2015 06:49:37 -0400 Received: from mail-wg0-f44.google.com ([74.125.82.44]:33083 "EHLO mail-wg0-f44.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753310AbbCPKte convert rfc822-to-8bit (ORCPT ); Mon, 16 Mar 2015 06:49:34 -0400 MIME-Version: 1.0 In-Reply-To: <5506B04D.1070506@lge.com> References: <1426248777-19768-1-git-send-email-r.peniaev@gmail.com> <5506B04D.1070506@lge.com> Date: Mon, 16 Mar 2015 19:49:32 +0900 Message-ID: Subject: Re: [PATCH 0/3] [RFC] mm/vmalloc: fix possible exhaustion of vmalloc space From: Roman Peniaev To: Gioh Kim Cc: Andrew Morton , Eric Dumazet , Joonsoo Kim , David Rientjes , WANG Chao , Fabian Frederick , Christoph Lameter , Rob Jones , linux-mm@kvack.org, "linux-kernel@vger.kernel.org" , "stable@vger.kernel.org" Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8BIT Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Mon, Mar 16, 2015 at 7:28 PM, Gioh Kim wrote: > > > 2015-03-13 오후 9:12에 Roman Pen 이(가) 쓴 글: >> Hello all. >> >> Recently I came across high fragmentation of vm_map_ram allocator: vmap_block >> has free space, but still new blocks continue to appear. Further investigation >> showed that certain mapping/unmapping sequence can exhaust vmalloc space. On >> small 32bit systems that's not a big problem, cause purging will be called soon >> on a first allocation failure (alloc_vmap_area), but on 64bit machines, e.g. >> x86_64 has 45 bits of vmalloc space, that can be a disaster. > > I think the problem you comments is already known so that I wrote comments about it as > "it could consume lots of address space through fragmentation". > > Could you tell me about your situation and reason why it should be avoided? In the first patch of this set I explicitly described the function, which exhausts vmalloc space without any chance to be purged: vm_map_ram allocator is greedy and firstly tries to occupy newly allocated block, even old blocks contain enough free space. This can be easily fixed if we put newly allocated block (which has enough space to complete further requests) to the tail of a free list, to give a chance to old blocks. Why it should be avoided? Strange question. For me it looks like a bug of an allocator, which should be fair and should not continuously allocate new blocks without lazy purging (seems vmap_lazy_nr and __purge_vmap_area_lazy were created exactly for those reasons: to avoid infinite allocations) -- Roman > > >> >> Fixing this I also did some tweaks in allocation logic of a new vmap block and >> replaced dirty bitmap with min/max dirty range values to make the logic simpler. >> >> I would like to receive comments on the following three patches. >> >> Thanks. >> >> Roman Pen (3): >> mm/vmalloc: fix possible exhaustion of vmalloc space caused by >> vm_map_ram allocator >> mm/vmalloc: occupy newly allocated vmap block just after allocation >> mm/vmalloc: get rid of dirty bitmap inside vmap_block structure >> >> mm/vmalloc.c | 94 ++++++++++++++++++++++++++++++++++-------------------------- >> 1 file changed, 54 insertions(+), 40 deletions(-) >> >> Cc: Andrew Morton >> Cc: Nick Piggin >> Cc: Eric Dumazet >> Cc: Joonsoo Kim >> Cc: David Rientjes >> Cc: WANG Chao >> Cc: Fabian Frederick >> Cc: Christoph Lameter >> Cc: Gioh Kim >> Cc: Rob Jones >> Cc: linux-mm@kvack.org >> Cc: linux-kernel@vger.kernel.org >> Cc: stable@vger.kernel.org >> From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: MIME-Version: 1.0 In-Reply-To: <5506B04D.1070506@lge.com> References: <1426248777-19768-1-git-send-email-r.peniaev@gmail.com> <5506B04D.1070506@lge.com> Date: Mon, 16 Mar 2015 19:49:32 +0900 Message-ID: Subject: Re: [PATCH 0/3] [RFC] mm/vmalloc: fix possible exhaustion of vmalloc space From: Roman Peniaev To: Gioh Kim Cc: Andrew Morton , Eric Dumazet , Joonsoo Kim , David Rientjes , WANG Chao , Fabian Frederick , Christoph Lameter , Rob Jones , linux-mm@kvack.org, "linux-kernel@vger.kernel.org" , "stable@vger.kernel.org" Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Sender: owner-linux-mm@kvack.org List-ID: On Mon, Mar 16, 2015 at 7:28 PM, Gioh Kim wrote: > > > 2015-03-13 =EC=98=A4=ED=9B=84 9:12=EC=97=90 Roman Pen =EC=9D=B4(=EA=B0=80= ) =EC=93=B4 =EA=B8=80: >> Hello all. >> >> Recently I came across high fragmentation of vm_map_ram allocator: vmap_= block >> has free space, but still new blocks continue to appear. Further invest= igation >> showed that certain mapping/unmapping sequence can exhaust vmalloc space= . On >> small 32bit systems that's not a big problem, cause purging will be call= ed soon >> on a first allocation failure (alloc_vmap_area), but on 64bit machines, = e.g. >> x86_64 has 45 bits of vmalloc space, that can be a disaster. > > I think the problem you comments is already known so that I wrote comment= s about it as > "it could consume lots of address space through fragmentation". > > Could you tell me about your situation and reason why it should be avoide= d? In the first patch of this set I explicitly described the function, which exhausts vmalloc space without any chance to be purged: vm_map_ram allocator is greedy and firstly tries to occupy newly allocated block, even old blocks contain enough free space. This can be easily fixed if we put newly allocated block (which has enough space to complete further requests) to the tail of a free list, to give a chance to old blocks. Why it should be avoided? Strange question. For me it looks like a bug of an allocator, which should be fair and should not continuously allocate new blocks without lazy purging (seems vmap_lazy_nr and __purge_vmap_area_lazy were created exactly for those reasons: to avoid infinite allocations) -- Roman > > >> >> Fixing this I also did some tweaks in allocation logic of a new vmap blo= ck and >> replaced dirty bitmap with min/max dirty range values to make the logic = simpler. >> >> I would like to receive comments on the following three patches. >> >> Thanks. >> >> Roman Pen (3): >> mm/vmalloc: fix possible exhaustion of vmalloc space caused by >> vm_map_ram allocator >> mm/vmalloc: occupy newly allocated vmap block just after allocation >> mm/vmalloc: get rid of dirty bitmap inside vmap_block structure >> >> mm/vmalloc.c | 94 ++++++++++++++++++++++++++++++++++------------------= -------- >> 1 file changed, 54 insertions(+), 40 deletions(-) >> >> Cc: Andrew Morton >> Cc: Nick Piggin >> Cc: Eric Dumazet >> Cc: Joonsoo Kim >> Cc: David Rientjes >> Cc: WANG Chao >> Cc: Fabian Frederick >> Cc: Christoph Lameter >> Cc: Gioh Kim >> Cc: Rob Jones >> Cc: linux-mm@kvack.org >> Cc: linux-kernel@vger.kernel.org >> Cc: stable@vger.kernel.org >> -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org