kernel hangs in 118th call to vmalloc

kernel hangs in 118th call to vmalloc

[Timur Tabi]

I'm writing a driver for the 2.4.2 kernel.  I need to use this kernel 
because this driver needs to be compatible with a stock Red Hat system. 
  Patches to the kernel are not an option.

The purpose of the driver is to locate a device that exists on a 
specific memory chip.  To help find it, I've written this routine:

#define CLEAR_BLOCK_SIZE 1048576UL        // must be a multiple of 1MB
#define CLEAR_BLOCK_COUNT ((PHYSICAL_HOP * 2) / CLEAR_BLOCK_SIZE)

void clear_out_memory(void)
{
     void *p[CLEAR_BLOCK_COUNT];
     unsigned i;
     unsigned long size = 0;

     for (i=0; i<CLEAR_BLOCK_COUNT; i++)
     {
         p[i] = vmalloc(CLEAR_BLOCK_SIZE);
         if (!p[i])
             break;
         size += CLEAR_BLOCK_SIZE;
     }

     while (--i)
         vfree(p[i]);

     printk("Paged %luMB of memory\n", size / 1048576UL);
}

What this routine does is call vmalloc() repeatedly for a number of 1MB 
chunks until it fails or until it's allocated 128MB (CLEAR_BLOCK_COUNT 
is equal to 128 in this case).  Then, it starts freeing them.

The side-effect of this routine is to page-out up to 128MB of RAM. 
Unfortunately, on a 128MB machine, the 118th call to vmalloc() hangs the 
system.  I was expecting it to return NULL instead.

Is this a bug in vmalloc()?  If so, is there a work-around that I can use?

Re: kernel hangs in 118th call to vmalloc

[Alan Cox]

> What this routine does is call vmalloc() repeatedly for a number of 1MB 
> chunks until it fails or until it's allocated 128MB (CLEAR_BLOCK_COUNT 
> is equal to 128 in this case).  Then, it starts freeing them.
> 
> The side-effect of this routine is to page-out up to 128MB of RAM. 
> Unfortunately, on a 128MB machine, the 118th call to vmalloc() hangs the 
> system.  I was expecting it to return NULL instead.
> 
> Is this a bug in vmalloc()?  If so, is there a work-around that I can use?

vmalloc shouldnt be hanging the box, although in 2.4.2 the out of memory 
handling is not too reliable. You have to understand vmalloc isnt meant to 
be used that way and the kernel gets priority over user space for allocs so
is able to get itself to the point it killed off all user space.

Alan

Re: kernel hangs in 118th call to vmalloc

[Timur Tabi]

Alan Cox wrote:
> vmalloc shouldnt be hanging the box, although in 2.4.2 the out of memory 
> handling is not too reliable. You have to understand vmalloc isnt meant to 
> be used that way and the kernel gets priority over user space for allocs so
> is able to get itself to the point it killed off all user space.

So you're saying it's a bug that I can't work around?

It's probably a moot point.  I've come up with a different algorithm 
that allocates all but 32MB of RAM, and it appears to work well.

I heard that 2.4.9 doesn't even run "thrash".  Is this true?  If so, why 
are these buggy VM's being released in the first place?

Re: kernel hangs in 118th call to vmalloc

[Alan Cox]

> > is able to get itself to the point it killed off all user space.
> 
> So you're saying it's a bug that I can't work around?

Its a design property vmalloc never had. It isnt meant to be abused that
way

> I heard that 2.4.9 doesn't even run "thrash".  Is this true?  If so, why 
> are these buggy VM's being released in the first place?

Lack of crystal balls.

The VM seemed ok until after 2.4.0 at which point it became clear it had
some rather large corner cases where it was not. Fixing them is an
iterative process and getting all cases right is hard. I'm very happy with
the 2.4.9-ac tree VM. It needs the inode cache handling resolved better
but it works

Re: kernel hangs in 118th call to vmalloc

[Eric W. Biederman]

Timur Tabi <ttabi@interactivesi.com> writes:

> I'm writing a driver for the 2.4.2 kernel.  I need to use this kernel because
> this driver needs to be compatible with a stock Red Hat system. Patches to the
> kernel are not an option.
> 
> The purpose of the driver is to locate a device that exists on a specific memory
> 
> chip.  To help find it, I've written this routine:

What is wrong with using SPD to detect interesting properties of
memory chips?  That should be safer and usually easier then what you
are trying now. 

Eric

Re: kernel hangs in 118th call to vmalloc

[Timur Tabi]

Eric W. Biederman wrote:

> What is wrong with using SPD to detect interesting properties of
> memory chips?  That should be safer and usually easier then what you
> are trying now. 

Our hardware does not interface with SPD.  So I can't use SPD to query the 
properties.  Besides, it wouldn't change anything if I did.  I still need to 
clear out RAM.