Re: Any limitations on bigmem usage?

[Doug McNaught <doug@wireboard.com>]

Matt Nelson <mnelson@dynatec.com> writes:

> I am about to embark on a data processing software project that will require a
> LOT of memory (about, ohhh, 6GB or so), and I was wondering if there are any
> limitations to how one can use very large chunks of memory under
> Linux. Specifically, is there anything to prevent me from malloc()ing 6GB of
> memory, then accessing that memory as I would any other buffer?  FYI, the
> application
> 
> will be buffering a stream of data, then performing a variety of calculations
> on large blocks of data at a time, before writing it out to a socket.

Pointers on IA32 are still 32 bits, so (as I understand it) each
process can address a maximum of 4GB.  You would have to allocate
multiple chunks (in shared memory or mmap()ed files, so they're
persistent) and map them in and out as needed (which could get hairy).

> I've been eyeing an 8-way Intel box with gobs of memory, but if there are
> subtle issues with using that much memory, I need to know now.

Best bet is probably to buy an Alpha machine, if you don't want to
spend time working around the 4GB limitation.

-Doug
-- 
The rain man gave me two cures; he said jump right in,
The first was Texas medicine--the second was just railroad gin,
And like a fool I mixed them, and it strangled up my mind,
Now people just get uglier, and I got no sense of time...          --Dylan