archive mirror
 help / color / mirror / Atom feed
From: Doug McNaught <>
To: Matt Nelson <>
Subject: Re: Any limitations on bigmem usage?
Date: 11 Jun 2001 20:38:09 -0400	[thread overview]
Message-ID: <> (raw)
In-Reply-To: <> <>
In-Reply-To: Doug McNaught's message of "11 Jun 2001 20:31:23 -0400"

Doug McNaught <> writes:

> Matt Nelson <> writes:
> > I am about to embark on a data processing software project that
> > will require a LOT of memory (about, ohhh, 6GB or so), and I was
> > wondering if there are any limitations to how one can use very
> > large chunks of memory under Linux. Specifically, is there
> > anything to prevent me from malloc()ing 6GB of memory, then
> > accessing that memory as I would any other buffer?  FYI, the
> > application will be buffering a stream of data, then performing a
> > variety of calculations on large blocks of data at a time, before
> > writing it out to a socket.
> Pointers on IA32 are still 32 bits, so (as I understand it) each
> process can address a maximum of 4GB.  You would have to allocate
> multiple chunks (in shared memory or mmap()ed files, so they're
> persistent) and map them in and out as needed (which could get hairy).

Sorry to followup to myself, but I just had another thought--if you
can split your task up into multiple processe such that no process
needs to address more than 4GB, an IA32 machine will work fine.  It
will probably still take more programming work than the naive approach
(malloc() huge chink, use it) that you could use on a 64-bit

The rain man gave me two cures; he said jump right in,
The first was Texas medicine--the second was just railroad gin,
And like a fool I mixed them, and it strangled up my mind,
Now people just get uglier, and I got no sense of time...          --Dylan

  reply	other threads:[~2001-06-12  0:38 UTC|newest]

Thread overview: 9+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2001-06-12  0:18 Matt Nelson
2001-06-12  0:31 ` Doug McNaught
2001-06-12  0:38   ` Doug McNaught [this message]
2001-06-12 14:03 ` Matt Nelson
2001-06-12 16:29 ` Alan Cox
2001-06-12 12:46   ` Rob Landley
2001-06-12 18:16     ` Rik van Riel
2001-06-12 18:34 Holzrichter, Bruce
2001-06-13 10:39 ` Ralf Baechle

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \ \ \ \ \
    --subject='Re: Any limitations on bigmem usage?' \

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).