Expand VM

Expand VM

[User &]

Hi all

I have one idea , and this is about expand virtual memory on linux boxes 
connected in LAN.
Example: Linux A is processing come information , and need more memory , so 
with this source , Linux A could access virtual memory on Linux B in LAN.
But i don´t know how translate the virtual address between Linux A and B , to 
have success in acess VM, or how to send all the process for Linux B to be 
processed.

Any ideas ?

Thanks
Breno

----------------------
WebMail Bandnet.com.br

Re: Expand VM

[Richard B. Johnson]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: TEXT/PLAIN; charset=US-ASCII, Size: 797 bytes --]

On Thu, 23 Jan 2003, User & wrote:

> 
> Hi all
> 
> I have one idea , and this is about expand virtual memory on linux boxes 
> connected in LAN.
> Example: Linux A is processing come information , and need more memory , so 
> with this source , Linux A could access virtual memory on Linux B in LAN.
> But i don´t know how translate the virtual address between Linux A and B , to 
> have success in acess VM, or how to send all the process for Linux B to be 
> processed.
> 
> Any ideas ?
> 
> Thanks
> Breno

Use a swap-file on another machine on the LAN to extend your virtual
memory if you run out of local swap-file space.


Cheers,
Dick Johnson
Penguin : Linux version 2.4.18 on an i686 machine (797.90 BogoMips).
Why is the government concerned about the lunatic fringe? Think about it.

Re: Expand VM

[Patrizio Bruno]

http://www.openmosix.org

On Thu, 2003-01-23 at 16:56, User & wrote:
> Hi all
> 
> I have one idea , and this is about expand virtual memory on linux boxes 
> connected in LAN.
> Example: Linux A is processing come information , and need more memory , so 
> with this source , Linux A could access virtual memory on Linux B in LAN.
> But i don´t know how translate the virtual address between Linux A and B , to 
> have success in acess VM, or how to send all the process for Linux B to be 
> processed.
> 
> Any ideas ?
> 
> Thanks
> Breno
> 
> ----------------------
> WebMail Bandnet.com.br
> 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
-- 
Patrizio Bruno <patrizio@dada.it>

Re: Expand VM

[John Bradford]

> Use a swap-file on another machine on the LAN to extend your virtual
> memory if you run out of local swap-file space.

What would be really good would be a multiple gigabyte solid state
'disk', on a shared SCSI bus...

John.

Re: Expand VM

[Sean Neakums]

commence  John Bradford quotation:

>> Use a swap-file on another machine on the LAN to extend your
>> virtual memory if you run out of local swap-file space.
>
> What would be really good would be a multiple gigabyte solid state
> 'disk', on a shared SCSI bus...

If you can afford gigabytes of solid state memory, you can surely
afford to properly fit your boxes out in the first place.

-- 
 /                          |
[|] Sean Neakums            | Size *does* matter.
[|] <sneakums@zork.net>     | That's why I use Emacs.
 \                          |

Re: Expand VM

[Valdis.Kletnieks]

[-- Attachment #1: Type: text/plain, Size: 1715 bytes --]

On Thu, 23 Jan 2003 12:56:27 -0300, User & <breno_silva@beta.bandnet.com.br>  said:

> I have one idea , and this is about expand virtual memory on linux boxes 
> connected in LAN.
> Example: Linux A is processing come information , and need more memory , so 
> with this source , Linux A could access virtual memory on Linux B in LAN.

We've seen *this* done before (remember diskless Sun3-50's?) - the /dev/swap
file would be a large file on an NFS mount from a server.  At the time, this
actually made performance sense, because the old 'Shoebox' drives the -50
came with were incredibly slow, and you could actually do an NFS operation
to a larger server (a -280 with Fujitsu SuperEagle disks, for instance) faster
than talking to the local disk.

These days, it's probably easier and cheaper to just buy more RAM and/or disk
for Linux A.

> But i don´t know how translate the virtual address between Linux A and B , to 
> have success in acess VM, or how to send all the process for Linux B to be 
> processed.

Sending the whole process to Linux B to be processed is called "process
migration", and is a difficult problem.  Moving the memory image of the
process is usually pretty easy.  What is difficult is moving things like
references to open files, file locks, and so on (what if the process is
actively writing to block 739 of /usr/foo/some.file, and the LinuxB machine
doesn't have a /usr/foo, or the permissions on some.file don't match, or
another process has it locked, or... ) There be nasty dragons in this.

You're probably better off buying more RAM and disk for your A machine.
-- 
				Valdis Kletnieks
				Computer Systems Senior Engineer
				Virginia Tech


[-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --]

Re: Expand VM

[John Bradford]

> >> Use a swap-file on another machine on the LAN to extend your
> >> virtual memory if you run out of local swap-file space.
> >
> > What would be really good would be a multiple gigabyte solid state
> > 'disk', on a shared SCSI bus...
> 
> If you can afford gigabytes of solid state memory, you can surely
> afford to properly fit your boxes out in the first place.

Good point :-)

What I'd really like to see is a single device which appears as two
logical IDE or SCSI devices, one of which has 512 Mb of battery backed
RAM, and the other one 512 Mb of EEPROM.

No need for expensive flash memory, just cheap DRAM, and a few NiMH
cells to keep the RAM contents intact when the main power was
disconnected.

I've seen loads of solid state devices based on flash memory, but few
that are based on battery backed DRAM :-(.

John.

Re: Expand VM

[kkonaka]

> I have one idea , and this is about expand virtual memory on linux boxes 
> connected in LAN.
...
> Any ideas ?

try SCI ? -- eg., http://sci-serv.inrialpes.fr/

kenji

Re: Expand VM

[Herman Oosthuysen]

There is a good reason for that: Power consumption.

A large SDRAM disk would require refresh logic etc.  So you will end up 
with something closely resembling bad notebook PC.

John Bradford wrote:
> 
> I've seen loads of solid state devices based on flash memory, but few
> that are based on battery backed DRAM :-(.
> 

Re: Expand VM

[John Bradford]

> > I've seen loads of solid state devices based on flash memory, but few
> > that are based on battery backed DRAM :-(.

> There is a good reason for that: Power consumption.
> 
> A large SDRAM disk would require refresh logic etc.  So you will end up 
> with something closely resembling bad notebook PC.

I suppose it makes sense for portable devices, but would it really be
that bad for desktop/server use?  I was only thinking of about a 24
hour battery backup time - to keep the contents overnight, for
example.

John.

Re: Expand VM

[User &]

Hi Valdis

Create a new VMA on Linux B for Linux A is easy , but i have a problem , the 
address of VMA is returned on Linux B , so the VMA created on Linux B can not 
be used for process of linux A.
The problem is "how can i return address of VMA created on LINUX B to Linux 
A , and use this space ?".

Thanks
Breno

On Thu, 23 Jan 2003 11:55:38 -0500, Valdis.Kletnieks wrote
> On Thu, 23 Jan 2003 12:56:27 -0300, User & 
> <breno_silva@beta.bandnet.com.br>  said:
> 
> > I have one idea , and this is about expand virtual memory on linux boxes 
> > connected in LAN.
> > Example: Linux A is processing come information , and need more memory , 
so 
> > with this source , Linux A could access virtual memory on Linux B in LAN.
> 
> We've seen *this* done before (remember diskless Sun3-50's?) - the /dev/swap
> file would be a large file on an NFS mount from a server.  At the 
> time, this actually made performance sense, because the old 
> 'Shoebox' drives the -50 came with were incredibly slow, and you 
> could actually do an NFS operation to a larger server (a -280 with 
> Fujitsu SuperEagle disks, for instance) faster than talking to the 
> local disk.
> 
> These days, it's probably easier and cheaper to just buy more RAM 
> and/or disk for Linux A.
> 
> > But i don´t know how translate the virtual address between Linux A and 
B , to 
> > have success in acess VM, or how to send all the process for Linux B to 
be 
> > processed.
> 
> Sending the whole process to Linux B to be processed is called "process
> migration", and is a difficult problem.  Moving the memory image of the
> process is usually pretty easy.  What is difficult is moving things like
> references to open files, file locks, and so on (what if the process 
> is actively writing to block 739 of /usr/foo/some.file, and the 
> LinuxB machine doesn't have a /usr/foo, or the permissions on 
> some.file don't match, or another process has it locked, or... ) 
> There be nasty dragons in this.
> 
> You're probably better off buying more RAM and disk for your A machine.
> -- 
> 				Valdis Kletnieks
> 				Computer Systems Senior Engineer
> 				Virginia Tech



----------------------
WebMail Bandnet.com.br

Re: Expand VM

[Herman Oosthuysen]

The trouble is that you will spend so much money on the power supply, 
that the product won't make economic sense.  It would be better to just 
use a general purpose PC mother board, PSU and a UPS, since then, you 
get tremendous economy of scale.

John Bradford wrote:
>>>I've seen loads of solid state devices based on flash memory, but few
>>>that are based on battery backed DRAM :-(.
> 
> 
>>There is a good reason for that: Power consumption.
>>
>>A large SDRAM disk would require refresh logic etc.  So you will end up 
>>with something closely resembling bad notebook PC.
> 
> 
> I suppose it makes sense for portable devices, but would it really be
> that bad for desktop/server use?  I was only thinking of about a 24
> hour battery backup time - to keep the contents overnight, for
> example.
> 
> John.
> 

Re: Expand VM

[Valdis.Kletnieks]

[-- Attachment #1: Type: text/plain, Size: 1019 bytes --]

On Thu, 23 Jan 2003 16:40:14 -0300, User & said:
> Create a new VMA on Linux B for Linux A is easy , but i have a problem , the 
> address of VMA is returned on Linux B , so the VMA created on Linux B can not
 
> be used for process of linux A.

It's unclear whether you're just trying to use B for added swap space, or
if you want B to actually run code.

If it's the former, all you have to do is allocate a lot of disk space on B,
and NFS export it to A, and then have A mount it (you might need to
use a loopback mount of a file on the NFS partition and then 'swapon' the
loopback - I dont think swapon will directly take an NFS file)

> The problem is "how can i return address of VMA created on LINUX B to Linux 
> A , and use this space ?".

If you're trying to get B to actually run code, it gets a lot more messy, as
you have to worry about open file descriptors, race conditions, and many
other things.
-- 
				Valdis Kletnieks
				Computer Systems Senior Engineer
				Virginia Tech


[-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --]

Re: Expand VM

[Thomas Cataldo]

On Thu, 2003-01-23 at 20:40, User & wrote:
> Hi Valdis
> 
> Create a new VMA on Linux B for Linux A is easy , but i have a problem , the 
> address of VMA is returned on Linux B , so the VMA created on Linux B can not 
> be used for process of linux A.
> The problem is "how can i return address of VMA created on LINUX B to Linux 
> A , and use this space ?".
> 


What you're looking for is openmosix. It does process migration and so
on..

> Thanks
> Breno
> 
> On Thu, 23 Jan 2003 11:55:38 -0500, Valdis.Kletnieks wrote
> > On Thu, 23 Jan 2003 12:56:27 -0300, User & 
> > <breno_silva@beta.bandnet.com.br>  said:
> > 
> > > I have one idea , and this is about expand virtual memory on linux boxes 
> > > connected in LAN.
> > > Example: Linux A is processing come information , and need more memory , 
> so 
> > > with this source , Linux A could access virtual memory on Linux B in LAN.
> > 
> > We've seen *this* done before (remember diskless Sun3-50's?) - the /dev/swap
> > file would be a large file on an NFS mount from a server.  At the 
> > time, this actually made performance sense, because the old 
> > 'Shoebox' drives the -50 came with were incredibly slow, and you 
> > could actually do an NFS operation to a larger server (a -280 with 
> > Fujitsu SuperEagle disks, for instance) faster than talking to the 
> > local disk.
> > 
> > These days, it's probably easier and cheaper to just buy more RAM 
> > and/or disk for Linux A.
> > 
> > > But i don´t know how translate the virtual address between Linux A and 
> B , to 
> > > have success in acess VM, or how to send all the process for Linux B to 
> be 
> > > processed.
> > 
> > Sending the whole process to Linux B to be processed is called "process
> > migration", and is a difficult problem.  Moving the memory image of the
> > process is usually pretty easy.  What is difficult is moving things like
> > references to open files, file locks, and so on (what if the process 
> > is actively writing to block 739 of /usr/foo/some.file, and the 
> > LinuxB machine doesn't have a /usr/foo, or the permissions on 
> > some.file don't match, or another process has it locked, or... ) 
> > There be nasty dragons in this.
> > 
> > You're probably better off buying more RAM and disk for your A machine.
> > -- 
> > 				Valdis Kletnieks
> > 				Computer Systems Senior Engineer
> > 				Virginia Tech
> 
> 
> 
> ----------------------
> WebMail Bandnet.com.br
> 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
-- 
Thomas Cataldo <tomc@compaqnet.fr>

Re: Expand VM

[Bill Davidsen]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: TEXT/PLAIN; charset=US-ASCII, Size: 759 bytes --]

On Thu, 23 Jan 2003, User & wrote:

> I have one idea , and this is about expand virtual memory on linux boxes 
> connected in LAN.
> Example: Linux A is processing come information , and need more memory , so 
> with this source , Linux A could access virtual memory on Linux B in LAN.
> But i don´t know how translate the virtual address between Linux A and B , to 
> have success in acess VM, or how to send all the process for Linux B to be 
> processed.
> 
> Any ideas ?

1 - NFS mount a big file and use that from swap space
2 - you could try the network block device stuff, at one time I believe I
had that working with RAID-1

-- 
bill davidsen <davidsen@tmr.com>
  CTO, TMR Associates, Inc
Doing interesting things with little computers since 1979.