All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH 0/4] compcache: compressed in-memory swapping
@ 2009-08-24  4:37 ` Nitin Gupta
  0 siblings, 0 replies; 6+ messages in thread
From: Nitin Gupta @ 2009-08-24  4:37 UTC (permalink / raw)
  To: akpm; +Cc: linux-kernel, linux-mm, linux-mm-cc

Hi,

Project home: http://compcache.googlecode.com/

It creates RAM based block devices which can be used (only) as swap disks.
Pages swapped to this device are compressed and stored in memory itself. This
is a big win over swapping to slow hard-disk which are typically used as swap
disk. For flash, these suffer from wear-leveling issues when used as swap disk
- so again its helpful. For swapless systems, it allows more apps to run for a
given amount of memory.

It can create multiple ramzswap devices (/dev/ramzswapX, X = 0, 1, 2, ...).
Each of these devices can have separate backing swap (file or disk partition)
which is used when incompressible page is found or memory limit for device is
reached.

A separate userspace utility called rzscontrol is used to manage individual
ramzswap devices.

* Testing notes

Tested on x86, x64, ARM
ARM:
 - Cortex-A8 (Beagleboard)
 - ARM11 (Android G1)
 - OMAP2420 (Nokia N810)

* Performance

All performance numbers/plots can be found at:
http://code.google.com/p/compcache/wiki/Performance

Below is a summary of this data:

General:
 - Swap R/W times are reduced from milliseconds (in case of hard disks)
down to microseconds.

Positive cases:
 - Shows 33% improvement in 'scan' benchmark which allocates given amount
of memory and linearly reads/writes to this region. This benchmark also
exposes *bottlenecks* in ramzswap code (global mutex) due to which this gain
is so small.
 - On Linux thin clients, it gives the effect of nearly doubling the amount of
memory.

Negative cases:
Any workload that has active working set w.r.t. filesystem cache that is
nearly equal to amount of RAM while has minimal anonymous memory requirement,
is expected to suffer maximum loss in performance with ramzswap enabled.

Iozone filesystem benchmark can simulate exactly this kind of workload.
As expected, this test shows performance loss of ~25% with ramzswap.


(Sorry for long patch[2/4] but its now very hard to split it up).

 Documentation/blockdev/00-INDEX       |    2 +
 Documentation/blockdev/ramzswap.txt   |   52 ++
 drivers/block/Kconfig                 |   22 +
 drivers/block/Makefile                |    1 +
 drivers/block/ramzswap/Makefile       |    2 +
 drivers/block/ramzswap/ramzswap.c     | 1511 +++++++++++++++++++++++++++++++++
 drivers/block/ramzswap/ramzswap.h     |  182 ++++
 drivers/block/ramzswap/xvmalloc.c     |  556 ++++++++++++
 drivers/block/ramzswap/xvmalloc.h     |   30 +
 drivers/block/ramzswap/xvmalloc_int.h |   86 ++
 include/linux/ramzswap_ioctl.h        |   51 ++
 include/linux/swap.h                  |    5 +
 mm/swapfile.c                         |   33 +
 13 files changed, 2533 insertions(+), 0 deletions(-)

^ permalink raw reply	[flat|nested] 6+ messages in thread

* [PATCH 0/4] compcache: compressed in-memory swapping
@ 2009-08-24  4:37 ` Nitin Gupta
  0 siblings, 0 replies; 6+ messages in thread
From: Nitin Gupta @ 2009-08-24  4:37 UTC (permalink / raw)
  To: akpm; +Cc: linux-kernel, linux-mm, linux-mm-cc

Hi,

Project home: http://compcache.googlecode.com/

It creates RAM based block devices which can be used (only) as swap disks.
Pages swapped to this device are compressed and stored in memory itself. This
is a big win over swapping to slow hard-disk which are typically used as swap
disk. For flash, these suffer from wear-leveling issues when used as swap disk
- so again its helpful. For swapless systems, it allows more apps to run for a
given amount of memory.

It can create multiple ramzswap devices (/dev/ramzswapX, X = 0, 1, 2, ...).
Each of these devices can have separate backing swap (file or disk partition)
which is used when incompressible page is found or memory limit for device is
reached.

A separate userspace utility called rzscontrol is used to manage individual
ramzswap devices.

* Testing notes

Tested on x86, x64, ARM
ARM:
 - Cortex-A8 (Beagleboard)
 - ARM11 (Android G1)
 - OMAP2420 (Nokia N810)

* Performance

All performance numbers/plots can be found at:
http://code.google.com/p/compcache/wiki/Performance

Below is a summary of this data:

General:
 - Swap R/W times are reduced from milliseconds (in case of hard disks)
down to microseconds.

Positive cases:
 - Shows 33% improvement in 'scan' benchmark which allocates given amount
of memory and linearly reads/writes to this region. This benchmark also
exposes *bottlenecks* in ramzswap code (global mutex) due to which this gain
is so small.
 - On Linux thin clients, it gives the effect of nearly doubling the amount of
memory.

Negative cases:
Any workload that has active working set w.r.t. filesystem cache that is
nearly equal to amount of RAM while has minimal anonymous memory requirement,
is expected to suffer maximum loss in performance with ramzswap enabled.

Iozone filesystem benchmark can simulate exactly this kind of workload.
As expected, this test shows performance loss of ~25% with ramzswap.


(Sorry for long patch[2/4] but its now very hard to split it up).

 Documentation/blockdev/00-INDEX       |    2 +
 Documentation/blockdev/ramzswap.txt   |   52 ++
 drivers/block/Kconfig                 |   22 +
 drivers/block/Makefile                |    1 +
 drivers/block/ramzswap/Makefile       |    2 +
 drivers/block/ramzswap/ramzswap.c     | 1511 +++++++++++++++++++++++++++++++++
 drivers/block/ramzswap/ramzswap.h     |  182 ++++
 drivers/block/ramzswap/xvmalloc.c     |  556 ++++++++++++
 drivers/block/ramzswap/xvmalloc.h     |   30 +
 drivers/block/ramzswap/xvmalloc_int.h |   86 ++
 include/linux/ramzswap_ioctl.h        |   51 ++
 include/linux/swap.h                  |    5 +
 mm/swapfile.c                         |   33 +
 13 files changed, 2533 insertions(+), 0 deletions(-)

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [PATCH 0/4] compcache: compressed in-memory swapping
  2009-08-24  4:37 ` Nitin Gupta
@ 2009-08-24 16:01   ` Randy Dunlap
  -1 siblings, 0 replies; 6+ messages in thread
From: Randy Dunlap @ 2009-08-24 16:01 UTC (permalink / raw)
  To: ngupta; +Cc: akpm, linux-kernel, linux-mm, linux-mm-cc

On Mon, 24 Aug 2009 10:07:33 +0530 Nitin Gupta wrote:

> (Sorry for long patch[2/4] but its now very hard to split it up).
> 
>  Documentation/blockdev/00-INDEX       |    2 +
>  Documentation/blockdev/ramzswap.txt   |   52 ++
>  drivers/block/Kconfig                 |   22 +
>  drivers/block/Makefile                |    1 +
>  drivers/block/ramzswap/Makefile       |    2 +

I can't find drivers/block/ramzswap/Makefile in the patches...

>  drivers/block/ramzswap/ramzswap.c     | 1511 +++++++++++++++++++++++++++++++++
>  drivers/block/ramzswap/ramzswap.h     |  182 ++++
>  drivers/block/ramzswap/xvmalloc.c     |  556 ++++++++++++
>  drivers/block/ramzswap/xvmalloc.h     |   30 +
>  drivers/block/ramzswap/xvmalloc_int.h |   86 ++
>  include/linux/ramzswap_ioctl.h        |   51 ++
>  include/linux/swap.h                  |    5 +
>  mm/swapfile.c                         |   33 +
>  13 files changed, 2533 insertions(+), 0 deletions(-)


---
~Randy
LPC 2009, Sept. 23-25, Portland, Oregon
http://linuxplumbersconf.org/2009/

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [PATCH 0/4] compcache: compressed in-memory swapping
@ 2009-08-24 16:01   ` Randy Dunlap
  0 siblings, 0 replies; 6+ messages in thread
From: Randy Dunlap @ 2009-08-24 16:01 UTC (permalink / raw)
  To: ngupta; +Cc: akpm, linux-kernel, linux-mm, linux-mm-cc

On Mon, 24 Aug 2009 10:07:33 +0530 Nitin Gupta wrote:

> (Sorry for long patch[2/4] but its now very hard to split it up).
> 
>  Documentation/blockdev/00-INDEX       |    2 +
>  Documentation/blockdev/ramzswap.txt   |   52 ++
>  drivers/block/Kconfig                 |   22 +
>  drivers/block/Makefile                |    1 +
>  drivers/block/ramzswap/Makefile       |    2 +

I can't find drivers/block/ramzswap/Makefile in the patches...

>  drivers/block/ramzswap/ramzswap.c     | 1511 +++++++++++++++++++++++++++++++++
>  drivers/block/ramzswap/ramzswap.h     |  182 ++++
>  drivers/block/ramzswap/xvmalloc.c     |  556 ++++++++++++
>  drivers/block/ramzswap/xvmalloc.h     |   30 +
>  drivers/block/ramzswap/xvmalloc_int.h |   86 ++
>  include/linux/ramzswap_ioctl.h        |   51 ++
>  include/linux/swap.h                  |    5 +
>  mm/swapfile.c                         |   33 +
>  13 files changed, 2533 insertions(+), 0 deletions(-)


---
~Randy
LPC 2009, Sept. 23-25, Portland, Oregon
http://linuxplumbersconf.org/2009/

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [PATCH 0/4] compcache: compressed in-memory swapping
  2009-08-24 16:01   ` Randy Dunlap
@ 2009-08-24 17:30     ` Nitin Gupta
  -1 siblings, 0 replies; 6+ messages in thread
From: Nitin Gupta @ 2009-08-24 17:30 UTC (permalink / raw)
  To: Randy Dunlap; +Cc: akpm, linux-kernel, linux-mm, linux-mm-cc

On Mon, Aug 24, 2009 at 9:31 PM, Randy Dunlap<randy.dunlap@oracle.com> wrote:
> On Mon, 24 Aug 2009 10:07:33 +0530 Nitin Gupta wrote:
>
>> (Sorry for long patch[2/4] but its now very hard to split it up).
>>
>>  Documentation/blockdev/00-INDEX       |    2 +
>>  Documentation/blockdev/ramzswap.txt   |   52 ++
>>  drivers/block/Kconfig                 |   22 +
>>  drivers/block/Makefile                |    1 +
>>  drivers/block/ramzswap/Makefile       |    2 +
>
> I can't find drivers/block/ramzswap/Makefile in the patches...
>

My bad. I missed it. Here it is:

obj-$(CONFIG_BLK_DEV_RAMZSWAP)  +=      ramzswap.o xvmalloc.o


I will send updated diffs with swap_lock fix in patch[3/4] with this Makefile
include once I get additional reviews.

Thanks,
Nitin

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [PATCH 0/4] compcache: compressed in-memory swapping
@ 2009-08-24 17:30     ` Nitin Gupta
  0 siblings, 0 replies; 6+ messages in thread
From: Nitin Gupta @ 2009-08-24 17:30 UTC (permalink / raw)
  To: Randy Dunlap; +Cc: akpm, linux-kernel, linux-mm, linux-mm-cc

On Mon, Aug 24, 2009 at 9:31 PM, Randy Dunlap<randy.dunlap@oracle.com> wrote:
> On Mon, 24 Aug 2009 10:07:33 +0530 Nitin Gupta wrote:
>
>> (Sorry for long patch[2/4] but its now very hard to split it up).
>>
>>  Documentation/blockdev/00-INDEX       |    2 +
>>  Documentation/blockdev/ramzswap.txt   |   52 ++
>>  drivers/block/Kconfig                 |   22 +
>>  drivers/block/Makefile                |    1 +
>>  drivers/block/ramzswap/Makefile       |    2 +
>
> I can't find drivers/block/ramzswap/Makefile in the patches...
>

My bad. I missed it. Here it is:

obj-$(CONFIG_BLK_DEV_RAMZSWAP)  +=      ramzswap.o xvmalloc.o


I will send updated diffs with swap_lock fix in patch[3/4] with this Makefile
include once I get additional reviews.

Thanks,
Nitin

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

^ permalink raw reply	[flat|nested] 6+ messages in thread

end of thread, other threads:[~2009-08-26  4:10 UTC | newest]

Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2009-08-24  4:37 [PATCH 0/4] compcache: compressed in-memory swapping Nitin Gupta
2009-08-24  4:37 ` Nitin Gupta
2009-08-24 16:01 ` Randy Dunlap
2009-08-24 16:01   ` Randy Dunlap
2009-08-24 17:30   ` Nitin Gupta
2009-08-24 17:30     ` Nitin Gupta

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.