[PATCH 4/4] compcache: documentation

[PATCH 4/4] compcache: documentation

[Nitin Gupta]

Short guide on how to setup and use ramzswap.

Signed-off-by: Nitin Gupta <ngupta@vflare.org>
---

 Documentation/blockdev/00-INDEX     |    2 +
 Documentation/blockdev/ramzswap.txt |   52 +++++++++++++++++++++++++++++++++++
 2 files changed, 54 insertions(+), 0 deletions(-)

diff --git a/Documentation/blockdev/00-INDEX b/Documentation/blockdev/00-INDEX
index c08df56..c1cb074 100644
--- a/Documentation/blockdev/00-INDEX
+++ b/Documentation/blockdev/00-INDEX
@@ -16,3 +16,5 @@ paride.txt
 	- information about the parallel port IDE subsystem.
 ramdisk.txt
 	- short guide on how to set up and use the RAM disk.
+ramzswap.txt
+	- short guide on how to setup compressed in-memory swap device.
diff --git a/Documentation/blockdev/ramzswap.txt b/Documentation/blockdev/ramzswap.txt
new file mode 100644
index 0000000..463dd2d
--- /dev/null
+++ b/Documentation/blockdev/ramzswap.txt
@@ -0,0 +1,52 @@
+ramzswap: Compressed RAM based swap device
+-------------------------------------------
+
+Project home: http://compcache.googlecode.com/
+
+* Introduction
+
+It creates RAM based block devices which can be used (only) as swap disks.
+Pages swapped to these devices are compressed and stored in memory itself.
+See project home for use cases, performance numbers and a lot more.
+
+It consists of three modules:
+ - xvmalloc.ko: memory allocator
+ - ramzswap.ko: virtual block device driver
+ - rzscontrol userspace utility: to control individual ramzswap devices
+
+* Usage
+
+Following shows a typical sequence of steps for using ramzswap.
+
+1) Load Modules:
+	modprobe ramzswap NUM_DEVICES=4
+	This creates 4 (uninitialized) devices: /dev/ramzswap{0,1,2,3}
+	(NUM_DEVICES parameter is optional. Default: 1)
+
+2) Initialize:
+	Use rzscontrol utility to configure and initialize individual
+	ramzswap devices. Example:
+	rzscontrol /dev/ramzswap2 --init # uses default value of disksize_kb
+
+	*See rzscontrol manpage for more details and examples*
+
+3) Activate:
+	swapon /dev/ramzswap2 # or any other initialized ramzswap device
+
+4) Stats:
+	rzscontrol /dev/ramzswap2 --stats
+
+5) Deactivate:
+	swapoff /dev/ramzswap2
+
+6) Reset:
+	rzscontrol /dev/ramzswap2 --reset
+	(This frees all the memory allocated for this device).
+
+
+Please report any problems at:
+ - Mailing list: linux-mm-cc at laptop dot org
+ - Issue tracker: http://code.google.com/p/compcache/issues/list
+
+Nitin Gupta
+ngupta@vflare.org

[PATCH 4/4] compcache: documentation

[Nitin Gupta]

Short guide on how to setup and use ramzswap.

Signed-off-by: Nitin Gupta <ngupta@vflare.org>
---

 Documentation/blockdev/00-INDEX     |    2 +
 Documentation/blockdev/ramzswap.txt |   52 +++++++++++++++++++++++++++++++++++
 2 files changed, 54 insertions(+), 0 deletions(-)

diff --git a/Documentation/blockdev/00-INDEX b/Documentation/blockdev/00-INDEX
index c08df56..c1cb074 100644
--- a/Documentation/blockdev/00-INDEX
+++ b/Documentation/blockdev/00-INDEX
@@ -16,3 +16,5 @@ paride.txt
 	- information about the parallel port IDE subsystem.
 ramdisk.txt
 	- short guide on how to set up and use the RAM disk.
+ramzswap.txt
+	- short guide on how to setup compressed in-memory swap device.
diff --git a/Documentation/blockdev/ramzswap.txt b/Documentation/blockdev/ramzswap.txt
new file mode 100644
index 0000000..463dd2d
--- /dev/null
+++ b/Documentation/blockdev/ramzswap.txt
@@ -0,0 +1,52 @@
+ramzswap: Compressed RAM based swap device
+-------------------------------------------
+
+Project home: http://compcache.googlecode.com/
+
+* Introduction
+
+It creates RAM based block devices which can be used (only) as swap disks.
+Pages swapped to these devices are compressed and stored in memory itself.
+See project home for use cases, performance numbers and a lot more.
+
+It consists of three modules:
+ - xvmalloc.ko: memory allocator
+ - ramzswap.ko: virtual block device driver
+ - rzscontrol userspace utility: to control individual ramzswap devices
+
+* Usage
+
+Following shows a typical sequence of steps for using ramzswap.
+
+1) Load Modules:
+	modprobe ramzswap NUM_DEVICES=4
+	This creates 4 (uninitialized) devices: /dev/ramzswap{0,1,2,3}
+	(NUM_DEVICES parameter is optional. Default: 1)
+
+2) Initialize:
+	Use rzscontrol utility to configure and initialize individual
+	ramzswap devices. Example:
+	rzscontrol /dev/ramzswap2 --init # uses default value of disksize_kb
+
+	*See rzscontrol manpage for more details and examples*
+
+3) Activate:
+	swapon /dev/ramzswap2 # or any other initialized ramzswap device
+
+4) Stats:
+	rzscontrol /dev/ramzswap2 --stats
+
+5) Deactivate:
+	swapoff /dev/ramzswap2
+
+6) Reset:
+	rzscontrol /dev/ramzswap2 --reset
+	(This frees all the memory allocated for this device).
+
+
+Please report any problems at:
+ - Mailing list: linux-mm-cc at laptop dot org
+ - Issue tracker: http://code.google.com/p/compcache/issues/list
+
+Nitin Gupta
+ngupta@vflare.org

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 4/4] compcache: documentation

[Balbir Singh]

On Mon, Aug 24, 2009 at 10:08 AM, Nitin Gupta<ngupta@vflare.org> wrote:
> Short guide on how to setup and use ramzswap.
>
> Signed-off-by: Nitin Gupta <ngupta@vflare.org>
> ---
>
>  Documentation/blockdev/00-INDEX     |    2 +
>  Documentation/blockdev/ramzswap.txt |   52 +++++++++++++++++++++++++++++++++++
>  2 files changed, 54 insertions(+), 0 deletions(-)
>
> diff --git a/Documentation/blockdev/00-INDEX b/Documentation/blockdev/00-INDEX
> index c08df56..c1cb074 100644
> --- a/Documentation/blockdev/00-INDEX
> +++ b/Documentation/blockdev/00-INDEX
> @@ -16,3 +16,5 @@ paride.txt
>        - information about the parallel port IDE subsystem.
>  ramdisk.txt
>        - short guide on how to set up and use the RAM disk.
> +ramzswap.txt
> +       - short guide on how to setup compressed in-memory swap device.
> diff --git a/Documentation/blockdev/ramzswap.txt b/Documentation/blockdev/ramzswap.txt
> new file mode 100644
> index 0000000..463dd2d
> --- /dev/null
> +++ b/Documentation/blockdev/ramzswap.txt
> @@ -0,0 +1,52 @@
> +ramzswap: Compressed RAM based swap device
> +-------------------------------------------
> +
> +Project home: http://compcache.googlecode.com/
> +
> +* Introduction
> +
> +It creates RAM based block devices which can be used (only) as swap disks.
> +Pages swapped to these devices are compressed and stored in memory itself.
> +See project home for use cases, performance numbers and a lot more.
> +
> +It consists of three modules:
> + - xvmalloc.ko: memory allocator

I've seen your case for a custom allocator, but why can't we

1) Refactor slob and use it
2) Do we care about the optimizations in SLUB w.r.t. scalability in
your module? If so.. will xvmalloc meet those requirements?

> + - ramzswap.ko: virtual block device driver
> + - rzscontrol userspace utility: to control individual ramzswap devices
> +
> +* Usage
> +
> +Following shows a typical sequence of steps for using ramzswap.
> +
> +1) Load Modules:
> +       modprobe ramzswap NUM_DEVICES=4
> +       This creates 4 (uninitialized) devices: /dev/ramzswap{0,1,2,3}
> +       (NUM_DEVICES parameter is optional. Default: 1)
> +
> +2) Initialize:
> +       Use rzscontrol utility to configure and initialize individual
> +       ramzswap devices. Example:
> +       rzscontrol /dev/ramzswap2 --init # uses default value of disksize_kb
> +
> +       *See rzscontrol manpage for more details and examples*
> +
> +3) Activate:
> +       swapon /dev/ramzswap2 # or any other initialized ramzswap device
> +
> +4) Stats:
> +       rzscontrol /dev/ramzswap2 --stats
> +
> +5) Deactivate:
> +       swapoff /dev/ramzswap2
> +
> +6) Reset:
> +       rzscontrol /dev/ramzswap2 --reset
> +       (This frees all the memory allocated for this device).

What level of compression have you observed? Any speed trade-offs?

Balbir

Re: [PATCH 4/4] compcache: documentation

[Balbir Singh]

On Mon, Aug 24, 2009 at 10:08 AM, Nitin Gupta<ngupta@vflare.org> wrote:
> Short guide on how to setup and use ramzswap.
>
> Signed-off-by: Nitin Gupta <ngupta@vflare.org>
> ---
>
>  Documentation/blockdev/00-INDEX     |    2 +
>  Documentation/blockdev/ramzswap.txt |   52 +++++++++++++++++++++++++++++++++++
>  2 files changed, 54 insertions(+), 0 deletions(-)
>
> diff --git a/Documentation/blockdev/00-INDEX b/Documentation/blockdev/00-INDEX
> index c08df56..c1cb074 100644
> --- a/Documentation/blockdev/00-INDEX
> +++ b/Documentation/blockdev/00-INDEX
> @@ -16,3 +16,5 @@ paride.txt
>        - information about the parallel port IDE subsystem.
>  ramdisk.txt
>        - short guide on how to set up and use the RAM disk.
> +ramzswap.txt
> +       - short guide on how to setup compressed in-memory swap device.
> diff --git a/Documentation/blockdev/ramzswap.txt b/Documentation/blockdev/ramzswap.txt
> new file mode 100644
> index 0000000..463dd2d
> --- /dev/null
> +++ b/Documentation/blockdev/ramzswap.txt
> @@ -0,0 +1,52 @@
> +ramzswap: Compressed RAM based swap device
> +-------------------------------------------
> +
> +Project home: http://compcache.googlecode.com/
> +
> +* Introduction
> +
> +It creates RAM based block devices which can be used (only) as swap disks.
> +Pages swapped to these devices are compressed and stored in memory itself.
> +See project home for use cases, performance numbers and a lot more.
> +
> +It consists of three modules:
> + - xvmalloc.ko: memory allocator

I've seen your case for a custom allocator, but why can't we

1) Refactor slob and use it
2) Do we care about the optimizations in SLUB w.r.t. scalability in
your module? If so.. will xvmalloc meet those requirements?

> + - ramzswap.ko: virtual block device driver
> + - rzscontrol userspace utility: to control individual ramzswap devices
> +
> +* Usage
> +
> +Following shows a typical sequence of steps for using ramzswap.
> +
> +1) Load Modules:
> +       modprobe ramzswap NUM_DEVICES=4
> +       This creates 4 (uninitialized) devices: /dev/ramzswap{0,1,2,3}
> +       (NUM_DEVICES parameter is optional. Default: 1)
> +
> +2) Initialize:
> +       Use rzscontrol utility to configure and initialize individual
> +       ramzswap devices. Example:
> +       rzscontrol /dev/ramzswap2 --init # uses default value of disksize_kb
> +
> +       *See rzscontrol manpage for more details and examples*
> +
> +3) Activate:
> +       swapon /dev/ramzswap2 # or any other initialized ramzswap device
> +
> +4) Stats:
> +       rzscontrol /dev/ramzswap2 --stats
> +
> +5) Deactivate:
> +       swapoff /dev/ramzswap2
> +
> +6) Reset:
> +       rzscontrol /dev/ramzswap2 --reset
> +       (This frees all the memory allocated for this device).

What level of compression have you observed? Any speed trade-offs?

Balbir

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 4/4] compcache: documentation

[Nitin Gupta]

On 08/25/2009 10:33 PM, Balbir Singh wrote:


>> +It consists of three modules:
>> + - xvmalloc.ko: memory allocator
>
> I've seen your case for a custom allocator, but why can't we
>
> 1) Refactor slob and use it

SLOB is fundamentally a different allocator. It looked at it in detail
but could not image how can I make it suitable for the project. SLOB
really does not fit it.

> 2) Do we care about the optimizations in SLUB w.r.t. scalability in
> your module? If so.. will xvmalloc meet those requirements?
>

Scalability is desired which xvmalloc lacks in its current state. My
plan is to have a wrapper around xvmalloc that creates per-cpu pools
and leave xvmalloc core simple. Along with this, detailed profiling
needs to be done to see where the bottlenecks are in the core itself.


>
> What level of compression have you observed? Any speed trade-offs?
>

All the performance numbers can be found at:
http://code.google.com/p/compcache/wiki/Performance

I also summarized these in patch [0/4]:
http://lkml.org/lkml/2009/8/24/8

The compression ratio is highly workload dependent. On "generic" desktop
workload, stats show:
  - ~80% of pages compressing to PAGE_SIZE/2 or less.
  - ~1% incompressible pages.


For the speed part, please refer to performance numbers at link above.
It show cases where it help or hurts the performance.

Thanks,
Nitin

Re: [PATCH 4/4] compcache: documentation

[Nitin Gupta]

On 08/25/2009 10:33 PM, Balbir Singh wrote:


>> +It consists of three modules:
>> + - xvmalloc.ko: memory allocator
>
> I've seen your case for a custom allocator, but why can't we
>
> 1) Refactor slob and use it

SLOB is fundamentally a different allocator. It looked at it in detail
but could not image how can I make it suitable for the project. SLOB
really does not fit it.

> 2) Do we care about the optimizations in SLUB w.r.t. scalability in
> your module? If so.. will xvmalloc meet those requirements?
>

Scalability is desired which xvmalloc lacks in its current state. My
plan is to have a wrapper around xvmalloc that creates per-cpu pools
and leave xvmalloc core simple. Along with this, detailed profiling
needs to be done to see where the bottlenecks are in the core itself.


>
> What level of compression have you observed? Any speed trade-offs?
>

All the performance numbers can be found at:
http://code.google.com/p/compcache/wiki/Performance

I also summarized these in patch [0/4]:
http://lkml.org/lkml/2009/8/24/8

The compression ratio is highly workload dependent. On "generic" desktop
workload, stats show:
  - ~80% of pages compressing to PAGE_SIZE/2 or less.
  - ~1% incompressible pages.


For the speed part, please refer to performance numbers at link above.
It show cases where it help or hurts the performance.

Thanks,
Nitin


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 4/4] compcache: documentation

[Balbir Singh]

* Nitin Gupta <ngupta@vflare.org> [2009-08-25 23:41:06]:

> On 08/25/2009 10:33 PM, Balbir Singh wrote:
>
>
>>> +It consists of three modules:
>>> + - xvmalloc.ko: memory allocator
>>
>> I've seen your case for a custom allocator, but why can't we
>>
>> 1) Refactor slob and use it
>
> SLOB is fundamentally a different allocator. It looked at it in detail
> but could not image how can I make it suitable for the project. SLOB
> really does not fit it.
>
>> 2) Do we care about the optimizations in SLUB w.r.t. scalability in
>> your module? If so.. will xvmalloc meet those requirements?
>>
>
> Scalability is desired which xvmalloc lacks in its current state. My
> plan is to have a wrapper around xvmalloc that creates per-cpu pools
> and leave xvmalloc core simple. Along with this, detailed profiling
> needs to be done to see where the bottlenecks are in the core itself.
>

I've not yet tested the patches, but adding another allocator does
worry me a bit. Do you intend to allow other users to consume the
allocator routines?

>
>>
>> What level of compression have you observed? Any speed trade-offs?
>>
>
> All the performance numbers can be found at:
> http://code.google.com/p/compcache/wiki/Performance
>
> I also summarized these in patch [0/4]:
> http://lkml.org/lkml/2009/8/24/8
>
> The compression ratio is highly workload dependent. On "generic" desktop
> workload, stats show:
>  - ~80% of pages compressing to PAGE_SIZE/2 or less.
>  - ~1% incompressible pages.
>
>
> For the speed part, please refer to performance numbers at link above.
> It show cases where it help or hurts the performance.
>

Thanks, I'll take a look at the links

-- 
	Balbir

Re: [PATCH 4/4] compcache: documentation

[Balbir Singh]

* Nitin Gupta <ngupta@vflare.org> [2009-08-25 23:41:06]:

> On 08/25/2009 10:33 PM, Balbir Singh wrote:
>
>
>>> +It consists of three modules:
>>> + - xvmalloc.ko: memory allocator
>>
>> I've seen your case for a custom allocator, but why can't we
>>
>> 1) Refactor slob and use it
>
> SLOB is fundamentally a different allocator. It looked at it in detail
> but could not image how can I make it suitable for the project. SLOB
> really does not fit it.
>
>> 2) Do we care about the optimizations in SLUB w.r.t. scalability in
>> your module? If so.. will xvmalloc meet those requirements?
>>
>
> Scalability is desired which xvmalloc lacks in its current state. My
> plan is to have a wrapper around xvmalloc that creates per-cpu pools
> and leave xvmalloc core simple. Along with this, detailed profiling
> needs to be done to see where the bottlenecks are in the core itself.
>

I've not yet tested the patches, but adding another allocator does
worry me a bit. Do you intend to allow other users to consume the
allocator routines?

>
>>
>> What level of compression have you observed? Any speed trade-offs?
>>
>
> All the performance numbers can be found at:
> http://code.google.com/p/compcache/wiki/Performance
>
> I also summarized these in patch [0/4]:
> http://lkml.org/lkml/2009/8/24/8
>
> The compression ratio is highly workload dependent. On "generic" desktop
> workload, stats show:
>  - ~80% of pages compressing to PAGE_SIZE/2 or less.
>  - ~1% incompressible pages.
>
>
> For the speed part, please refer to performance numbers at link above.
> It show cases where it help or hurts the performance.
>

Thanks, I'll take a look at the links

-- 
	Balbir

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 4/4] compcache: documentation

[Nitin Gupta]

On 08/29/2009 01:23 AM, Balbir Singh wrote:
> * Nitin Gupta<ngupta@vflare.org>  [2009-08-25 23:41:06]:
>
>> On 08/25/2009 10:33 PM, Balbir Singh wrote:
>>
>>
>>>> +It consists of three modules:
>>>> + - xvmalloc.ko: memory allocator
>>>
>>> I've seen your case for a custom allocator, but why can't we
>>>
>>> 1) Refactor slob and use it
>>
>> SLOB is fundamentally a different allocator. It looked at it in detail
>> but could not image how can I make it suitable for the project. SLOB
>> really does not fit it.
>>
>>> 2) Do we care about the optimizations in SLUB w.r.t. scalability in
>>> your module? If so.. will xvmalloc meet those requirements?
>>>
>>
>> Scalability is desired which xvmalloc lacks in its current state. My
>> plan is to have a wrapper around xvmalloc that creates per-cpu pools
>> and leave xvmalloc core simple. Along with this, detailed profiling
>> needs to be done to see where the bottlenecks are in the core itself.
>>
>
> I've not yet tested the patches, but adding another allocator does
> worry me a bit. Do you intend to allow other users to consume the
> allocator routines?
>

No. This allocator is not compiled as separate module, does not export any
symbol and is compiled with ramzswap. So, no one else can use it.

Thanks,
Nitin

Re: [PATCH 4/4] compcache: documentation

[Nitin Gupta]

On 08/29/2009 01:23 AM, Balbir Singh wrote:
> * Nitin Gupta<ngupta@vflare.org>  [2009-08-25 23:41:06]:
>
>> On 08/25/2009 10:33 PM, Balbir Singh wrote:
>>
>>
>>>> +It consists of three modules:
>>>> + - xvmalloc.ko: memory allocator
>>>
>>> I've seen your case for a custom allocator, but why can't we
>>>
>>> 1) Refactor slob and use it
>>
>> SLOB is fundamentally a different allocator. It looked at it in detail
>> but could not image how can I make it suitable for the project. SLOB
>> really does not fit it.
>>
>>> 2) Do we care about the optimizations in SLUB w.r.t. scalability in
>>> your module? If so.. will xvmalloc meet those requirements?
>>>
>>
>> Scalability is desired which xvmalloc lacks in its current state. My
>> plan is to have a wrapper around xvmalloc that creates per-cpu pools
>> and leave xvmalloc core simple. Along with this, detailed profiling
>> needs to be done to see where the bottlenecks are in the core itself.
>>
>
> I've not yet tested the patches, but adding another allocator does
> worry me a bit. Do you intend to allow other users to consume the
> allocator routines?
>

No. This allocator is not compiled as separate module, does not export any
symbol and is compiled with ramzswap. So, no one else can use it.

Thanks,
Nitin

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>