zswap: How to determine whether it is compressing swap pages?

zswap: How to determine whether it is compressing swap pages?

[Martin Steigerwald]

Hi Seth, hi everyone,

Yesterday I build 3.11-rc1 with CONFIG_ZSWAP and wanted to test it.

I added zswap.enabled=1 and get:

martin@merkaba:~> dmesg | grep zswap
[    0.000000] Command line: BOOT_IMAGE=/vmlinuz-3.11.0-rc1-tp520+ 
root=/dev/mapper/merkaba-debian ro rootflags=subvol=root init=/bin/systemd 
cgroup_enable=memory threadirqs i915.i915_enable_rc6=7 zcache zswap.enabled=1
[    0.000000] Kernel command line: BOOT_IMAGE=/vmlinuz-3.11.0-rc1-tp520+ 
root=/dev/mapper/merkaba-debian ro rootflags=subvol=root init=/bin/systemd 
cgroup_enable=memory threadirqs i915.i915_enable_rc6=7 zcache zswap.enabled=1
[    1.452443] zswap: loading zswap
[    1.452465] zswap: using lzo compressor


I did a stress -m 1 --vm-keep --vm-bytes 4G on this 8 GB ThinkPad T520 in 
order to allocate some swap.

Still I think zswap didn´t do anything:

merkaba:/sys/kernel/debug/zswap> grep . *
duplicate_entry:0
pool_limit_hit:0
pool_pages:0
reject_alloc_fail:0
reject_compress_poor:0
reject_kmemcache_fail:0
reject_reclaim_fail:0
stored_pages:0
written_back_pages:0


However:

merkaba:/sys/kernel/slab/zswap_entry> grep . *
aliases:9
align:8
grep: alloc_calls: Die angeforderte Funktion ist nicht implementiert
cache_dma:0
cpu_partial:0
cpu_slabs:4 N0=4
destroy_by_rcu:0
grep: free_calls: Die angeforderte Funktion ist nicht implementiert
hwcache_align:0
min_partial:5
objects:2550 N0=2550
object_size:48
objects_partial:0
objs_per_slab:85
order:0
partial:0
poison:0
reclaim_account:0
red_zone:0
remote_node_defrag_ratio:100
reserved:0
sanity_checks:0
slabs:30 N0=30
slabs_cpu_partial:0(0)
slab_size:48
store_user:0
total_objects:2550 N0=2550
trace:0

It has some objects it seems.


How do I know whether zswap actually does something?

Will zswap work even with zcache enabled? As I understand zcache compresses 
swap device pages on the block device level in addition to compressing read 
cache pages of usual filesystems. Which one takes precedence, zcache or zswap? 
Can I disable zcache for swap device?



Here is dmesg for zcache:

martin@merkaba:~> dmesg | grep zcache
[    0.000000] Command line: BOOT_IMAGE=/vmlinuz-3.11.0-rc1-tp520+ 
root=/dev/mapper/merkaba-debian ro rootflags=subvol=root init=/bin/systemd 
cgroup_enable=memory threadirqs i915.i915_enable_rc6=7 zcache zswap.enabled=1
[    0.000000] Kernel command line: BOOT_IMAGE=/vmlinuz-3.11.0-rc1-tp520+ 
root=/dev/mapper/merkaba-debian ro rootflags=subvol=root init=/bin/systemd 
cgroup_enable=memory threadirqs i915.i915_enable_rc6=7 zcache zswap.enabled=1
[    1.453531] zcache: using lzo compressor
[    1.453634] zcache: cleancache enabled using kernel transcendent memory and 
compression buddies
[    1.453679] zcache: frontswap enabled using kernel transcendent memory and 
compression buddies
[    1.453722] zcache: frontswap_ops overridden
[    5.358288] zcache: created ephemeral local tmem pool, id=0
[    8.155684] zcache: created persistent local tmem pool, id=1
[    8.331680] zcache: created ephemeral local tmem pool, id=2
[    8.593235] zcache: created ephemeral local tmem pool, id=3
[    8.743330] zcache: created ephemeral local tmem pool, id=4


Thanks,
-- 
Martin 'Helios' Steigerwald - http://www.Lichtvoll.de
GPG: 03B0 0D6C 0040 0710 4AFA  B82F 991B EAAC A599 84C7

zswap: How to determine whether it is compressing swap pages?

[Martin Steigerwald]

Hi Seth, hi everyone,

Yesterday I build 3.11-rc1 with CONFIG_ZSWAP and wanted to test it.

I added zswap.enabled=1 and get:

martin@merkaba:~> dmesg | grep zswap
[    0.000000] Command line: BOOT_IMAGE=/vmlinuz-3.11.0-rc1-tp520+ 
root=/dev/mapper/merkaba-debian ro rootflags=subvol=root init=/bin/systemd 
cgroup_enable=memory threadirqs i915.i915_enable_rc6=7 zcache zswap.enabled=1
[    0.000000] Kernel command line: BOOT_IMAGE=/vmlinuz-3.11.0-rc1-tp520+ 
root=/dev/mapper/merkaba-debian ro rootflags=subvol=root init=/bin/systemd 
cgroup_enable=memory threadirqs i915.i915_enable_rc6=7 zcache zswap.enabled=1
[    1.452443] zswap: loading zswap
[    1.452465] zswap: using lzo compressor


I did a stress -m 1 --vm-keep --vm-bytes 4G on this 8 GB ThinkPad T520 in 
order to allocate some swap.

Still I think zswap didn´t do anything:

merkaba:/sys/kernel/debug/zswap> grep . *
duplicate_entry:0
pool_limit_hit:0
pool_pages:0
reject_alloc_fail:0
reject_compress_poor:0
reject_kmemcache_fail:0
reject_reclaim_fail:0
stored_pages:0
written_back_pages:0


However:

merkaba:/sys/kernel/slab/zswap_entry> grep . *
aliases:9
align:8
grep: alloc_calls: Die angeforderte Funktion ist nicht implementiert
cache_dma:0
cpu_partial:0
cpu_slabs:4 N0=4
destroy_by_rcu:0
grep: free_calls: Die angeforderte Funktion ist nicht implementiert
hwcache_align:0
min_partial:5
objects:2550 N0=2550
object_size:48
objects_partial:0
objs_per_slab:85
order:0
partial:0
poison:0
reclaim_account:0
red_zone:0
remote_node_defrag_ratio:100
reserved:0
sanity_checks:0
slabs:30 N0=30
slabs_cpu_partial:0(0)
slab_size:48
store_user:0
total_objects:2550 N0=2550
trace:0

It has some objects it seems.


How do I know whether zswap actually does something?

Will zswap work even with zcache enabled? As I understand zcache compresses 
swap device pages on the block device level in addition to compressing read 
cache pages of usual filesystems. Which one takes precedence, zcache or zswap? 
Can I disable zcache for swap device?



Here is dmesg for zcache:

martin@merkaba:~> dmesg | grep zcache
[    0.000000] Command line: BOOT_IMAGE=/vmlinuz-3.11.0-rc1-tp520+ 
root=/dev/mapper/merkaba-debian ro rootflags=subvol=root init=/bin/systemd 
cgroup_enable=memory threadirqs i915.i915_enable_rc6=7 zcache zswap.enabled=1
[    0.000000] Kernel command line: BOOT_IMAGE=/vmlinuz-3.11.0-rc1-tp520+ 
root=/dev/mapper/merkaba-debian ro rootflags=subvol=root init=/bin/systemd 
cgroup_enable=memory threadirqs i915.i915_enable_rc6=7 zcache zswap.enabled=1
[    1.453531] zcache: using lzo compressor
[    1.453634] zcache: cleancache enabled using kernel transcendent memory and 
compression buddies
[    1.453679] zcache: frontswap enabled using kernel transcendent memory and 
compression buddies
[    1.453722] zcache: frontswap_ops overridden
[    5.358288] zcache: created ephemeral local tmem pool, id=0
[    8.155684] zcache: created persistent local tmem pool, id=1
[    8.331680] zcache: created ephemeral local tmem pool, id=2
[    8.593235] zcache: created ephemeral local tmem pool, id=3
[    8.743330] zcache: created ephemeral local tmem pool, id=4


Thanks,
-- 
Martin 'Helios' Steigerwald - http://www.Lichtvoll.de
GPG: 03B0 0D6C 0040 0710 4AFA  B82F 991B EAAC A599 84C7

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: zswap: How to determine whether it is compressing swap pages?

[Wanpeng Li]

On Wed, Jul 17, 2013 at 12:04:38PM +0200, Martin Steigerwald wrote:
>Hi Seth, hi everyone,
>
>Yesterday I build 3.11-rc1 with CONFIG_ZSWAP and wanted to test it.
>
>I added zswap.enabled=1 and get:
>
>martin@merkaba:~> dmesg | grep zswap
>[    0.000000] Command line: BOOT_IMAGE=/vmlinuz-3.11.0-rc1-tp520+ 
>root=/dev/mapper/merkaba-debian ro rootflags=subvol=root init=/bin/systemd 
>cgroup_enable=memory threadirqs i915.i915_enable_rc6=7 zcache zswap.enabled=1
>[    0.000000] Kernel command line: BOOT_IMAGE=/vmlinuz-3.11.0-rc1-tp520+ 
>root=/dev/mapper/merkaba-debian ro rootflags=subvol=root init=/bin/systemd 
>cgroup_enable=memory threadirqs i915.i915_enable_rc6=7 zcache zswap.enabled=1
>[    1.452443] zswap: loading zswap
>[    1.452465] zswap: using lzo compressor
>
>
>I did a stress -m 1 --vm-keep --vm-bytes 4G on this 8 GB ThinkPad T520 in 
>order to allocate some swap.
>

You can check /sys/kernel/debug/frontswap/succ_stores, if succ_stores is
0 it means that the memory pressure is still not heavy and none pages
need be swapped out.

>Still I think zswap didn't do anything:
>
>merkaba:/sys/kernel/debug/zswap> grep . *
>duplicate_entry:0
>pool_limit_hit:0
>pool_pages:0
>reject_alloc_fail:0
>reject_compress_poor:0
>reject_kmemcache_fail:0
>reject_reclaim_fail:0
>stored_pages:0
>written_back_pages:0
>
>
>However:
>
>merkaba:/sys/kernel/slab/zswap_entry> grep . *
>aliases:9
>align:8
>grep: alloc_calls: Die angeforderte Funktion ist nicht implementiert
>cache_dma:0
>cpu_partial:0
>cpu_slabs:4 N0=4
>destroy_by_rcu:0
>grep: free_calls: Die angeforderte Funktion ist nicht implementiert
>hwcache_align:0
>min_partial:5
>objects:2550 N0=2550
>object_size:48
>objects_partial:0
>objs_per_slab:85
>order:0
>partial:0
>poison:0
>reclaim_account:0
>red_zone:0
>remote_node_defrag_ratio:100
>reserved:0
>sanity_checks:0
>slabs:30 N0=30
>slabs_cpu_partial:0(0)
>slab_size:48
>store_user:0
>total_objects:2550 N0=2550
>trace:0
>
>It has some objects it seems.
>
>
>How do I know whether zswap actually does something?
>
>Will zswap work even with zcache enabled? As I understand zcache compresses 
>swap device pages on the block device level in addition to compressing read 
>cache pages of usual filesystems. Which one takes precedence, zcache or zswap? 
>Can I disable zcache for swap device?
>
>

zcache compression in file-cache and swap-cache layer.
zram compression in block layer.
zswap compression in swap-cache layer.

>
>Here is dmesg for zcache:
>
>martin@merkaba:~> dmesg | grep zcache
>[    0.000000] Command line: BOOT_IMAGE=/vmlinuz-3.11.0-rc1-tp520+ 
>root=/dev/mapper/merkaba-debian ro rootflags=subvol=root init=/bin/systemd 
>cgroup_enable=memory threadirqs i915.i915_enable_rc6=7 zcache zswap.enabled=1
>[    0.000000] Kernel command line: BOOT_IMAGE=/vmlinuz-3.11.0-rc1-tp520+ 
>root=/dev/mapper/merkaba-debian ro rootflags=subvol=root init=/bin/systemd 
>cgroup_enable=memory threadirqs i915.i915_enable_rc6=7 zcache zswap.enabled=1
>[    1.453531] zcache: using lzo compressor
>[    1.453634] zcache: cleancache enabled using kernel transcendent memory and 
>compression buddies
>[    1.453679] zcache: frontswap enabled using kernel transcendent memory and 
>compression buddies
>[    1.453722] zcache: frontswap_ops overridden
>[    5.358288] zcache: created ephemeral local tmem pool, id=0
>[    8.155684] zcache: created persistent local tmem pool, id=1
>[    8.331680] zcache: created ephemeral local tmem pool, id=2
>[    8.593235] zcache: created ephemeral local tmem pool, id=3
>[    8.743330] zcache: created ephemeral local tmem pool, id=4
>

This means zcache is configured for compressing file-cache pages and anonymous pages.

Regards,
Wanpeng Li 

>
>Thanks,
>-- 
>Martin 'Helios' Steigerwald - http://www.Lichtvoll.de
>GPG: 03B0 0D6C 0040 0710 4AFA  B82F 991B EAAC A599 84C7
>
>--
>To unsubscribe, send a message with 'unsubscribe linux-mm' in
>the body to majordomo@kvack.org.  For more info on Linux MM,
>see: http://www.linux-mm.org/ .
>Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: zswap: How to determine whether it is compressing swap pages?

[Wanpeng Li]

On Wed, Jul 17, 2013 at 12:04:38PM +0200, Martin Steigerwald wrote:
>Hi Seth, hi everyone,
>
>Yesterday I build 3.11-rc1 with CONFIG_ZSWAP and wanted to test it.
>
>I added zswap.enabled=1 and get:
>
>martin@merkaba:~> dmesg | grep zswap
>[    0.000000] Command line: BOOT_IMAGE=/vmlinuz-3.11.0-rc1-tp520+ 
>root=/dev/mapper/merkaba-debian ro rootflags=subvol=root init=/bin/systemd 
>cgroup_enable=memory threadirqs i915.i915_enable_rc6=7 zcache zswap.enabled=1
>[    0.000000] Kernel command line: BOOT_IMAGE=/vmlinuz-3.11.0-rc1-tp520+ 
>root=/dev/mapper/merkaba-debian ro rootflags=subvol=root init=/bin/systemd 
>cgroup_enable=memory threadirqs i915.i915_enable_rc6=7 zcache zswap.enabled=1
>[    1.452443] zswap: loading zswap
>[    1.452465] zswap: using lzo compressor
>
>
>I did a stress -m 1 --vm-keep --vm-bytes 4G on this 8 GB ThinkPad T520 in 
>order to allocate some swap.
>

You can check /sys/kernel/debug/frontswap/succ_stores, if succ_stores is
0 it means that the memory pressure is still not heavy and none pages
need be swapped out.

>Still I think zswap didn´t do anything:
>
>merkaba:/sys/kernel/debug/zswap> grep . *
>duplicate_entry:0
>pool_limit_hit:0
>pool_pages:0
>reject_alloc_fail:0
>reject_compress_poor:0
>reject_kmemcache_fail:0
>reject_reclaim_fail:0
>stored_pages:0
>written_back_pages:0
>
>
>However:
>
>merkaba:/sys/kernel/slab/zswap_entry> grep . *
>aliases:9
>align:8
>grep: alloc_calls: Die angeforderte Funktion ist nicht implementiert
>cache_dma:0
>cpu_partial:0
>cpu_slabs:4 N0=4
>destroy_by_rcu:0
>grep: free_calls: Die angeforderte Funktion ist nicht implementiert
>hwcache_align:0
>min_partial:5
>objects:2550 N0=2550
>object_size:48
>objects_partial:0
>objs_per_slab:85
>order:0
>partial:0
>poison:0
>reclaim_account:0
>red_zone:0
>remote_node_defrag_ratio:100
>reserved:0
>sanity_checks:0
>slabs:30 N0=30
>slabs_cpu_partial:0(0)
>slab_size:48
>store_user:0
>total_objects:2550 N0=2550
>trace:0
>
>It has some objects it seems.
>
>
>How do I know whether zswap actually does something?
>
>Will zswap work even with zcache enabled? As I understand zcache compresses 
>swap device pages on the block device level in addition to compressing read 
>cache pages of usual filesystems. Which one takes precedence, zcache or zswap? 
>Can I disable zcache for swap device?
>
>

zcache compression in file-cache and swap-cache layer.
zram compression in block layer.
zswap compression in swap-cache layer.

>
>Here is dmesg for zcache:
>
>martin@merkaba:~> dmesg | grep zcache
>[    0.000000] Command line: BOOT_IMAGE=/vmlinuz-3.11.0-rc1-tp520+ 
>root=/dev/mapper/merkaba-debian ro rootflags=subvol=root init=/bin/systemd 
>cgroup_enable=memory threadirqs i915.i915_enable_rc6=7 zcache zswap.enabled=1
>[    0.000000] Kernel command line: BOOT_IMAGE=/vmlinuz-3.11.0-rc1-tp520+ 
>root=/dev/mapper/merkaba-debian ro rootflags=subvol=root init=/bin/systemd 
>cgroup_enable=memory threadirqs i915.i915_enable_rc6=7 zcache zswap.enabled=1
>[    1.453531] zcache: using lzo compressor
>[    1.453634] zcache: cleancache enabled using kernel transcendent memory and 
>compression buddies
>[    1.453679] zcache: frontswap enabled using kernel transcendent memory and 
>compression buddies
>[    1.453722] zcache: frontswap_ops overridden
>[    5.358288] zcache: created ephemeral local tmem pool, id=0
>[    8.155684] zcache: created persistent local tmem pool, id=1
>[    8.331680] zcache: created ephemeral local tmem pool, id=2
>[    8.593235] zcache: created ephemeral local tmem pool, id=3
>[    8.743330] zcache: created ephemeral local tmem pool, id=4
>

This means zcache is configured for compressing file-cache pages and anonymous pages.

Regards,
Wanpeng Li 

>
>Thanks,
>-- 
>Martin 'Helios' Steigerwald - http://www.Lichtvoll.de
>GPG: 03B0 0D6C 0040 0710 4AFA  B82F 991B EAAC A599 84C7
>
>--
>To unsubscribe, send a message with 'unsubscribe linux-mm' in
>the body to majordomo@kvack.org.  For more info on Linux MM,
>see: http://www.linux-mm.org/ .
>Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: zswap: How to determine whether it is compressing swap pages?

[Bob Liu]

Hi Martin,

On 07/17/2013 06:04 PM, Martin Steigerwald wrote:
> Hi Seth, hi everyone,
>
> Yesterday I build 3.11-rc1 with CONFIG_ZSWAP and wanted to test it.
>
> I added zswap.enabled=1 and get:
>
> martin@merkaba:~> dmesg | grep zswap
> [    0.000000] Command line: BOOT_IMAGE=/vmlinuz-3.11.0-rc1-tp520+
> root=/dev/mapper/merkaba-debian ro rootflags=subvol=root init=/bin/systemd
> cgroup_enable=memory threadirqs i915.i915_enable_rc6=7 zcache zswap.enabled=1
> [    0.000000] Kernel command line: BOOT_IMAGE=/vmlinuz-3.11.0-rc1-tp520+
> root=/dev/mapper/merkaba-debian ro rootflags=subvol=root init=/bin/systemd
> cgroup_enable=memory threadirqs i915.i915_enable_rc6=7 zcache zswap.enabled=1
> [    1.452443] zswap: loading zswap
> [    1.452465] zswap: using lzo compressor
>
>
> I did a stress -m 1 --vm-keep --vm-bytes 4G on this 8 GB ThinkPad T520 in
> order to allocate some swap.
>

Thank you for your testing.
I'm glad to see there is new people interested with memory compression.

> Still I think zswap didn´t do anything:
>
> merkaba:/sys/kernel/debug/zswap> grep . *
> duplicate_entry:0
> pool_limit_hit:0
> pool_pages:0
> reject_alloc_fail:0
> reject_compress_poor:0
> reject_kmemcache_fail:0
> reject_reclaim_fail:0
> stored_pages:0
> written_back_pages:0
>
>
> However:
>
> merkaba:/sys/kernel/slab/zswap_entry> grep . *
> aliases:9
> align:8
> grep: alloc_calls: Die angeforderte Funktion ist nicht implementiert
> cache_dma:0
> cpu_partial:0
> cpu_slabs:4 N0=4
> destroy_by_rcu:0
> grep: free_calls: Die angeforderte Funktion ist nicht implementiert
> hwcache_align:0
> min_partial:5
> objects:2550 N0=2550
> object_size:48
> objects_partial:0
> objs_per_slab:85
> order:0
> partial:0
> poison:0
> reclaim_account:0
> red_zone:0
> remote_node_defrag_ratio:100
> reserved:0
> sanity_checks:0
> slabs:30 N0=30
> slabs_cpu_partial:0(0)
> slab_size:48
> store_user:0
> total_objects:2550 N0=2550
> trace:0
>
> It has some objects it seems.
>
>
> How do I know whether zswap actually does something?
>
> Will zswap work even with zcache enabled? As I understand zcache compresses
> swap device pages on the block device level in addition to compressing read
> cache pages of usual filesystems. Which one takes precedence, zcache or zswap?
> Can I disable zcache for swap device?
>

Please disable zcache and try again.

>
>
> Here is dmesg for zcache:
>
> martin@merkaba:~> dmesg | grep zcache
> [    0.000000] Command line: BOOT_IMAGE=/vmlinuz-3.11.0-rc1-tp520+
> root=/dev/mapper/merkaba-debian ro rootflags=subvol=root init=/bin/systemd
> cgroup_enable=memory threadirqs i915.i915_enable_rc6=7 zcache zswap.enabled=1
> [    0.000000] Kernel command line: BOOT_IMAGE=/vmlinuz-3.11.0-rc1-tp520+
> root=/dev/mapper/merkaba-debian ro rootflags=subvol=root init=/bin/systemd
> cgroup_enable=memory threadirqs i915.i915_enable_rc6=7 zcache zswap.enabled=1
> [    1.453531] zcache: using lzo compressor
> [    1.453634] zcache: cleancache enabled using kernel transcendent memory and
> compression buddies
> [    1.453679] zcache: frontswap enabled using kernel transcendent memory and
> compression buddies
> [    1.453722] zcache: frontswap_ops overridden
> [    5.358288] zcache: created ephemeral local tmem pool, id=0
> [    8.155684] zcache: created persistent local tmem pool, id=1
> [    8.331680] zcache: created ephemeral local tmem pool, id=2
> [    8.593235] zcache: created ephemeral local tmem pool, id=3
> [    8.743330] zcache: created ephemeral local tmem pool, id=4
>
>
> Thanks,
>

-- 
Regards,
-Bob

Re: zswap: How to determine whether it is compressing swap pages?

[Bob Liu]

Hi Martin,

On 07/17/2013 06:04 PM, Martin Steigerwald wrote:
> Hi Seth, hi everyone,
>
> Yesterday I build 3.11-rc1 with CONFIG_ZSWAP and wanted to test it.
>
> I added zswap.enabled=1 and get:
>
> martin@merkaba:~> dmesg | grep zswap
> [    0.000000] Command line: BOOT_IMAGE=/vmlinuz-3.11.0-rc1-tp520+
> root=/dev/mapper/merkaba-debian ro rootflags=subvol=root init=/bin/systemd
> cgroup_enable=memory threadirqs i915.i915_enable_rc6=7 zcache zswap.enabled=1
> [    0.000000] Kernel command line: BOOT_IMAGE=/vmlinuz-3.11.0-rc1-tp520+
> root=/dev/mapper/merkaba-debian ro rootflags=subvol=root init=/bin/systemd
> cgroup_enable=memory threadirqs i915.i915_enable_rc6=7 zcache zswap.enabled=1
> [    1.452443] zswap: loading zswap
> [    1.452465] zswap: using lzo compressor
>
>
> I did a stress -m 1 --vm-keep --vm-bytes 4G on this 8 GB ThinkPad T520 in
> order to allocate some swap.
>

Thank you for your testing.
I'm glad to see there is new people interested with memory compression.

> Still I think zswap didn't do anything:
>
> merkaba:/sys/kernel/debug/zswap> grep . *
> duplicate_entry:0
> pool_limit_hit:0
> pool_pages:0
> reject_alloc_fail:0
> reject_compress_poor:0
> reject_kmemcache_fail:0
> reject_reclaim_fail:0
> stored_pages:0
> written_back_pages:0
>
>
> However:
>
> merkaba:/sys/kernel/slab/zswap_entry> grep . *
> aliases:9
> align:8
> grep: alloc_calls: Die angeforderte Funktion ist nicht implementiert
> cache_dma:0
> cpu_partial:0
> cpu_slabs:4 N0=4
> destroy_by_rcu:0
> grep: free_calls: Die angeforderte Funktion ist nicht implementiert
> hwcache_align:0
> min_partial:5
> objects:2550 N0=2550
> object_size:48
> objects_partial:0
> objs_per_slab:85
> order:0
> partial:0
> poison:0
> reclaim_account:0
> red_zone:0
> remote_node_defrag_ratio:100
> reserved:0
> sanity_checks:0
> slabs:30 N0=30
> slabs_cpu_partial:0(0)
> slab_size:48
> store_user:0
> total_objects:2550 N0=2550
> trace:0
>
> It has some objects it seems.
>
>
> How do I know whether zswap actually does something?
>
> Will zswap work even with zcache enabled? As I understand zcache compresses
> swap device pages on the block device level in addition to compressing read
> cache pages of usual filesystems. Which one takes precedence, zcache or zswap?
> Can I disable zcache for swap device?
>

Please disable zcache and try again.

>
>
> Here is dmesg for zcache:
>
> martin@merkaba:~> dmesg | grep zcache
> [    0.000000] Command line: BOOT_IMAGE=/vmlinuz-3.11.0-rc1-tp520+
> root=/dev/mapper/merkaba-debian ro rootflags=subvol=root init=/bin/systemd
> cgroup_enable=memory threadirqs i915.i915_enable_rc6=7 zcache zswap.enabled=1
> [    0.000000] Kernel command line: BOOT_IMAGE=/vmlinuz-3.11.0-rc1-tp520+
> root=/dev/mapper/merkaba-debian ro rootflags=subvol=root init=/bin/systemd
> cgroup_enable=memory threadirqs i915.i915_enable_rc6=7 zcache zswap.enabled=1
> [    1.453531] zcache: using lzo compressor
> [    1.453634] zcache: cleancache enabled using kernel transcendent memory and
> compression buddies
> [    1.453679] zcache: frontswap enabled using kernel transcendent memory and
> compression buddies
> [    1.453722] zcache: frontswap_ops overridden
> [    5.358288] zcache: created ephemeral local tmem pool, id=0
> [    8.155684] zcache: created persistent local tmem pool, id=1
> [    8.331680] zcache: created ephemeral local tmem pool, id=2
> [    8.593235] zcache: created ephemeral local tmem pool, id=3
> [    8.743330] zcache: created ephemeral local tmem pool, id=4
>
>
> Thanks,
>

-- 
Regards,
-Bob

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: zswap: How to determine whether it is compressing swap pages?

[Martin Steigerwald]

Am Mittwoch, 17. Juli 2013, 18:42:18 schrieb Bob Liu:
> On 07/17/2013 06:04 PM, Martin Steigerwald wrote:
> > Hi Seth, hi everyone,
> >
> > Yesterday I build 3.11-rc1 with CONFIG_ZSWAP and wanted to test it.
> >
> > I added zswap.enabled=1 and get:
> >
> > martin@merkaba:~> dmesg | grep zswap
> > [    0.000000] Command line: BOOT_IMAGE=/vmlinuz-3.11.0-rc1-tp520+
> > root=/dev/mapper/merkaba-debian ro rootflags=subvol=root init=/bin/systemd
> > cgroup_enable=memory threadirqs i915.i915_enable_rc6=7 zcache zswap.enabled=1
> > [    0.000000] Kernel command line: BOOT_IMAGE=/vmlinuz-3.11.0-rc1-tp520+
> > root=/dev/mapper/merkaba-debian ro rootflags=subvol=root init=/bin/systemd
> > cgroup_enable=memory threadirqs i915.i915_enable_rc6=7 zcache zswap.enabled=1
> > [    1.452443] zswap: loading zswap
> > [    1.452465] zswap: using lzo compressor
> >
> >
> > I did a stress -m 1 --vm-keep --vm-bytes 4G on this 8 GB ThinkPad T520 in
> > order to allocate some swap.
> >
> 
> Thank you for your testing.
> I'm glad to see there is new people interested with memory compression.
> 
> > Still I think zswap didn´t do anything:
> >
> > merkaba:/sys/kernel/debug/zswap> grep . *
> > duplicate_entry:0
> > pool_limit_hit:0
> > pool_pages:0
> > reject_alloc_fail:0
> > reject_compress_poor:0
> > reject_kmemcache_fail:0
> > reject_reclaim_fail:0
> > stored_pages:0
> > written_back_pages:0
> >
> >
> > However:
> >
> > merkaba:/sys/kernel/slab/zswap_entry> grep . *
> > aliases:9
> > align:8
> > grep: alloc_calls: Die angeforderte Funktion ist nicht implementiert
> > cache_dma:0
> > cpu_partial:0
> > cpu_slabs:4 N0=4
> > destroy_by_rcu:0
> > grep: free_calls: Die angeforderte Funktion ist nicht implementiert
> > hwcache_align:0
> > min_partial:5
> > objects:2550 N0=2550
> > object_size:48
> > objects_partial:0
> > objs_per_slab:85
> > order:0
> > partial:0
> > poison:0
> > reclaim_account:0
> > red_zone:0
> > remote_node_defrag_ratio:100
> > reserved:0
> > sanity_checks:0
> > slabs:30 N0=30
> > slabs_cpu_partial:0(0)
> > slab_size:48
> > store_user:0
> > total_objects:2550 N0=2550
> > trace:0
> >
> > It has some objects it seems.
> >
> >
> > How do I know whether zswap actually does something?
> >
> > Will zswap work even with zcache enabled? As I understand zcache compresses
> > swap device pages on the block device level in addition to compressing read
> > cache pages of usual filesystems. Which one takes precedence, zcache or zswap?
> > Can I disable zcache for swap device?
> >
> 
> Please disable zcache and try again.

Okay, this seemed to work.

Shortly after starting stress I got:

merkaba:/sys/kernel/debug/zswap> grep . *
duplicate_entry:0
pool_limit_hit:0
pool_pages:170892
reject_alloc_fail:0
reject_compress_poor:0
reject_kmemcache_fail:0
reject_reclaim_fail:0
stored_pages:341791
written_back_pages:0


then zcache reduced pool size again – while stress was still running:

merkaba:/sys/kernel/debug/zswap> grep . *
duplicate_entry:0
pool_limit_hit:0
pool_pages:38
reject_alloc_fail:0
reject_compress_poor:0
reject_kmemcache_fail:0
reject_reclaim_fail:0
stored_pages:66
written_back_pages:0


I assume that on heavy memory pressure zcache shrinks pool again in oder
to free memory for other activities? Is that correct?

So zswap would help most on moderate, not heavy and bulky memory pressure?


I was not able to reproduce above behavior even while watching with

merkaba:/sys/kernel/debug/zswap#130> while true; do date; grep . * ; sleep 1 ; done


Zswap just doesn´t seem to store packages on that workload anymore.

I will keep it running in regular workloads (two KDE sessions with Akonadi
and Nepomuk) and observe it a bit.


Is there any way to run zcache concurrently with zswap? I.e. use zcache only
for read caches for filesystem and zswap for swap?

What is better suited for swap? zswap or zcache?

Thanks,
-- 
Martin 'Helios' Steigerwald - http://www.Lichtvoll.de
GPG: 03B0 0D6C 0040 0710 4AFA  B82F 991B EAAC A599 84C7

Re: zswap: How to determine whether it is compressing swap pages?

[Martin Steigerwald]

Am Mittwoch, 17. Juli 2013, 18:42:18 schrieb Bob Liu:
> On 07/17/2013 06:04 PM, Martin Steigerwald wrote:
> > Hi Seth, hi everyone,
> >
> > Yesterday I build 3.11-rc1 with CONFIG_ZSWAP and wanted to test it.
> >
> > I added zswap.enabled=1 and get:
> >
> > martin@merkaba:~> dmesg | grep zswap
> > [    0.000000] Command line: BOOT_IMAGE=/vmlinuz-3.11.0-rc1-tp520+
> > root=/dev/mapper/merkaba-debian ro rootflags=subvol=root init=/bin/systemd
> > cgroup_enable=memory threadirqs i915.i915_enable_rc6=7 zcache zswap.enabled=1
> > [    0.000000] Kernel command line: BOOT_IMAGE=/vmlinuz-3.11.0-rc1-tp520+
> > root=/dev/mapper/merkaba-debian ro rootflags=subvol=root init=/bin/systemd
> > cgroup_enable=memory threadirqs i915.i915_enable_rc6=7 zcache zswap.enabled=1
> > [    1.452443] zswap: loading zswap
> > [    1.452465] zswap: using lzo compressor
> >
> >
> > I did a stress -m 1 --vm-keep --vm-bytes 4G on this 8 GB ThinkPad T520 in
> > order to allocate some swap.
> >
> 
> Thank you for your testing.
> I'm glad to see there is new people interested with memory compression.
> 
> > Still I think zswap didn´t do anything:
> >
> > merkaba:/sys/kernel/debug/zswap> grep . *
> > duplicate_entry:0
> > pool_limit_hit:0
> > pool_pages:0
> > reject_alloc_fail:0
> > reject_compress_poor:0
> > reject_kmemcache_fail:0
> > reject_reclaim_fail:0
> > stored_pages:0
> > written_back_pages:0
> >
> >
> > However:
> >
> > merkaba:/sys/kernel/slab/zswap_entry> grep . *
> > aliases:9
> > align:8
> > grep: alloc_calls: Die angeforderte Funktion ist nicht implementiert
> > cache_dma:0
> > cpu_partial:0
> > cpu_slabs:4 N0=4
> > destroy_by_rcu:0
> > grep: free_calls: Die angeforderte Funktion ist nicht implementiert
> > hwcache_align:0
> > min_partial:5
> > objects:2550 N0=2550
> > object_size:48
> > objects_partial:0
> > objs_per_slab:85
> > order:0
> > partial:0
> > poison:0
> > reclaim_account:0
> > red_zone:0
> > remote_node_defrag_ratio:100
> > reserved:0
> > sanity_checks:0
> > slabs:30 N0=30
> > slabs_cpu_partial:0(0)
> > slab_size:48
> > store_user:0
> > total_objects:2550 N0=2550
> > trace:0
> >
> > It has some objects it seems.
> >
> >
> > How do I know whether zswap actually does something?
> >
> > Will zswap work even with zcache enabled? As I understand zcache compresses
> > swap device pages on the block device level in addition to compressing read
> > cache pages of usual filesystems. Which one takes precedence, zcache or zswap?
> > Can I disable zcache for swap device?
> >
> 
> Please disable zcache and try again.

Okay, this seemed to work.

Shortly after starting stress I got:

merkaba:/sys/kernel/debug/zswap> grep . *
duplicate_entry:0
pool_limit_hit:0
pool_pages:170892
reject_alloc_fail:0
reject_compress_poor:0
reject_kmemcache_fail:0
reject_reclaim_fail:0
stored_pages:341791
written_back_pages:0


then zcache reduced pool size again – while stress was still running:

merkaba:/sys/kernel/debug/zswap> grep . *
duplicate_entry:0
pool_limit_hit:0
pool_pages:38
reject_alloc_fail:0
reject_compress_poor:0
reject_kmemcache_fail:0
reject_reclaim_fail:0
stored_pages:66
written_back_pages:0


I assume that on heavy memory pressure zcache shrinks pool again in oder
to free memory for other activities? Is that correct?

So zswap would help most on moderate, not heavy and bulky memory pressure?


I was not able to reproduce above behavior even while watching with

merkaba:/sys/kernel/debug/zswap#130> while true; do date; grep . * ; sleep 1 ; done


Zswap just doesn´t seem to store packages on that workload anymore.

I will keep it running in regular workloads (two KDE sessions with Akonadi
and Nepomuk) and observe it a bit.


Is there any way to run zcache concurrently with zswap? I.e. use zcache only
for read caches for filesystem and zswap for swap?

What is better suited for swap? zswap or zcache?

Thanks,
-- 
Martin 'Helios' Steigerwald - http://www.Lichtvoll.de
GPG: 03B0 0D6C 0040 0710 4AFA  B82F 991B EAAC A599 84C7

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: zswap: How to determine whether it is compressing swap pages?

[Martin Steigerwald]

Am Mittwoch, 17. Juli 2013, 09:38:34 schrieb Seth Jennings:
> On Wed, Jul 17, 2013 at 01:41:44PM +0200, Martin Steigerwald wrote:
> > Is there any way to run zcache concurrently with zswap? I.e. use zcache only
> > for read caches for filesystem and zswap for swap?
> 
> No, at least not with zcache's frontswap features enabled.  frontswap is a very
> simple API that allows only one "backend" to register with it at a time.  So
> that means _either_ zswap or zcache.
> 
> The only way they can be used in a meaningful way together is to use the
> "nofrontswap" zcache option in the kernel boot parameters to prevent
> zcache overriding zswap's frontswap registration.
> 
> But the general answer is no, they shouldn't be used together.
>
> 
> > 
> > What is better suited for swap? zswap or zcache?
> 
> zswap targets the specific case of caching swapped out pages in a compressed
> cache and this is much simpler than zcache. zswap is also in mainline as of
> 3.11-rc1.

Thanks.

Okay, then I will test zswap for now. I have a nice use case for it: Playing
PlaneShift while a full KDE session is open with 8 GB of RAM. The PlaneShift
client easily takes 2 GB RSS and to complicate matters I think there is even
a mem leak either in Intel Mesa driver or in PS client. zswap may not help
much with that I think. This brought down my laptop several times with a
storm to swap which locked the machine - no mouse movements possible -
for minutes while using the SSD like wild (LED constantly lid).

Currently I see zswap did some work:

merkaba:/sys/kernel/debug/zswap> grep . *                  
duplicate_entry:0
pool_limit_hit:0
pool_pages:14565
reject_alloc_fail:0
reject_compress_poor:1905
reject_kmemcache_fail:0
reject_reclaim_fail:0
stored_pages:29092
written_back_pages:0

About a hour later:

merkaba:/sys/kernel/debug/zswap> grep . *                
duplicate_entry:0
pool_limit_hit:0
pool_pages:18924
reject_alloc_fail:0
reject_compress_poor:1907
reject_kmemcache_fail:0
reject_reclaim_fail:0
stored_pages:37820
written_back_pages:0

> zcache, a driver in the staging tree, is much more complex offers some other
> functionality like compressed page/file cache for certain filesystems using
> cleancache and a remote-RAM system called RAMster.

I´d be interested in the cleancache stuff, but I wonder whether it would make
much of a difference with a desktop workload.

Anyway, for a while I focus on testing zswap.

-- 
Martin 'Helios' Steigerwald - http://www.Lichtvoll.de
GPG: 03B0 0D6C 0040 0710 4AFA  B82F 991B EAAC A599 84C7

Re: zswap: How to determine whether it is compressing swap pages?

[Martin Steigerwald]

Am Mittwoch, 17. Juli 2013, 09:38:34 schrieb Seth Jennings:
> On Wed, Jul 17, 2013 at 01:41:44PM +0200, Martin Steigerwald wrote:
> > Is there any way to run zcache concurrently with zswap? I.e. use zcache only
> > for read caches for filesystem and zswap for swap?
> 
> No, at least not with zcache's frontswap features enabled.  frontswap is a very
> simple API that allows only one "backend" to register with it at a time.  So
> that means _either_ zswap or zcache.
> 
> The only way they can be used in a meaningful way together is to use the
> "nofrontswap" zcache option in the kernel boot parameters to prevent
> zcache overriding zswap's frontswap registration.
> 
> But the general answer is no, they shouldn't be used together.
>
> 
> > 
> > What is better suited for swap? zswap or zcache?
> 
> zswap targets the specific case of caching swapped out pages in a compressed
> cache and this is much simpler than zcache. zswap is also in mainline as of
> 3.11-rc1.

Thanks.

Okay, then I will test zswap for now. I have a nice use case for it: Playing
PlaneShift while a full KDE session is open with 8 GB of RAM. The PlaneShift
client easily takes 2 GB RSS and to complicate matters I think there is even
a mem leak either in Intel Mesa driver or in PS client. zswap may not help
much with that I think. This brought down my laptop several times with a
storm to swap which locked the machine - no mouse movements possible -
for minutes while using the SSD like wild (LED constantly lid).

Currently I see zswap did some work:

merkaba:/sys/kernel/debug/zswap> grep . *                  
duplicate_entry:0
pool_limit_hit:0
pool_pages:14565
reject_alloc_fail:0
reject_compress_poor:1905
reject_kmemcache_fail:0
reject_reclaim_fail:0
stored_pages:29092
written_back_pages:0

About a hour later:

merkaba:/sys/kernel/debug/zswap> grep . *                
duplicate_entry:0
pool_limit_hit:0
pool_pages:18924
reject_alloc_fail:0
reject_compress_poor:1907
reject_kmemcache_fail:0
reject_reclaim_fail:0
stored_pages:37820
written_back_pages:0

> zcache, a driver in the staging tree, is much more complex offers some other
> functionality like compressed page/file cache for certain filesystems using
> cleancache and a remote-RAM system called RAMster.

I´d be interested in the cleancache stuff, but I wonder whether it would make
much of a difference with a desktop workload.

Anyway, for a while I focus on testing zswap.

-- 
Martin 'Helios' Steigerwald - http://www.Lichtvoll.de
GPG: 03B0 0D6C 0040 0710 4AFA  B82F 991B EAAC A599 84C7

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: zswap: How to determine whether it is compressing swap pages?

[Bob Liu]

Hi Martin,

On 07/18/2013 03:38 AM, Martin Steigerwald wrote:
> Am Mittwoch, 17. Juli 2013, 09:38:34 schrieb Seth Jennings:
>> On Wed, Jul 17, 2013 at 01:41:44PM +0200, Martin Steigerwald wrote:
>>> Is there any way to run zcache concurrently with zswap? I.e. use zcache only
>>> for read caches for filesystem and zswap for swap?
>>
>> No, at least not with zcache's frontswap features enabled.  frontswap is a very
>> simple API that allows only one "backend" to register with it at a time.  So
>> that means _either_ zswap or zcache.
>>
>> The only way they can be used in a meaningful way together is to use the
>> "nofrontswap" zcache option in the kernel boot parameters to prevent
>> zcache overriding zswap's frontswap registration.
>>
>> But the general answer is no, they shouldn't be used together.
>>
>>
>>>
>>> What is better suited for swap? zswap or zcache?
>>
>> zswap targets the specific case of caching swapped out pages in a compressed
>> cache and this is much simpler than zcache. zswap is also in mainline as of
>> 3.11-rc1.
> 
> Thanks.
> 
> Okay, then I will test zswap for now. I have a nice use case for it: Playing

Could you make some test by kernel compiling? Something like kernbench.
During my testing, I found that the swap ins/outs operations reduced but
the kernel compile time didn't reduce accordingly.

-- 
Regards,
-Bob

Re: zswap: How to determine whether it is compressing swap pages?

[Bob Liu]

Hi Martin,

On 07/18/2013 03:38 AM, Martin Steigerwald wrote:
> Am Mittwoch, 17. Juli 2013, 09:38:34 schrieb Seth Jennings:
>> On Wed, Jul 17, 2013 at 01:41:44PM +0200, Martin Steigerwald wrote:
>>> Is there any way to run zcache concurrently with zswap? I.e. use zcache only
>>> for read caches for filesystem and zswap for swap?
>>
>> No, at least not with zcache's frontswap features enabled.  frontswap is a very
>> simple API that allows only one "backend" to register with it at a time.  So
>> that means _either_ zswap or zcache.
>>
>> The only way they can be used in a meaningful way together is to use the
>> "nofrontswap" zcache option in the kernel boot parameters to prevent
>> zcache overriding zswap's frontswap registration.
>>
>> But the general answer is no, they shouldn't be used together.
>>
>>
>>>
>>> What is better suited for swap? zswap or zcache?
>>
>> zswap targets the specific case of caching swapped out pages in a compressed
>> cache and this is much simpler than zcache. zswap is also in mainline as of
>> 3.11-rc1.
> 
> Thanks.
> 
> Okay, then I will test zswap for now. I have a nice use case for it: Playing

Could you make some test by kernel compiling? Something like kernbench.
During my testing, I found that the swap ins/outs operations reduced but
the kernel compile time didn't reduce accordingly.

-- 
Regards,
-Bob

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: zswap: How to determine whether it is compressing swap pages?

[Valdis.Kletnieks]

[-- Attachment #1: Type: text/plain, Size: 405 bytes --]

On Thu, 18 Jul 2013 20:43:40 +0800, Bob Liu said:

> Could you make some test by kernel compiling? Something like kernbench.
> During my testing, I found that the swap ins/outs operations reduced but
> the kernel compile time didn't reduce accordingly.

If your kernel source tree is cache-cold, the swap in/out activity is
probably hidden and ahrd to notice among all the disk I/O to read the source in.

[-- Attachment #2: Type: application/pgp-signature, Size: 865 bytes --]