From: ahferroin7@gmail.com (Austin S. Hemmelgarn)
To: kernelnewbies@lists.kernelnewbies.org
Subject: Creating compressed backing_store as swapfile
Date: Mon, 5 Nov 2018 11:07:12 -0500 [thread overview]
Message-ID: <79d0c96a-a0a2-63ec-db91-42fd349d50c1@gmail.com> (raw)
In-Reply-To: <20181105155815.i654i5ctmfpqhggj@angband.pl>
On 11/5/2018 10:58 AM, Adam Borowski wrote:
> On Mon, Nov 05, 2018 at 08:31:46PM +0530, Pintu Agarwal wrote:
>> Hi,
>>
>> I have one requirement:
>> I wanted to have a swapfile (64MB to 256MB) on my system.
>> But I wanted the data to be compressed and stored on the disk in my swapfile.
>> [Similar to zram, but compressed data should be moved to disk, instead of RAM].
>>
>> Note: I wanted to optimize RAM space, so performance is not important
>> right now for our requirement.
>>
>> So, what are the options available, to perform this in 4.x kernel version.
>> My Kernel: 4.9.x
>> Board: any - (arm64 mostly).
>>
>> As I know, following are the choices:
>> 1) ZRAM: But it compresses and store data in RAM itself
>> 2) frontswap + zswap : Didn't explore much on this, not sure if this
>> is helpful for our case.
>> 3) Manually creating swapfile: but how to compress it ?
>> 4) Any other options ?
>
> Loop device on any filesystem that can compress (such as btrfs)? The
> performance would suck, though -- besides the indirection of loop, btrfs
> compresses in blocks of 128KB while swap wants 4KB writes. Other similar
> option is qemu-nbd -- it can use compressed disk images and expose them to a
> (local) nbd client.
Swap on any type of a networked storage device (NBD, iSCSI, ATAoE, etc)
served from the local system is _really_ risky. The moment the local
server process for the storage device gets forced out to swap, you deadlock.
Performance isn't _too_ bad for the BTRFS case though (I've actually
tested this before), just make sure you disable direct I/O mode on the
loop device, otherwise you run the risk of data corruption.
WARNING: multiple messages have this Message-ID (diff)
From: "Austin S. Hemmelgarn" <ahferroin7@gmail.com>
To: Adam Borowski <kilobyte@angband.pl>,
Pintu Agarwal <pintu.ping@gmail.com>
Cc: linux-mm@kvack.org, open list <linux-kernel@vger.kernel.org>,
kernelnewbies@kernelnewbies.org
Subject: Re: Creating compressed backing_store as swapfile
Date: Mon, 5 Nov 2018 11:07:12 -0500 [thread overview]
Message-ID: <79d0c96a-a0a2-63ec-db91-42fd349d50c1@gmail.com> (raw)
Message-ID: <20181105160712.fTt_JKZId18um_FmrfGQNq5BvrcXCDq5aZ1clZonNhQ@z> (raw)
In-Reply-To: <20181105155815.i654i5ctmfpqhggj@angband.pl>
On 11/5/2018 10:58 AM, Adam Borowski wrote:
> On Mon, Nov 05, 2018 at 08:31:46PM +0530, Pintu Agarwal wrote:
>> Hi,
>>
>> I have one requirement:
>> I wanted to have a swapfile (64MB to 256MB) on my system.
>> But I wanted the data to be compressed and stored on the disk in my swapfile.
>> [Similar to zram, but compressed data should be moved to disk, instead of RAM].
>>
>> Note: I wanted to optimize RAM space, so performance is not important
>> right now for our requirement.
>>
>> So, what are the options available, to perform this in 4.x kernel version.
>> My Kernel: 4.9.x
>> Board: any - (arm64 mostly).
>>
>> As I know, following are the choices:
>> 1) ZRAM: But it compresses and store data in RAM itself
>> 2) frontswap + zswap : Didn't explore much on this, not sure if this
>> is helpful for our case.
>> 3) Manually creating swapfile: but how to compress it ?
>> 4) Any other options ?
>
> Loop device on any filesystem that can compress (such as btrfs)? The
> performance would suck, though -- besides the indirection of loop, btrfs
> compresses in blocks of 128KB while swap wants 4KB writes. Other similar
> option is qemu-nbd -- it can use compressed disk images and expose them to a
> (local) nbd client.
Swap on any type of a networked storage device (NBD, iSCSI, ATAoE, etc)
served from the local system is _really_ risky. The moment the local
server process for the storage device gets forced out to swap, you deadlock.
Performance isn't _too_ bad for the BTRFS case though (I've actually
tested this before), just make sure you disable direct I/O mode on the
loop device, otherwise you run the risk of data corruption.
_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
next prev parent reply other threads:[~2018-11-05 16:07 UTC|newest]
Thread overview: 18+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-11-05 15:01 Creating compressed backing_store as swapfile Pintu Agarwal
2018-11-05 15:01 ` Pintu Agarwal
[not found] ` <20181105155815.i654i5ctmfpqhggj@angband.pl>
2018-11-05 16:07 ` Austin S. Hemmelgarn [this message]
2018-11-05 16:07 ` Austin S. Hemmelgarn
2018-11-05 16:14 ` valdis.kletnieks at vt.edu
2018-11-05 16:14 ` valdis.kletnieks
2018-11-05 16:28 ` Austin S. Hemmelgarn
2018-11-05 16:28 ` Austin S. Hemmelgarn
2018-11-05 16:53 ` valdis.kletnieks at vt.edu
2018-11-05 16:53 ` valdis.kletnieks
2018-11-05 16:55 ` Austin S. Hemmelgarn
2018-11-05 16:55 ` Austin S. Hemmelgarn
2018-11-08 9:51 ` Pintu Agarwal
2018-11-08 9:51 ` Pintu Agarwal
2018-11-05 16:12 ` valdis.kletnieks at vt.edu
2018-11-05 16:12 ` valdis.kletnieks
2018-11-08 9:46 ` Pintu Agarwal
2018-11-08 9:46 ` Pintu Agarwal
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=79d0c96a-a0a2-63ec-db91-42fd349d50c1@gmail.com \
--to=ahferroin7@gmail.com \
--cc=kernelnewbies@lists.kernelnewbies.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).