From mboxrd@z Thu Jan 1 00:00:00 1970 From: pintu.ping@gmail.com (Pintu Agarwal) Date: Thu, 8 Nov 2018 15:21:58 +0530 Subject: Creating compressed backing_store as swapfile In-Reply-To: <79d0c96a-a0a2-63ec-db91-42fd349d50c1@gmail.com> References: <20181105155815.i654i5ctmfpqhggj@angband.pl> <79d0c96a-a0a2-63ec-db91-42fd349d50c1@gmail.com> Message-ID: To: kernelnewbies@lists.kernelnewbies.org List-Id: kernelnewbies.lists.kernelnewbies.org On Mon, Nov 5, 2018 at 9:37 PM Austin S. Hemmelgarn wrote: > > On 11/5/2018 10:58 AM, Adam Borowski wrote: > > On Mon, Nov 05, 2018 at 08:31:46PM +0530, Pintu Agarwal wrote: > >> Hi, > >> > >> I have one requirement: > >> I wanted to have a swapfile (64MB to 256MB) on my system. > >> But I wanted the data to be compressed and stored on the disk in my swapfile. > >> [Similar to zram, but compressed data should be moved to disk, instead of RAM]. > >> > >> Note: I wanted to optimize RAM space, so performance is not important > >> right now for our requirement. > >> > >> So, what are the options available, to perform this in 4.x kernel version. > >> My Kernel: 4.9.x > >> Board: any - (arm64 mostly). > >> > >> As I know, following are the choices: > >> 1) ZRAM: But it compresses and store data in RAM itself > >> 2) frontswap + zswap : Didn't explore much on this, not sure if this > >> is helpful for our case. > >> 3) Manually creating swapfile: but how to compress it ? > >> 4) Any other options ? > > > > Loop device on any filesystem that can compress (such as btrfs)? The > > performance would suck, though -- besides the indirection of loop, btrfs > > compresses in blocks of 128KB while swap wants 4KB writes. Other similar > > option is qemu-nbd -- it can use compressed disk images and expose them to a > > (local) nbd client. > > Swap on any type of a networked storage device (NBD, iSCSI, ATAoE, etc) > served from the local system is _really_ risky. The moment the local > server process for the storage device gets forced out to swap, you deadlock. > > Performance isn't _too_ bad for the BTRFS case though (I've actually > tested this before), just make sure you disable direct I/O mode on the > loop device, otherwise you run the risk of data corruption. Sorry, btrfs is not an option for us. We want something more lighter weight as our requirement is just < 200 MBs. From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-vs1-xe43.google.com ([2607:f8b0:4864:20::e43]) by shelob.surriel.com with esmtps (TLSv1.2:ECDHE-RSA-AES128-GCM-SHA256:128) (Exim 4.91) (envelope-from ) id 1gKgyw-0005FE-HH for kernelnewbies@kernelnewbies.org; Thu, 08 Nov 2018 04:52:10 -0500 Received: by mail-vs1-xe43.google.com with SMTP id x64so11221362vsa.5 for ; Thu, 08 Nov 2018 01:52:10 -0800 (PST) MIME-Version: 1.0 References: <20181105155815.i654i5ctmfpqhggj@angband.pl> <79d0c96a-a0a2-63ec-db91-42fd349d50c1@gmail.com> In-Reply-To: <79d0c96a-a0a2-63ec-db91-42fd349d50c1@gmail.com> From: Pintu Agarwal Date: Thu, 8 Nov 2018 15:21:58 +0530 Message-ID: Subject: Re: Creating compressed backing_store as swapfile To: ahferroin7@gmail.com Cc: kilobyte@angband.pl, kernelnewbies@kernelnewbies.org, open list , linux-mm@kvack.org List-Id: Learn about the Linux kernel List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: kernelnewbies-bounces@kernelnewbies.org Message-ID: <20181108095158.3uNU00tdURfASpRsR_udAAWGjfdRbYIYemsR5sd0JFk@z> On Mon, Nov 5, 2018 at 9:37 PM Austin S. Hemmelgarn wrote: > > On 11/5/2018 10:58 AM, Adam Borowski wrote: > > On Mon, Nov 05, 2018 at 08:31:46PM +0530, Pintu Agarwal wrote: > >> Hi, > >> > >> I have one requirement: > >> I wanted to have a swapfile (64MB to 256MB) on my system. > >> But I wanted the data to be compressed and stored on the disk in my swapfile. > >> [Similar to zram, but compressed data should be moved to disk, instead of RAM]. > >> > >> Note: I wanted to optimize RAM space, so performance is not important > >> right now for our requirement. > >> > >> So, what are the options available, to perform this in 4.x kernel version. > >> My Kernel: 4.9.x > >> Board: any - (arm64 mostly). > >> > >> As I know, following are the choices: > >> 1) ZRAM: But it compresses and store data in RAM itself > >> 2) frontswap + zswap : Didn't explore much on this, not sure if this > >> is helpful for our case. > >> 3) Manually creating swapfile: but how to compress it ? > >> 4) Any other options ? > > > > Loop device on any filesystem that can compress (such as btrfs)? The > > performance would suck, though -- besides the indirection of loop, btrfs > > compresses in blocks of 128KB while swap wants 4KB writes. Other similar > > option is qemu-nbd -- it can use compressed disk images and expose them to a > > (local) nbd client. > > Swap on any type of a networked storage device (NBD, iSCSI, ATAoE, etc) > served from the local system is _really_ risky. The moment the local > server process for the storage device gets forced out to swap, you deadlock. > > Performance isn't _too_ bad for the BTRFS case though (I've actually > tested this before), just make sure you disable direct I/O mode on the > loop device, otherwise you run the risk of data corruption. Sorry, btrfs is not an option for us. We want something more lighter weight as our requirement is just < 200 MBs. _______________________________________________ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies