From: Sergey Senozhatsky <senozhatsky@chromium.org> To: Ian Wienand <iwienand@redhat.com> Cc: Minchan Kim <minchan@kernel.org>, Petr Vorel <pvorel@suse.cz>, ltp@lists.linux.it, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, Nitin Gupta <ngupta@vflare.org>, Sergey Senozhatsky <senozhatsky@chromium.org>, Jens Axboe <axboe@kernel.dk>, OGAWA Hirofumi <hirofumi@mail.parknet.co.jp>, Martin Doucha <mdoucha@suse.cz>, Yang Xu <xuyang2018.jy@fujitsu.com> Subject: Re: [PATCH 0/1] Possible bug in zram on ppc64le on vfat Date: Mon, 7 Aug 2023 14:19:17 +0900 [thread overview] Message-ID: <20230807051917.GC907732@google.com> (raw) In-Reply-To: <ZNB2kORYiKdl3vSq@fedora19.localdomain> On (23/08/07 14:44), Ian Wienand wrote: [..] > > At this point, because this test fills from /dev/zero, the zsmalloc > pool doesn't actually have anything in it. The filesystem metadata is > in-use from the writes, and is not written out as compressed data. > The zram page de-duplication has kicked in, and instead of handles to > zsmalloc areas for data we just have "this is a page of zeros" > recorded. So this is correctly reflecting that fact that we don't > actually have anything compressed stored at this time. > > > >> If we do a "sync" then redisply the mm_stat after, we get > > >> 26214400 2842 65536 26214400 196608 399 0 0 > > Now we've finished writing all our zeros and have synced, we would > have finished updating vfat allocations, etc. So this gets compressed > and written, and we're back to have some small FS metadata compressed > in our 1 page of zsmalloc allocations. > > I think what is probably "special" about this reproducer system is > that it is slow enough to allow the zero allocation to persist between > the end of the test writes and examining the stats. > > I'd be happy for any thoughts on the likelyhood of this! Thanks for looking into this. Yes, the fact that /dev/urandom shows non-zero values in mm_stat means that we don't have any fishy going on in zram but instead very likely have ZRAM_SAME pages, which don't reach zsmalloc pool and don't use any physical pages. And this is what 145 is in mm_stat that was posted earlier. We have 145 pages that are filled with the same bytes pattern: > > >> however, /sys/block/zram1/mm_stat shows > > >> 9502720 0 0 26214400 196608 145 0 0 > If we think this is right; then the point of the end of this test [1] > is ensure a high reported compression ratio on the device, presumably > to ensure the compression is working. Filling it with urandom would > be unreliable in this regard. I think what we want to do is something > highly compressable like alternate lengths of 0x00 and 0xFF. So var-lengths 0x00/0xff should work, just make sure that you don't have a pattern of sizeof(unsigned long) length. I think fio had an option to generate bin data with a certain level of compress-ability. If that option works then maybe you can just use fio with some static buffer_compress_percentage configuration.
WARNING: multiple messages have this Message-ID (diff)
From: Sergey Senozhatsky <senozhatsky@chromium.org> To: Ian Wienand <iwienand@redhat.com> Cc: Jens Axboe <axboe@kernel.dk>, OGAWA Hirofumi <hirofumi@mail.parknet.co.jp>, linux-kernel@vger.kernel.org, linux-block@vger.kernel.org, Minchan Kim <minchan@kernel.org>, Nitin Gupta <ngupta@vflare.org>, linux-kselftest@vger.kernel.org, Sergey Senozhatsky <senozhatsky@chromium.org>, ltp@lists.linux.it Subject: Re: [LTP] [PATCH 0/1] Possible bug in zram on ppc64le on vfat Date: Mon, 7 Aug 2023 14:19:17 +0900 [thread overview] Message-ID: <20230807051917.GC907732@google.com> (raw) In-Reply-To: <ZNB2kORYiKdl3vSq@fedora19.localdomain> On (23/08/07 14:44), Ian Wienand wrote: [..] > > At this point, because this test fills from /dev/zero, the zsmalloc > pool doesn't actually have anything in it. The filesystem metadata is > in-use from the writes, and is not written out as compressed data. > The zram page de-duplication has kicked in, and instead of handles to > zsmalloc areas for data we just have "this is a page of zeros" > recorded. So this is correctly reflecting that fact that we don't > actually have anything compressed stored at this time. > > > >> If we do a "sync" then redisply the mm_stat after, we get > > >> 26214400 2842 65536 26214400 196608 399 0 0 > > Now we've finished writing all our zeros and have synced, we would > have finished updating vfat allocations, etc. So this gets compressed > and written, and we're back to have some small FS metadata compressed > in our 1 page of zsmalloc allocations. > > I think what is probably "special" about this reproducer system is > that it is slow enough to allow the zero allocation to persist between > the end of the test writes and examining the stats. > > I'd be happy for any thoughts on the likelyhood of this! Thanks for looking into this. Yes, the fact that /dev/urandom shows non-zero values in mm_stat means that we don't have any fishy going on in zram but instead very likely have ZRAM_SAME pages, which don't reach zsmalloc pool and don't use any physical pages. And this is what 145 is in mm_stat that was posted earlier. We have 145 pages that are filled with the same bytes pattern: > > >> however, /sys/block/zram1/mm_stat shows > > >> 9502720 0 0 26214400 196608 145 0 0 > If we think this is right; then the point of the end of this test [1] > is ensure a high reported compression ratio on the device, presumably > to ensure the compression is working. Filling it with urandom would > be unreliable in this regard. I think what we want to do is something > highly compressable like alternate lengths of 0x00 and 0xFF. So var-lengths 0x00/0xff should work, just make sure that you don't have a pattern of sizeof(unsigned long) length. I think fio had an option to generate bin data with a certain level of compress-ability. If that option works then maybe you can just use fio with some static buffer_compress_percentage configuration. -- Mailing list info: https://lists.linux.it/listinfo/ltp
next prev parent reply other threads:[~2023-08-07 5:19 UTC|newest] Thread overview: 44+ messages / expand[flat|nested] mbox.gz Atom feed top 2022-11-07 19:11 [PATCH 0/1] Possible bug in zram on ppc64le on vfat Petr Vorel 2022-11-07 19:11 ` [LTP] " Petr Vorel 2022-11-07 19:11 ` [PATCH 1/1] zram01.sh: Workaround division by 0 on vfat on ppc64le Petr Vorel 2022-11-07 19:11 ` [LTP] " Petr Vorel 2022-11-21 7:14 ` Li Wang 2022-11-21 8:59 ` Petr Vorel 2022-11-21 8:59 ` Petr Vorel 2023-09-18 9:38 ` Richard Palethorpe 2022-11-07 21:25 ` [PATCH 0/1] Possible bug in zram on ppc64le on vfat Minchan Kim 2022-11-07 21:25 ` [LTP] " Minchan Kim 2022-11-07 21:47 ` Petr Vorel 2022-11-07 21:47 ` [LTP] " Petr Vorel 2022-11-07 22:42 ` Petr Vorel 2022-11-07 22:42 ` [LTP] " Petr Vorel 2022-11-08 1:05 ` Sergey Senozhatsky 2022-11-08 1:05 ` [LTP] " Sergey Senozhatsky 2022-11-09 22:08 ` Petr Vorel 2022-11-09 22:08 ` [LTP] " Petr Vorel 2022-11-10 23:04 ` Minchan Kim 2022-11-10 23:04 ` [LTP] " Minchan Kim 2022-11-11 9:29 ` Petr Vorel 2022-11-11 9:29 ` [LTP] " Petr Vorel 2022-11-10 14:29 ` Martin Doucha 2022-11-10 14:29 ` [LTP] " Martin Doucha 2022-11-11 0:48 ` Sergey Senozhatsky 2022-11-11 0:48 ` [LTP] " Sergey Senozhatsky 2022-11-21 9:41 ` Petr Vorel 2022-11-21 9:41 ` [LTP] " Petr Vorel 2022-11-22 14:56 ` Martin Doucha 2022-11-22 14:56 ` Martin Doucha 2022-11-22 15:07 ` Petr Vorel 2022-11-22 15:07 ` [LTP] " Petr Vorel 2022-11-29 4:38 ` Sergey Senozhatsky 2022-11-29 4:38 ` [LTP] " Sergey Senozhatsky 2023-05-02 15:23 ` Martin Doucha 2023-05-02 15:23 ` [LTP] " Martin Doucha 2022-11-11 9:18 ` Petr Vorel 2022-11-11 9:18 ` [LTP] " Petr Vorel 2023-08-04 6:37 ` Ian Wienand 2023-08-04 6:37 ` Ian Wienand 2023-08-07 4:44 ` Ian Wienand 2023-08-07 4:44 ` [LTP] " Ian Wienand 2023-08-07 5:19 ` Sergey Senozhatsky [this message] 2023-08-07 5:19 ` Sergey Senozhatsky
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20230807051917.GC907732@google.com \ --to=senozhatsky@chromium.org \ --cc=axboe@kernel.dk \ --cc=hirofumi@mail.parknet.co.jp \ --cc=iwienand@redhat.com \ --cc=linux-block@vger.kernel.org \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-kselftest@vger.kernel.org \ --cc=ltp@lists.linux.it \ --cc=mdoucha@suse.cz \ --cc=minchan@kernel.org \ --cc=ngupta@vflare.org \ --cc=pvorel@suse.cz \ --cc=xuyang2018.jy@fujitsu.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.