All of lore.kernel.org
 help / color / mirror / Atom feed
From: Sergey Senozhatsky <senozhatsky@chromium.org>
To: Ian Wienand <iwienand@redhat.com>
Cc: Minchan Kim <minchan@kernel.org>, Petr Vorel <pvorel@suse.cz>,
	ltp@lists.linux.it, linux-block@vger.kernel.org,
	linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org,
	Nitin Gupta <ngupta@vflare.org>,
	Sergey Senozhatsky <senozhatsky@chromium.org>,
	Jens Axboe <axboe@kernel.dk>,
	OGAWA Hirofumi <hirofumi@mail.parknet.co.jp>,
	Martin Doucha <mdoucha@suse.cz>,
	Yang Xu <xuyang2018.jy@fujitsu.com>
Subject: Re: [PATCH 0/1] Possible bug in zram on ppc64le on vfat
Date: Mon, 7 Aug 2023 14:19:17 +0900	[thread overview]
Message-ID: <20230807051917.GC907732@google.com> (raw)
In-Reply-To: <ZNB2kORYiKdl3vSq@fedora19.localdomain>

On (23/08/07 14:44), Ian Wienand wrote:
[..]
> 
> At this point, because this test fills from /dev/zero, the zsmalloc
> pool doesn't actually have anything in it.  The filesystem metadata is
> in-use from the writes, and is not written out as compressed data.
> The zram page de-duplication has kicked in, and instead of handles to
> zsmalloc areas for data we just have "this is a page of zeros"
> recorded.  So this is correctly reflecting that fact that we don't
> actually have anything compressed stored at this time.
> 
> >  >> If we do a "sync" then redisply the mm_stat after, we get
> >  >>   26214400     2842    65536 26214400   196608      399        0        0
> 
> Now we've finished writing all our zeros and have synced, we would
> have finished updating vfat allocations, etc.  So this gets compressed
> and written, and we're back to have some small FS metadata compressed
> in our 1 page of zsmalloc allocations.
> 
> I think what is probably "special" about this reproducer system is
> that it is slow enough to allow the zero allocation to persist between
> the end of the test writes and examining the stats.
> 
> I'd be happy for any thoughts on the likelyhood of this!

Thanks for looking into this.

Yes, the fact that /dev/urandom shows non-zero values in mm_stat means
that we don't have any fishy going on in zram but instead very likely
have ZRAM_SAME pages, which don't reach zsmalloc pool and don't use any
physical pages.

And this is what 145 is in mm_stat that was posted earlier. We have 145
pages that are filled with the same bytes pattern:

> >  >> however, /sys/block/zram1/mm_stat shows
> >  >>   9502720        0        0 26214400   196608      145        0        0

> If we think this is right; then the point of the end of this test [1]
> is ensure a high reported compression ratio on the device, presumably
> to ensure the compression is working.  Filling it with urandom would
> be unreliable in this regard.  I think what we want to do is something
> highly compressable like alternate lengths of 0x00 and 0xFF.

So var-lengths 0x00/0xff should work, just make sure that you don't have
a pattern of sizeof(unsigned long) length.

I think fio had an option to generate bin data with a certain level
of compress-ability. If that option works then maybe you can just use
fio with some static buffer_compress_percentage configuration.

WARNING: multiple messages have this Message-ID (diff)
From: Sergey Senozhatsky <senozhatsky@chromium.org>
To: Ian Wienand <iwienand@redhat.com>
Cc: Jens Axboe <axboe@kernel.dk>,
	OGAWA Hirofumi <hirofumi@mail.parknet.co.jp>,
	linux-kernel@vger.kernel.org, linux-block@vger.kernel.org,
	Minchan Kim <minchan@kernel.org>, Nitin Gupta <ngupta@vflare.org>,
	linux-kselftest@vger.kernel.org,
	Sergey Senozhatsky <senozhatsky@chromium.org>,
	ltp@lists.linux.it
Subject: Re: [LTP] [PATCH 0/1] Possible bug in zram on ppc64le on vfat
Date: Mon, 7 Aug 2023 14:19:17 +0900	[thread overview]
Message-ID: <20230807051917.GC907732@google.com> (raw)
In-Reply-To: <ZNB2kORYiKdl3vSq@fedora19.localdomain>

On (23/08/07 14:44), Ian Wienand wrote:
[..]
> 
> At this point, because this test fills from /dev/zero, the zsmalloc
> pool doesn't actually have anything in it.  The filesystem metadata is
> in-use from the writes, and is not written out as compressed data.
> The zram page de-duplication has kicked in, and instead of handles to
> zsmalloc areas for data we just have "this is a page of zeros"
> recorded.  So this is correctly reflecting that fact that we don't
> actually have anything compressed stored at this time.
> 
> >  >> If we do a "sync" then redisply the mm_stat after, we get
> >  >>   26214400     2842    65536 26214400   196608      399        0        0
> 
> Now we've finished writing all our zeros and have synced, we would
> have finished updating vfat allocations, etc.  So this gets compressed
> and written, and we're back to have some small FS metadata compressed
> in our 1 page of zsmalloc allocations.
> 
> I think what is probably "special" about this reproducer system is
> that it is slow enough to allow the zero allocation to persist between
> the end of the test writes and examining the stats.
> 
> I'd be happy for any thoughts on the likelyhood of this!

Thanks for looking into this.

Yes, the fact that /dev/urandom shows non-zero values in mm_stat means
that we don't have any fishy going on in zram but instead very likely
have ZRAM_SAME pages, which don't reach zsmalloc pool and don't use any
physical pages.

And this is what 145 is in mm_stat that was posted earlier. We have 145
pages that are filled with the same bytes pattern:

> >  >> however, /sys/block/zram1/mm_stat shows
> >  >>   9502720        0        0 26214400   196608      145        0        0

> If we think this is right; then the point of the end of this test [1]
> is ensure a high reported compression ratio on the device, presumably
> to ensure the compression is working.  Filling it with urandom would
> be unreliable in this regard.  I think what we want to do is something
> highly compressable like alternate lengths of 0x00 and 0xFF.

So var-lengths 0x00/0xff should work, just make sure that you don't have
a pattern of sizeof(unsigned long) length.

I think fio had an option to generate bin data with a certain level
of compress-ability. If that option works then maybe you can just use
fio with some static buffer_compress_percentage configuration.

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

  reply	other threads:[~2023-08-07  5:19 UTC|newest]

Thread overview: 44+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-11-07 19:11 [PATCH 0/1] Possible bug in zram on ppc64le on vfat Petr Vorel
2022-11-07 19:11 ` [LTP] " Petr Vorel
2022-11-07 19:11 ` [PATCH 1/1] zram01.sh: Workaround division by 0 on vfat on ppc64le Petr Vorel
2022-11-07 19:11   ` [LTP] " Petr Vorel
2022-11-21  7:14   ` Li Wang
2022-11-21  8:59     ` Petr Vorel
2022-11-21  8:59       ` Petr Vorel
2023-09-18  9:38       ` Richard Palethorpe
2022-11-07 21:25 ` [PATCH 0/1] Possible bug in zram on ppc64le on vfat Minchan Kim
2022-11-07 21:25   ` [LTP] " Minchan Kim
2022-11-07 21:47   ` Petr Vorel
2022-11-07 21:47     ` [LTP] " Petr Vorel
2022-11-07 22:42     ` Petr Vorel
2022-11-07 22:42       ` [LTP] " Petr Vorel
2022-11-08  1:05       ` Sergey Senozhatsky
2022-11-08  1:05         ` [LTP] " Sergey Senozhatsky
2022-11-09 22:08         ` Petr Vorel
2022-11-09 22:08           ` [LTP] " Petr Vorel
2022-11-10 23:04     ` Minchan Kim
2022-11-10 23:04       ` [LTP] " Minchan Kim
2022-11-11  9:29       ` Petr Vorel
2022-11-11  9:29         ` [LTP] " Petr Vorel
2022-11-10 14:29   ` Martin Doucha
2022-11-10 14:29     ` [LTP] " Martin Doucha
2022-11-11  0:48     ` Sergey Senozhatsky
2022-11-11  0:48       ` [LTP] " Sergey Senozhatsky
2022-11-21  9:41       ` Petr Vorel
2022-11-21  9:41         ` [LTP] " Petr Vorel
2022-11-22 14:56       ` Martin Doucha
2022-11-22 14:56         ` Martin Doucha
2022-11-22 15:07         ` Petr Vorel
2022-11-22 15:07           ` [LTP] " Petr Vorel
2022-11-29  4:38           ` Sergey Senozhatsky
2022-11-29  4:38             ` [LTP] " Sergey Senozhatsky
2023-05-02 15:23             ` Martin Doucha
2023-05-02 15:23               ` [LTP] " Martin Doucha
2022-11-11  9:18     ` Petr Vorel
2022-11-11  9:18       ` [LTP] " Petr Vorel
2023-08-04  6:37   ` Ian Wienand
2023-08-04  6:37     ` Ian Wienand
2023-08-07  4:44     ` Ian Wienand
2023-08-07  4:44       ` [LTP] " Ian Wienand
2023-08-07  5:19       ` Sergey Senozhatsky [this message]
2023-08-07  5:19         ` Sergey Senozhatsky

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20230807051917.GC907732@google.com \
    --to=senozhatsky@chromium.org \
    --cc=axboe@kernel.dk \
    --cc=hirofumi@mail.parknet.co.jp \
    --cc=iwienand@redhat.com \
    --cc=linux-block@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-kselftest@vger.kernel.org \
    --cc=ltp@lists.linux.it \
    --cc=mdoucha@suse.cz \
    --cc=minchan@kernel.org \
    --cc=ngupta@vflare.org \
    --cc=pvorel@suse.cz \
    --cc=xuyang2018.jy@fujitsu.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.