From: Vitaly Wool <vitaly.wool@konsulko.com>
To: "Song Bao Hua (Barry Song)" <song.bao.hua@hisilicon.com>
Cc: Shakeel Butt <shakeelb@google.com>,
Minchan Kim <minchan@kernel.org>, Mike Galbraith <efault@gmx.de>,
LKML <linux-kernel@vger.kernel.org>,
linux-mm <linux-mm@kvack.org>,
Sebastian Andrzej Siewior <bigeasy@linutronix.de>,
NitinGupta <ngupta@vflare.org>,
Sergey Senozhatsky <sergey.senozhatsky.work@gmail.com>,
Andrew Morton <akpm@linux-foundation.org>
Subject: Re: [PATCH] zsmalloc: do not use bit_spin_lock
Date: Mon, 21 Dec 2020 23:11:39 +0100 [thread overview]
Message-ID: <CAM4kBB+xUa8zXSRSuB0z5FCdPNmUpDfcC4Vqu7wzAkf0b+RXqw@mail.gmail.com> (raw)
In-Reply-To: <8cc0e01fd03245a4994f2e0f54b264fa@hisilicon.com>
On Mon, Dec 21, 2020 at 10:30 PM Song Bao Hua (Barry Song)
<song.bao.hua@hisilicon.com> wrote:
>
>
>
> > -----Original Message-----
> > From: Shakeel Butt [mailto:shakeelb@google.com]
> > Sent: Tuesday, December 22, 2020 10:03 AM
> > To: Song Bao Hua (Barry Song) <song.bao.hua@hisilicon.com>
> > Cc: Vitaly Wool <vitaly.wool@konsulko.com>; Minchan Kim <minchan@kernel.org>;
> > Mike Galbraith <efault@gmx.de>; LKML <linux-kernel@vger.kernel.org>; linux-mm
> > <linux-mm@kvack.org>; Sebastian Andrzej Siewior <bigeasy@linutronix.de>;
> > NitinGupta <ngupta@vflare.org>; Sergey Senozhatsky
> > <sergey.senozhatsky.work@gmail.com>; Andrew Morton
> > <akpm@linux-foundation.org>
> > Subject: Re: [PATCH] zsmalloc: do not use bit_spin_lock
> >
> > On Mon, Dec 21, 2020 at 12:06 PM Song Bao Hua (Barry Song)
> > <song.bao.hua@hisilicon.com> wrote:
> > >
> > >
> > >
> > > > -----Original Message-----
> > > > From: Shakeel Butt [mailto:shakeelb@google.com]
> > > > Sent: Tuesday, December 22, 2020 8:50 AM
> > > > To: Vitaly Wool <vitaly.wool@konsulko.com>
> > > > Cc: Minchan Kim <minchan@kernel.org>; Mike Galbraith <efault@gmx.de>; LKML
> > > > <linux-kernel@vger.kernel.org>; linux-mm <linux-mm@kvack.org>; Song Bao
> > Hua
> > > > (Barry Song) <song.bao.hua@hisilicon.com>; Sebastian Andrzej Siewior
> > > > <bigeasy@linutronix.de>; NitinGupta <ngupta@vflare.org>; Sergey
> > Senozhatsky
> > > > <sergey.senozhatsky.work@gmail.com>; Andrew Morton
> > > > <akpm@linux-foundation.org>
> > > > Subject: Re: [PATCH] zsmalloc: do not use bit_spin_lock
> > > >
> > > > On Mon, Dec 21, 2020 at 11:20 AM Vitaly Wool <vitaly.wool@konsulko.com>
> > wrote:
> > > > >
> > > > > On Mon, Dec 21, 2020 at 6:24 PM Minchan Kim <minchan@kernel.org> wrote:
> > > > > >
> > > > > > On Sun, Dec 20, 2020 at 02:22:28AM +0200, Vitaly Wool wrote:
> > > > > > > zsmalloc takes bit spinlock in its _map() callback and releases it
> > > > > > > only in unmap() which is unsafe and leads to zswap complaining
> > > > > > > about scheduling in atomic context.
> > > > > > >
> > > > > > > To fix that and to improve RT properties of zsmalloc, remove that
> > > > > > > bit spinlock completely and use a bit flag instead.
> > > > > >
> > > > > > I don't want to use such open code for the lock.
> > > > > >
> > > > > > I see from Mike's patch, recent zswap change introduced the lockdep
> > > > > > splat bug and you want to improve zsmalloc to fix the zswap bug and
> > > > > > introduce this patch with allowing preemption enabling.
> > > > >
> > > > > This understanding is upside down. The code in zswap you are referring
> > > > > to is not buggy. You may claim that it is suboptimal but there is
> > > > > nothing wrong in taking a mutex.
> > > > >
> > > >
> > > > Is this suboptimal for all or just the hardware accelerators? Sorry, I
> > > > am not very familiar with the crypto API. If I select lzo or lz4 as a
> > > > zswap compressor will the [de]compression be async or sync?
> > >
> > > Right now, in crypto subsystem, new drivers are required to write based on
> > > async APIs. The old sync API can't work in new accelerator drivers as they
> > > are not supported at all.
> > >
> > > Old drivers are used to sync, but they've got async wrappers to support async
> > > APIs. Eg.
> > > crypto: acomp - add support for lz4 via scomp
> > >
> > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/
> > crypto/lz4.c?id=8cd9330e0a615c931037d4def98b5ce0d540f08d
> > >
> > > crypto: acomp - add support for lzo via scomp
> > >
> > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/
> > crypto/lzo.c?id=ac9d2c4b39e022d2c61486bfc33b730cfd02898e
> > >
> > > so they are supporting async APIs but they are still working in sync mode
> > as
> > > those old drivers don't sleep.
> > >
> >
> > Good to know that those are sync because I want them to be sync.
> > Please note that zswap is a cache in front of a real swap and the load
> > operation is latency sensitive as it comes in the page fault path and
> > directly impacts the applications. I doubt decompressing synchronously
> > a 4k page on a cpu will be costlier than asynchronously decompressing
> > the same page from hardware accelerators.
>
> If you read the old paper:
> https://www.ibm.com/support/pages/new-linux-zswap-compression-functionality
> Because the hardware accelerator speeds up compression, looking at the zswap
> metrics we observed that there were more store and load requests in a given
> amount of time, which filled up the zswap pool faster than a software
> compression run. Because of this behavior, we set the max_pool_percent
> parameter to 30 for the hardware compression runs - this means that zswap
> can use up to 30% of the 10GB of total memory.
>
> So using hardware accelerators, we get a chance to speed up compression
> while decreasing cpu utilization.
>
> BTW, If it is not easy to change zsmalloc, one quick workaround we might do
> in zswap is adding the below after applying Mike's original patch:
>
> if(in_atomic()) /* for zsmalloc */
> while(!try_wait_for_completion(&req->done);
> else /* for zbud, z3fold */
> crypto_wait_req(....);
I don't think I'm going to ack this, sorry.
Best regards,
Vitaly
> crypto_wait_req() is actually doing wait_for_completion():
> static inline int crypto_wait_req(int err, struct crypto_wait *wait)
> {
> switch (err) {
> case -EINPROGRESS:
> case -EBUSY:
> wait_for_completion(&wait->completion);
> reinit_completion(&wait->completion);
> err = wait->err;
> break;
> }
>
> return err;
> }
>
> Thanks
> Barry
next prev parent reply other threads:[~2020-12-21 22:11 UTC|newest]
Thread overview: 47+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-12-19 10:04 [patch] zswap: fix zswap_frontswap_load() vs zsmalloc::map/unmap() might_sleep() splat Mike Galbraith
2020-12-19 10:12 ` Mike Galbraith
2020-12-19 10:20 ` Vitaly Wool
2020-12-19 10:27 ` Mike Galbraith
2020-12-19 10:46 ` Vitaly Wool
2020-12-19 10:59 ` Mike Galbraith
2020-12-19 11:03 ` Mike Galbraith
2020-12-20 0:22 ` [PATCH] zsmalloc: do not use bit_spin_lock Vitaly Wool
2020-12-20 1:18 ` Matthew Wilcox
2020-12-20 7:21 ` Vitaly Wool
2021-01-14 16:17 ` Sebastian Andrzej Siewior
2020-12-20 1:23 ` Mike Galbraith
2020-12-20 4:11 ` Mike Galbraith
2020-12-20 7:47 ` Mike Galbraith
2020-12-20 21:20 ` Song Bao Hua (Barry Song)
2020-12-20 22:10 ` Mike Galbraith
2020-12-20 1:56 ` Mike Galbraith
2020-12-21 17:24 ` Minchan Kim
2020-12-21 19:20 ` Vitaly Wool
2020-12-21 19:50 ` Shakeel Butt
2020-12-21 20:05 ` Song Bao Hua (Barry Song)
2020-12-21 21:02 ` Shakeel Butt
2020-12-21 21:25 ` Song Bao Hua (Barry Song)
2020-12-21 22:11 ` Vitaly Wool [this message]
2020-12-21 22:42 ` Song Bao Hua (Barry Song)
2020-12-21 23:35 ` Song Bao Hua (Barry Song)
2020-12-22 0:59 ` Vitaly Wool
2020-12-22 1:10 ` Song Bao Hua (Barry Song)
2020-12-22 1:42 ` Song Bao Hua (Barry Song)
2020-12-22 1:57 ` Vitaly Wool
2020-12-22 2:07 ` Song Bao Hua (Barry Song)
2020-12-22 2:10 ` Song Bao Hua (Barry Song)
2020-12-22 9:44 ` Vitaly Wool
2020-12-22 21:06 ` Song Bao Hua (Barry Song)
2020-12-23 0:11 ` Vitaly Wool
2020-12-23 12:44 ` tiantao (H)
2020-12-23 18:25 ` Vitaly Wool
2021-01-14 16:18 ` Sebastian Andrzej Siewior
2021-01-14 16:29 ` Vitaly Wool
2021-01-14 16:56 ` Sebastian Andrzej Siewior
2021-01-14 17:15 ` Vitaly Wool
2021-01-14 17:18 ` Sebastian Andrzej Siewior
2020-12-21 22:46 ` Shakeel Butt
2020-12-21 23:02 ` Song Bao Hua (Barry Song)
2020-12-22 9:20 ` David Laight
2020-12-22 9:32 ` Vitaly Wool
2020-12-21 20:22 ` Minchan Kim
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=CAM4kBB+xUa8zXSRSuB0z5FCdPNmUpDfcC4Vqu7wzAkf0b+RXqw@mail.gmail.com \
--to=vitaly.wool@konsulko.com \
--cc=akpm@linux-foundation.org \
--cc=bigeasy@linutronix.de \
--cc=efault@gmx.de \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=minchan@kernel.org \
--cc=ngupta@vflare.org \
--cc=sergey.senozhatsky.work@gmail.com \
--cc=shakeelb@google.com \
--cc=song.bao.hua@hisilicon.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).