From: David Rientjes <rientjes@google.com>
To: Christoph Hellwig <hch@lst.de>
Cc: "Lendacky, Thomas" <Thomas.Lendacky@amd.com>,
Keith Busch <kbusch@kernel.org>, Jens Axboe <axboe@kernel.dk>,
"Singh, Brijesh" <brijesh.singh@amd.com>,
Ming Lei <ming.lei@redhat.com>, Peter Gonda <pgonda@google.com>,
Jianxiong Gao <jxgao@google.com>,
"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
"x86@kernel.org" <x86@kernel.org>,
"iommu@lists.linux-foundation.org"
<iommu@lists.linux-foundation.org>
Subject: Re: [bug] __blk_mq_run_hw_queue suspicious rcu usage
Date: Wed, 27 Nov 2019 14:11:28 -0800 (PST) [thread overview]
Message-ID: <alpine.DEB.2.21.1911271359000.135363@chino.kir.corp.google.com> (raw)
In-Reply-To: <20190918132242.GA16133@lst.de>
On Wed, 18 Sep 2019, Christoph Hellwig wrote:
> On Tue, Sep 17, 2019 at 06:41:02PM +0000, Lendacky, Thomas wrote:
> > > diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
> > > --- a/drivers/nvme/host/pci.c
> > > +++ b/drivers/nvme/host/pci.c
> > > @@ -1613,7 +1613,8 @@ static int nvme_alloc_admin_tags(struct nvme_dev *dev)
> > > dev->admin_tagset.timeout = ADMIN_TIMEOUT;
> > > dev->admin_tagset.numa_node = dev_to_node(dev->dev);
> > > dev->admin_tagset.cmd_size = sizeof(struct nvme_iod);
> > > - dev->admin_tagset.flags = BLK_MQ_F_NO_SCHED;
> > > + dev->admin_tagset.flags = BLK_MQ_F_NO_SCHED |
> > > + BLK_MQ_F_BLOCKING;
> >
> > I think you want to only set the BLK_MQ_F_BLOCKING if the DMA is required
> > to be unencrypted. Unfortunately, force_dma_unencrypted() can't be called
> > from a module. Is there a DMA API that could be called to get that info?
>
> The DMA API must support non-blocking calls, and various drivers rely
> on that. So we need to provide that even for the SEV case. If the
> actual blocking can't be made to work we'll need to wire up the DMA
> pool in kernel/dma/remap.c for it (and probably move it to separate
> file).
>
Resurrecting this thread from a couple months ago because it appears that
this is still an issue with 5.4 guests.
dma_pool_alloc(), regardless of whether mem_flags allows blocking or not,
can always sleep if the device's DMA must be unencrypted and
mem_encrypt_active() == true. We know this because vm_unmap_aliases() can
always block.
NVMe's setup of PRPs and SGLs uses dma_pool_alloc(GFP_ATOMIC) but when
this is a SEV-enabled guest this allocation may block due to the
possibility of allocating DMA coherent memory through dma_direct_alloc().
It seems like one solution would be to add significant latency by doing
BLK_MQ_F_BLOCKING if force_dma_unencrypted() is true for the device but
this comes with significant downsides.
So we're left with making dma_pool_alloc(GFP_ATOMIC) actually be atomic
even when the DMA needs to be unencrypted for SEV. Christoph's suggestion
was to wire up dmapool in kernel/dma/remap.c for this. Is that necessary
to be done for all devices that need to do dma_pool_alloc(GFP_ATOMIC) or
can we do it within the DMA API itself so it's transparent to the driver?
Thomas/Brijesh: separately, it seems the use of set_memory_encrypted() or
set_memory_decrypted() must be possible without blocking; is this only an
issue from the DMA API point of view or can it be done elsewhere?
next prev parent reply other threads:[~2019-11-27 22:11 UTC|newest]
Thread overview: 13+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-09-04 21:40 [bug] __blk_mq_run_hw_queue suspicious rcu usage David Rientjes
2019-09-05 6:06 ` Christoph Hellwig
2019-09-05 22:37 ` David Rientjes
2019-09-16 23:45 ` David Rientjes
2019-09-17 18:23 ` David Rientjes
2019-09-17 18:32 ` Jens Axboe
2019-09-17 18:41 ` Lendacky, Thomas
2019-09-18 13:22 ` Christoph Hellwig
2019-11-27 22:11 ` David Rientjes [this message]
2019-11-28 6:40 ` Christoph Hellwig
2019-12-13 0:07 ` David Rientjes
2019-12-13 9:33 ` David Rientjes
2019-12-15 5:38 ` David Rientjes
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=alpine.DEB.2.21.1911271359000.135363@chino.kir.corp.google.com \
--to=rientjes@google.com \
--cc=Thomas.Lendacky@amd.com \
--cc=axboe@kernel.dk \
--cc=brijesh.singh@amd.com \
--cc=hch@lst.de \
--cc=iommu@lists.linux-foundation.org \
--cc=jxgao@google.com \
--cc=kbusch@kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=ming.lei@redhat.com \
--cc=pgonda@google.com \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).