From: Florian Fainelli <f.fainelli@gmail.com> To: Robin Murphy <robin.murphy@arm.com>, linux-kernel@vger.kernel.org, Christoph Hellwig <hch@lst.de> Cc: Jonathan Corbet <corbet@lwn.net>, opendmb@gmail.com, "Paul E. McKenney" <paulmck@kernel.org>, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>, Mauro Carvalho Chehab <mchehab+huawei@kernel.org>, Viresh Kumar <viresh.kumar@linaro.org>, Randy Dunlap <rdunlap@infradead.org>, "open list:DOCUMENTATION" <linux-doc@vger.kernel.org>, Peter Zijlstra <peterz@infradead.org>, "open list:SWIOTLB SUBSYSTEM" <iommu@lists.linux-foundation.org>, Andrew Morton <akpm@linux-foundation.org>, Mike Kravetz <mike.kravetz@oracle.com>, Thomas Gleixner <tglx@linutronix.de> Subject: Re: [PATCH] swiotlb: Add swiotlb=off to disable SWIOTLB Date: Thu, 18 Mar 2021 17:48:12 -0700 [thread overview] Message-ID: <a7a858e9-d6e6-cd06-85fc-8826e45d460e@gmail.com> (raw) In-Reply-To: <3dd81519-4a73-efb8-abf0-0b766f993a8b@arm.com> On 3/18/2021 4:35 PM, Robin Murphy wrote: > On 2021-03-18 21:31, Florian Fainelli wrote: >> >> >> On 3/18/2021 12:53 PM, Robin Murphy wrote: >>> On 2021-03-18 19:43, Florian Fainelli wrote: >>>> >>>> >>>> On 3/18/2021 12:34 PM, Robin Murphy wrote: >>>>> On 2021-03-18 19:22, Florian Fainelli wrote: >>>>>> >>>>>> >>>>>> On 3/18/2021 12:18 PM, Florian Fainelli wrote: >>>>>>> It may be useful to disable the SWIOTLB completely for testing or >>>>>>> when a >>>>>>> platform is known not to have any DRAM addressing limitations >>>>>>> what so >>>>>>> ever. >>>>> >>>>> Isn't that what "swiotlb=noforce" is for? If you're confident that >>>>> we've >>>>> really ironed out *all* the awkward corners that used to blow up if >>>>> various internal bits were left uninitialised, then it would make >>>>> sense >>>>> to just tweak the implementation of what we already have. >>>> >>>> swiotlb=noforce does prevent dma_direct_map_page() from resorting to >>>> the >>>> swiotlb, however what I am also after is reclaiming these 64MB of >>>> default SWIOTLB bounce buffering memory because my systems run with >>>> large amounts of reserved memory into ZONE_MOVABLE and everything in >>>> ZONE_NORMAL is precious at that point. >>> >>> It also forces io_tlb_nslabs to the minimum, so it should be claiming >>> considerably less than 64MB. IIRC the original proposal *did* skip >>> initialisation completely, but that turned up the aforementioned issues. >> >> AFAICT in that case we will have iotlb_n_slabs will set to 1, which will >> still make us allocate io_tlb_n_slabs << IO_TLB_SHIFT bytes in >> swiotlb_init(), which still gives us 64MB. > > Eh? When did 2KB become 64MB? IO_TLB_SHIFT is 11, so that's at most one > page in anyone's money... Yes, sorry incorrect shift applied here. Still, and I believe this is what you mean below, architecture code setting swiotlb_force = SWIOTLB_NO_FORCE does not result in not allocating the SWIOTLB, because io_tlb_nslabs is still left set to 0 so swiotlb_init() will proceed with allocating the default size. > >>>>> I wouldn't necessarily disagree with adding "off" as an additional >>>>> alias >>>>> for "noforce", though, since it does come across as a bit wacky for >>>>> general use. >>>>> >>>>>>> Signed-off-by: Florian Fainelli <f.fainelli@gmail.com> >>>>>> >>>>>> Christoph, in addition to this change, how would you feel if we >>>>>> qualified the swiotlb_init() in arch/arm/mm/init.c with a: >>>>>> >>>>>> >>>>>> if (memblock_end_of_DRAM() >= SZ_4G) >>>>>> swiotlb_init(1) >>>>> >>>>> Modulo "swiotlb=force", of course ;) >>>> >>>> Indeed, we would need to handle that case as well. Does it sound >>>> reasonable to do that to you as well? >>> >>> I wouldn't like it done to me personally, but for arm64, observe what >>> mem_init() in arch/arm64/mm/init.c already does. > > In fact I should have looked more closely at that myself - checking > debugfs on my 4GB arm64 board actually shows io_tlb_nslabs = 0, and > indeed we are bypassing initialisation completely and (ab)using > SWIOTLB_NO_FORCE to cover it up, so I guess it probably *is* safe now > for the noforce option to do the same for itself and save even that one > page. OK, I can submit a patch that does that. 5.12-rc3 works correctly for me here as well and only allocates SWIOTLB when needed which in our case is either: - we have DRAM at PA >= 4GB - we have limited peripherals (Raspberry Pi 4 derivative) that can only address the lower 1GB Now let's see if we can get ARM 32-bit to match :) -- Florian
WARNING: multiple messages have this Message-ID (diff)
From: Florian Fainelli <f.fainelli@gmail.com> To: Robin Murphy <robin.murphy@arm.com>, linux-kernel@vger.kernel.org, Christoph Hellwig <hch@lst.de> Cc: opendmb@gmail.com, "Paul E. McKenney" <paulmck@kernel.org>, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>, Mauro Carvalho Chehab <mchehab+huawei@kernel.org>, Viresh Kumar <viresh.kumar@linaro.org>, Jonathan Corbet <corbet@lwn.net>, "open list:DOCUMENTATION" <linux-doc@vger.kernel.org>, Peter Zijlstra <peterz@infradead.org>, "open list:SWIOTLB SUBSYSTEM" <iommu@lists.linux-foundation.org>, Randy Dunlap <rdunlap@infradead.org>, Andrew Morton <akpm@linux-foundation.org>, Thomas Gleixner <tglx@linutronix.de>, Mike Kravetz <mike.kravetz@oracle.com> Subject: Re: [PATCH] swiotlb: Add swiotlb=off to disable SWIOTLB Date: Thu, 18 Mar 2021 17:48:12 -0700 [thread overview] Message-ID: <a7a858e9-d6e6-cd06-85fc-8826e45d460e@gmail.com> (raw) In-Reply-To: <3dd81519-4a73-efb8-abf0-0b766f993a8b@arm.com> On 3/18/2021 4:35 PM, Robin Murphy wrote: > On 2021-03-18 21:31, Florian Fainelli wrote: >> >> >> On 3/18/2021 12:53 PM, Robin Murphy wrote: >>> On 2021-03-18 19:43, Florian Fainelli wrote: >>>> >>>> >>>> On 3/18/2021 12:34 PM, Robin Murphy wrote: >>>>> On 2021-03-18 19:22, Florian Fainelli wrote: >>>>>> >>>>>> >>>>>> On 3/18/2021 12:18 PM, Florian Fainelli wrote: >>>>>>> It may be useful to disable the SWIOTLB completely for testing or >>>>>>> when a >>>>>>> platform is known not to have any DRAM addressing limitations >>>>>>> what so >>>>>>> ever. >>>>> >>>>> Isn't that what "swiotlb=noforce" is for? If you're confident that >>>>> we've >>>>> really ironed out *all* the awkward corners that used to blow up if >>>>> various internal bits were left uninitialised, then it would make >>>>> sense >>>>> to just tweak the implementation of what we already have. >>>> >>>> swiotlb=noforce does prevent dma_direct_map_page() from resorting to >>>> the >>>> swiotlb, however what I am also after is reclaiming these 64MB of >>>> default SWIOTLB bounce buffering memory because my systems run with >>>> large amounts of reserved memory into ZONE_MOVABLE and everything in >>>> ZONE_NORMAL is precious at that point. >>> >>> It also forces io_tlb_nslabs to the minimum, so it should be claiming >>> considerably less than 64MB. IIRC the original proposal *did* skip >>> initialisation completely, but that turned up the aforementioned issues. >> >> AFAICT in that case we will have iotlb_n_slabs will set to 1, which will >> still make us allocate io_tlb_n_slabs << IO_TLB_SHIFT bytes in >> swiotlb_init(), which still gives us 64MB. > > Eh? When did 2KB become 64MB? IO_TLB_SHIFT is 11, so that's at most one > page in anyone's money... Yes, sorry incorrect shift applied here. Still, and I believe this is what you mean below, architecture code setting swiotlb_force = SWIOTLB_NO_FORCE does not result in not allocating the SWIOTLB, because io_tlb_nslabs is still left set to 0 so swiotlb_init() will proceed with allocating the default size. > >>>>> I wouldn't necessarily disagree with adding "off" as an additional >>>>> alias >>>>> for "noforce", though, since it does come across as a bit wacky for >>>>> general use. >>>>> >>>>>>> Signed-off-by: Florian Fainelli <f.fainelli@gmail.com> >>>>>> >>>>>> Christoph, in addition to this change, how would you feel if we >>>>>> qualified the swiotlb_init() in arch/arm/mm/init.c with a: >>>>>> >>>>>> >>>>>> if (memblock_end_of_DRAM() >= SZ_4G) >>>>>> swiotlb_init(1) >>>>> >>>>> Modulo "swiotlb=force", of course ;) >>>> >>>> Indeed, we would need to handle that case as well. Does it sound >>>> reasonable to do that to you as well? >>> >>> I wouldn't like it done to me personally, but for arm64, observe what >>> mem_init() in arch/arm64/mm/init.c already does. > > In fact I should have looked more closely at that myself - checking > debugfs on my 4GB arm64 board actually shows io_tlb_nslabs = 0, and > indeed we are bypassing initialisation completely and (ab)using > SWIOTLB_NO_FORCE to cover it up, so I guess it probably *is* safe now > for the noforce option to do the same for itself and save even that one > page. OK, I can submit a patch that does that. 5.12-rc3 works correctly for me here as well and only allocates SWIOTLB when needed which in our case is either: - we have DRAM at PA >= 4GB - we have limited peripherals (Raspberry Pi 4 derivative) that can only address the lower 1GB Now let's see if we can get ARM 32-bit to match :) -- Florian _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
next prev parent reply other threads:[~2021-03-19 0:49 UTC|newest] Thread overview: 36+ messages / expand[flat|nested] mbox.gz Atom feed top 2021-03-18 19:18 [PATCH] swiotlb: Add swiotlb=off to disable SWIOTLB Florian Fainelli 2021-03-18 19:18 ` Florian Fainelli 2021-03-18 19:22 ` Florian Fainelli 2021-03-18 19:22 ` Florian Fainelli 2021-03-18 19:34 ` Robin Murphy 2021-03-18 19:34 ` Robin Murphy 2021-03-18 19:43 ` Florian Fainelli 2021-03-18 19:43 ` Florian Fainelli 2021-03-18 19:53 ` Robin Murphy 2021-03-18 19:53 ` Robin Murphy 2021-03-18 21:31 ` Florian Fainelli 2021-03-18 21:31 ` Florian Fainelli 2021-03-18 23:35 ` Robin Murphy 2021-03-18 23:35 ` Robin Murphy 2021-03-19 0:48 ` Florian Fainelli [this message] 2021-03-19 0:48 ` Florian Fainelli 2021-03-19 2:34 ` Konrad Rzeszutek Wilk 2021-03-19 2:34 ` Konrad Rzeszutek Wilk 2021-03-19 4:00 ` [PATCH] swiotlb: Make SWIOTLB_NO_FORCE perform no allocation Florian Fainelli 2021-03-19 4:00 ` Florian Fainelli 2021-03-19 5:01 ` Konrad Rzeszutek Wilk 2021-03-19 5:01 ` Konrad Rzeszutek Wilk 2021-03-21 3:37 ` [PATCH v2] " Florian Fainelli 2021-03-21 3:37 ` Florian Fainelli 2021-03-22 7:46 ` Christoph Hellwig 2021-03-22 7:46 ` Christoph Hellwig 2021-03-23 1:53 ` [PATCH v3] " Florian Fainelli 2021-03-23 1:53 ` Florian Fainelli 2021-03-24 8:42 ` Christoph Hellwig 2021-03-24 8:42 ` Christoph Hellwig 2021-04-09 3:13 ` Florian Fainelli 2021-04-09 3:13 ` Florian Fainelli 2021-04-09 19:32 ` Konrad Rzeszutek Wilk 2021-04-09 19:32 ` Konrad Rzeszutek Wilk 2021-04-09 20:33 ` Florian Fainelli 2021-04-09 20:33 ` Florian Fainelli
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=a7a858e9-d6e6-cd06-85fc-8826e45d460e@gmail.com \ --to=f.fainelli@gmail.com \ --cc=akpm@linux-foundation.org \ --cc=corbet@lwn.net \ --cc=hch@lst.de \ --cc=iommu@lists.linux-foundation.org \ --cc=konrad.wilk@oracle.com \ --cc=linux-doc@vger.kernel.org \ --cc=linux-kernel@vger.kernel.org \ --cc=mchehab+huawei@kernel.org \ --cc=mike.kravetz@oracle.com \ --cc=opendmb@gmail.com \ --cc=paulmck@kernel.org \ --cc=peterz@infradead.org \ --cc=rdunlap@infradead.org \ --cc=robin.murphy@arm.com \ --cc=tglx@linutronix.de \ --cc=viresh.kumar@linaro.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.