From: Steve Wahl <steve.wahl@hpe.com>
To: "Tian, Kevin" <kevin.tian@intel.com>
Cc: David Woodhouse <dwmw2@infradead.org>,
Baolu Lu <baolu.lu@linux.intel.com>,
Steve Wahl <steve.wahl@hpe.com>, "Rodel, Jorg" <jroedel@suse.de>,
Kyung Min Park <kyung.min.park@intel.com>,
Will Deacon <will@kernel.org>,
"iommu@lists.linux-foundation.org"
<iommu@lists.linux-foundation.org>,
Mike Travis <mike.travis@hpe.com>,
Dimitri Sivanich <sivanich@hpe.com>,
"Anderson, Russ" <russ.anderson@hpe.com>,
"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH] iommu/vt-d: Increase DMAR_UNITS_SUPPORTED
Date: Fri, 6 May 2022 10:26:18 -0500 [thread overview]
Message-ID: <YnU+GuPGiFcBXQJg@swahl-home.5wahls.com> (raw)
In-Reply-To: <BL1PR11MB5271099D98542F8A3F877D4E8CC59@BL1PR11MB5271.namprd11.prod.outlook.com>
On Fri, May 06, 2022 at 08:12:11AM +0000, Tian, Kevin wrote:
> > From: David Woodhouse <dwmw2@infradead.org>
> > Sent: Friday, May 6, 2022 3:17 PM
> >
> > On Fri, 2022-05-06 at 06:49 +0000, Tian, Kevin wrote:
> > > > From: Baolu Lu <baolu.lu@linux.intel.com>
> > > >
> > > > > --- a/include/linux/dmar.h
> > > > > +++ b/include/linux/dmar.h
> > > > > @@ -19,7 +19,7 @@
> > > > > struct acpi_dmar_header;
> > > > >
> > > > > #ifdef CONFIG_X86
> > > > > -# define DMAR_UNITS_SUPPORTED MAX_IO_APICS
> > > > > +# define DMAR_UNITS_SUPPORTED 640
> > > > > #else
> > > > > # define DMAR_UNITS_SUPPORTED 64
> > > > > #endif
> > >
> > > ... is it necessary to permanently do 10x increase which wastes memory
> > > on most platforms which won't have such need.
> >
> > I was just looking at that. It mostly adds about 3½ KiB to each struct
> > dmar_domain.
> >
> > I think the only actual static array is the dmar_seq_ids bitmap which
> > grows to 640 *bits* which is fairly negligible, and the main growth is
> > that it adds about 3½ KiB to each struct dmar_domain for the
> > iommu_refcnt[] and iommu_did[] arrays.
>
> Thanks for the quick experiment! though the added material is
> negligible it's cleaner to me if having a way to configure it as
> discussed below.
>
> >
> > > Does it make more sense to have a configurable approach similar to
> > > CONFIG_NR_CPUS? or even better can we just replace those static
> > > arrays with dynamic allocation so removing this restriction
> > > completely?
> >
> > Hotplug makes that fun, but I suppose you only need to grow the array
> > in a given struct dmar_domain if you actually add a device to it that's
> > behind a newly added IOMMU. I don't know if the complexity of making it
> > fully dynamic is worth it though. We could make it a config option,
> > and/or a command line option (perhaps automatically derived from
> > CONFIG_NR_CPUS).
>
> either config option or command line option is OK to me. Probably
> the former is simpler given no need to dynamically expand the
> static array. btw though deriving from CONFIG_NR_CPUS could work
> in this case it is unclear why tying the two together is necessary in
> concept, e.g. is there guarantee that the number of IOMMUs must
> be smaller than the number of CPUs in a platform?
>
> >
> > If it wasn't for hotplug, I think we'd know the right number by the
> > time we actually need it anyway, wouldn't we? Can we have a heuristic
> > for how many DMAR units are likely to be hotplugged? Is it as simple as
> > the ratio of present to not-yet-present CPUs in MADT?
>
> Probably. But I don't have enough knowledge on DMAR hotplug to
> judge (e.g. whether it's strictly tied to CPU hotplug and if yes whether
> there could be multiple IOMMUs hotplugged together with a CPU
> socket)...
>
> Thanks
> Kevin
Would anyone be more comfortable if we only increase the limit where
MAXSMP is set?
i.e.
#if defined(CONFIG_X86) && defined(CONFIG_MAXSMP)
# define DMAR_UNITS_SUPPORTED 640
#elif defined(CONFIG_X86)
# define DMAR_UNITS_SUPPORTED MAX_IO_APICS
#else
# define DMAR_UNITS_SUPPORTED 64
#endif
Thank you all for your time looking at this.
--> Steve Wahl
--
Steve Wahl, Hewlett Packard Enterprise
next prev parent reply other threads:[~2022-05-06 15:26 UTC|newest]
Thread overview: 37+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-05-05 19:46 [PATCH] iommu/vt-d: Increase DMAR_UNITS_SUPPORTED Steve Wahl
2022-05-06 5:57 ` Baolu Lu
2022-05-06 6:49 ` Tian, Kevin
2022-05-06 7:10 ` Rodel, Jorg
2022-05-06 7:47 ` Tian, Kevin
2022-05-06 7:16 ` David Woodhouse
2022-05-06 8:12 ` Tian, Kevin
2022-05-06 15:26 ` Steve Wahl [this message]
2022-05-10 1:16 ` Tian, Kevin
2022-05-10 19:06 ` Steve Wahl
2022-05-11 3:36 ` Tian, Kevin
2022-05-12 15:13 ` [PATCH v2] iommu/vt-d: Make DMAR_UNITS_SUPPORTED a config setting Steve Wahl
2022-05-12 23:12 ` Steve Wahl
2022-05-13 2:09 ` Baolu Lu
2022-05-18 19:58 ` Steve Wahl
2022-05-23 6:43 ` Tian, Kevin
2022-06-13 20:38 ` Jerry Snitselaar
2022-06-14 1:33 ` Baolu Lu
2022-06-13 20:57 ` Jerry Snitselaar
2022-06-14 1:36 ` Baolu Lu
2022-06-14 1:44 ` Jerry Snitselaar
2022-06-14 1:51 ` Baolu Lu
2022-06-14 1:54 ` Jerry Snitselaar
2022-06-14 2:21 ` Baolu Lu
2022-06-14 16:45 ` Steve Wahl
2022-06-14 19:01 ` Jerry Snitselaar
2022-06-14 21:12 ` Steve Wahl
2022-06-15 1:38 ` Baolu Lu
2022-06-15 15:02 ` Steve Wahl
2022-06-15 18:36 ` [PATCH v3] " Steve Wahl
2022-06-15 18:39 ` Jerry Snitselaar
2022-06-22 14:52 ` Baolu Lu
2022-06-22 15:05 ` Jerry Snitselaar
2022-06-22 15:11 ` Steve Wahl
2022-06-23 2:29 ` Baolu Lu
2022-06-23 2:51 ` Jerry Snitselaar
2022-06-23 3:38 ` Baolu Lu
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=YnU+GuPGiFcBXQJg@swahl-home.5wahls.com \
--to=steve.wahl@hpe.com \
--cc=baolu.lu@linux.intel.com \
--cc=dwmw2@infradead.org \
--cc=iommu@lists.linux-foundation.org \
--cc=jroedel@suse.de \
--cc=kevin.tian@intel.com \
--cc=kyung.min.park@intel.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mike.travis@hpe.com \
--cc=russ.anderson@hpe.com \
--cc=sivanich@hpe.com \
--cc=will@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).