iommu.lists.linux-foundation.org archive mirror
 help / color / mirror / Atom feed
From: Brice Goglin <Brice.Goglin@inria.fr>
To: Jonathan Cameron <Jonathan.Cameron@Huawei.com>,
	"Song Bao Hua (Barry Song)" <song.bao.hua@hisilicon.com>
Cc: Will Deacon <will@kernel.org>, Linuxarm <linuxarm@huawei.com>,
	"iommu@lists.linux-foundation.org"
	<iommu@lists.linux-foundation.org>,
	"robin.murphy@arm.com" <robin.murphy@arm.com>,
	"hch@lst.de" <hch@lst.de>,
	"linux-arm-kernel@lists.infradead.org"
	<linux-arm-kernel@lists.infradead.org>
Subject: Re: [PATCH] iommu/arm-smmu-v3: expose numa_node attribute to users in sysfs
Date: Wed, 8 Jul 2020 08:28:59 +0200	[thread overview]
Message-ID: <1b731c0a-d5ad-ce1e-9489-f616671959cc@inria.fr> (raw)
In-Reply-To: <20200706092649.0000676d@Huawei.com>


Le 06/07/2020 à 10:26, Jonathan Cameron a écrit :
> On Sun, 5 Jul 2020 09:53:58 +0000
> "Song Bao Hua (Barry Song)" <song.bao.hua@hisilicon.com> wrote:
>
>>> -----Original Message-----
>>> From: Will Deacon [mailto:will@kernel.org]
>>> Sent: Saturday, July 4, 2020 4:22 AM
>>> To: Song Bao Hua (Barry Song) <song.bao.hua@hisilicon.com>
>>> Cc: robin.murphy@arm.com; hch@lst.de; m.szyprowski@samsung.com;
>>> iommu@lists.linux-foundation.org; linux-arm-kernel@lists.infradead.org;
>>> Linuxarm <linuxarm@huawei.com>
>>> Subject: Re: [PATCH] iommu/arm-smmu-v3: expose numa_node attribute to
>>> users in sysfs
>>>
>>> On Sat, May 30, 2020 at 09:15:05PM +1200, Barry Song wrote:  
>>>> As tests show the latency of dma_unmap can increase dramatically while
>>>> calling them cross NUMA nodes, especially cross CPU packages, eg.
>>>> 300ns vs 800ns while waiting for the completion of CMD_SYNC in an
>>>> empty command queue. The large latency causing by remote node will
>>>> in turn make contention of the command queue more serious, and enlarge
>>>> the latency of DMA users within local NUMA nodes.
>>>>
>>>> Users might intend to enforce NUMA locality with the consideration of
>>>> the position of SMMU. The patch provides minor benefit by presenting
>>>> this information to users directly, as they might want to know it without
>>>> checking hardware spec at all.  
>>> I don't think that's a very good reason to expose things to userspace.
>>> I know sysfs shouldn't be treated as ABI, but the grim reality is that
>>> once somebody relies on this stuff then we can't change it, so I'd
>>> rather avoid exposing it unless it's absolutely necessary.  
>> Will, thanks for taking a look!
>>
>> I am not sure if it is absolutely necessary, but it is useful to users. The whole story started
>> from some users who wanted to know the hardware topology very clear by reading some
>> sysfs node just like they are able to do that for pci devices. The intention is that users can
>> know hardware topology of various devices easily from linux since they maybe don't know
>> all the hardware details.
>>
>> For pci devices, kernel has done that. And there are some other drivers out of pci
>> exposing numa_node as well. It seems it is hard to say it is absolutely necessary
>> for them too since sysfs shouldn't be treated as ABI. 
> Brice,
>
> Given hwloc is probably the most demanding user of topology information
> currently...
>
> How useful would this info be for hwloc and hwloc users?
> Sort of feels like it might be useful in some cases.
>
> The very brief description of what we have here is exposing the numa node
> of an IOMMU.  The discussion also diverted into whether it just makes sense
> to expose this for all platform devices or even do it at the device level.


Hello

We don't have anything about IOMMU in hwloc so far, likely because its
locality never mattered in the past? I guess we'll get some user
requests for it once more platforms show this issue and some
performance-critical applications are not happy with it.

Can you clarify what the whole machine topology look like? Are we
talking about some PCI devices being attached to one socket but talking
to the IOMMU of the other socket?

Brice

_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

      reply	other threads:[~2020-07-08  8:28 UTC|newest]

Thread overview: 7+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-05-30  9:15 [PATCH] iommu/arm-smmu-v3: expose numa_node attribute to users in sysfs Barry Song
2020-06-01 13:13 ` Robin Murphy
2020-06-01 20:43   ` Song Bao Hua (Barry Song)
2020-07-03 16:21 ` Will Deacon
2020-07-05  9:53   ` Song Bao Hua (Barry Song)
2020-07-06  8:26     ` Jonathan Cameron
2020-07-08  6:28       ` Brice Goglin [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1b731c0a-d5ad-ce1e-9489-f616671959cc@inria.fr \
    --to=brice.goglin@inria.fr \
    --cc=Jonathan.Cameron@Huawei.com \
    --cc=hch@lst.de \
    --cc=iommu@lists.linux-foundation.org \
    --cc=linux-arm-kernel@lists.infradead.org \
    --cc=linuxarm@huawei.com \
    --cc=robin.murphy@arm.com \
    --cc=song.bao.hua@hisilicon.com \
    --cc=will@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).