From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2D6F4C64E7B for ; Mon, 30 Nov 2020 03:07:23 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id AD27920757 for ; Mon, 30 Nov 2020 03:07:22 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=intel.onmicrosoft.com header.i=@intel.onmicrosoft.com header.b="VVYKwH6V" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org AD27920757 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=intel.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.40633.73537 (Exim 4.92) (envelope-from ) id 1kjZWu-0007Rg-C9; Mon, 30 Nov 2020 03:07:08 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 40633.73537; Mon, 30 Nov 2020 03:07:08 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1kjZWu-0007RZ-8p; Mon, 30 Nov 2020 03:07:08 +0000 Received: by outflank-mailman (input) for mailman id 40633; Mon, 30 Nov 2020 03:07:07 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1kjZWt-0007RU-OJ for xen-devel@lists.xenproject.org; Mon, 30 Nov 2020 03:07:07 +0000 Received: from mga07.intel.com (unknown [134.134.136.100]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id e7fe85b0-0154-4540-934b-9406e029c01b; Mon, 30 Nov 2020 03:07:04 +0000 (UTC) Received: from orsmga005.jf.intel.com ([10.7.209.41]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 Nov 2020 19:07:03 -0800 Received: from orsmsx605.amr.corp.intel.com ([10.22.229.18]) by orsmga005.jf.intel.com with ESMTP; 29 Nov 2020 19:07:03 -0800 Received: from orsmsx605.amr.corp.intel.com (10.22.229.18) by ORSMSX605.amr.corp.intel.com (10.22.229.18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.1713.5; Sun, 29 Nov 2020 19:07:02 -0800 Received: from ORSEDG601.ED.cps.intel.com (10.7.248.6) by orsmsx605.amr.corp.intel.com (10.22.229.18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.1713.5 via Frontend Transport; Sun, 29 Nov 2020 19:07:02 -0800 Received: from NAM10-BN7-obe.outbound.protection.outlook.com (104.47.70.103) by edgegateway.intel.com (134.134.137.102) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.1713.5; Sun, 29 Nov 2020 19:06:56 -0800 Received: from MWHPR11MB1645.namprd11.prod.outlook.com (2603:10b6:301:b::12) by MW3PR11MB4569.namprd11.prod.outlook.com (2603:10b6:303:54::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3611.25; Mon, 30 Nov 2020 03:06:54 +0000 Received: from MWHPR11MB1645.namprd11.prod.outlook.com ([fe80::31c9:44c6:7323:61ac]) by MWHPR11MB1645.namprd11.prod.outlook.com ([fe80::31c9:44c6:7323:61ac%7]) with mapi id 15.20.3611.025; Mon, 30 Nov 2020 03:06:54 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: e7fe85b0-0154-4540-934b-9406e029c01b IronPort-SDR: 7x5EfeQ6LRUMslcqprJIhhakWdj5yenc81tRIBK7KSnCVjq4KT/hJXFu/BLxOg1YqHZI6fgDuu S6zBnDktAa5w== X-IronPort-AV: E=McAfee;i="6000,8403,9820"; a="236700356" X-IronPort-AV: E=Sophos;i="5.78,379,1599548400"; d="scan'208";a="236700356" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False IronPort-SDR: KOkg9CcsSOTE69ilnfcGgYIxohDbg21j5zwuvStvNniDxf0eiCmvB0qrdPifOxphQRRFfxpt0B n7EtHknCna1Q== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.78,379,1599548400"; d="scan'208";a="548908692" ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=glxYktrDvShonLEjb3MXrLujlKaSiQ1h+dRLlsnrIprQ0IgsXGS0NKmVGdvvctOj8CeCeWx2pFGlFCZ9feW1h7Kwqas6ZwJ7tof9RxB6kcMk2ZYN6KQiIIwLjSDEgy5iY58/nP9BDf0nE7Nn/7KXpdnJGO+oESYfpwXZncmJlI+Hg0M4jCjWDwu9GBjChgWHmd/JasRXuf/HZy6cy6wIWtR7a4H7i6H0QgNlhzplhXqSOeNWanl7qJWgK1GKsMVrQqQ2hXOlFpxFwTM5DU8vPaQJJt9QM2TarllE3H3fGCfU38HT6PLWF/XfF4hNypkQqXg62YvVhWuwSNVwq6RpVw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=iiovQkWpoKh7jYGCVmlBa94pRx4OT7IoM7ixCi81ijM=; b=GAz+Pgrpoqy6Rc8uXoYliPL5HZNGgfvxsGsOBanTMFKBNBAhYBBroMQs96ypMZMpYqZZhHZ5SWdokdrpsx+9XeFor/CM30RD4X4m7xYVCOjZBi9qCC7Bne/LsbNSlZCJ8LElzB2tqMcz+1dqQDIArXMk4xBJ1TAWFunBZtf1PnDr8GA7fe3Lq4HTo1C0kzhrcYBStAB458bWAEeJgit8amH7kjkSLUP56yBN0YxrZcfO3koZ3efkkNhaOUBzHlieEVN0OlRq9MsagwxFLRvziaTAWdkEGqFFzZgz5Us/M7hcaIney7nl7DHwz0JckBess5yNmPpnQY4dZhj3rewCyQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel.onmicrosoft.com; s=selector2-intel-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=iiovQkWpoKh7jYGCVmlBa94pRx4OT7IoM7ixCi81ijM=; b=VVYKwH6VGc8VG9YfuUKJ/tFXoC0d5UWlktE/sDxBYxSkuhhTnCxfBvdHJ4+k0f3jKk+V6tm9lpMLSW/SiVUbg8g3DsjmYvaQebKfpeKeSPcRlmoBFXZcQVNxl7F8mOx6WVnWVVJzNHW4Aa+QojptxsqOzpNqnCZxpWUVZOP+N18= From: "Tian, Kevin" To: Paul Durrant , "xen-devel@lists.xenproject.org" CC: Paul Durrant Subject: RE: [PATCH v10 5/7] vtd: use a bit field for root_entry Thread-Topic: [PATCH v10 5/7] vtd: use a bit field for root_entry Thread-Index: AQHWv0COaWiEdvBtMUu0J5DvTo0K7angCUXA Date: Mon, 30 Nov 2020 03:06:54 +0000 Message-ID: References: <20201120132440.1141-1-paul@xen.org> <20201120132440.1141-6-paul@xen.org> In-Reply-To: <20201120132440.1141-6-paul@xen.org> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: dlp-version: 11.5.1.3 dlp-product: dlpe-windows dlp-reaction: no-action authentication-results: xen.org; dkim=none (message not signed) header.d=none;xen.org; dmarc=none action=none header.from=intel.com; x-originating-ip: [192.198.147.217] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: b337982f-bbeb-4bc4-8d98-08d894dcfe02 x-ms-traffictypediagnostic: MW3PR11MB4569: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:6430; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: NkIaJuH6cRjroQIHG870PuCzkes3spRu/6Upo/aiQoRxlg5Aij8+lfcSwef72zC4NEXCcnRml/G0cIYAFckCX41Z/hncriz7gHxp+GGlnqEQJJuGksTmyB2hTNv51aA8VN/DTfO9Qr29DUHaRgD/S9XbErh8YgAQe6rqcu1IsB/roSxNJk8ko80U9feDgjRxUa5XCjGoVaxQ8+w7ir+uUQ9YLu33NY554h+V1t6f0reCCi7ZDl/zWy0VwCiee94yXIv4buasli3CNhdb0DvvQImwFg1njPEziAXKXZjRck7RLzFLDQqoT5pBHNtJjGmXf7wfgP7nl520ueF6u5Z1SQ== x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MWHPR11MB1645.namprd11.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(39860400002)(136003)(346002)(396003)(376002)(366004)(33656002)(66946007)(52536014)(83380400001)(66556008)(64756008)(8936002)(66446008)(66476007)(5660300002)(76116006)(8676002)(2906002)(478600001)(9686003)(26005)(316002)(110136005)(55016002)(4326008)(6506007)(186003)(71200400001)(86362001)(7696005);DIR:OUT;SFP:1102; x-ms-exchange-antispam-messagedata: =?us-ascii?Q?jvwlXsSu4GW/MQMHwjwZcGwT6dhfKa+WF8qMN1PJAVZ5GlxwjWlsZCZAltx3?= =?us-ascii?Q?4nGC4VbLu82nt+uXuaktVgfZs71EIEzB1tv7/zoOOC75u9S+1VaKTyTR9Xag?= =?us-ascii?Q?ZfBNrT8n2fFlLtoz3AQLb6RmGqApI5smQ1tPTdzO5pt/teed3EmiP66J4Jui?= =?us-ascii?Q?7/OrE91NyQ7oyi6swgWMptCvndY2y5GUBUnlHEcDoskBfXyKJr6ehDZJwG1i?= =?us-ascii?Q?NxF9MiPMLKjx5P/EkhXShSytRn8OeXGkJ4DFG0Swt9lO4Jc/XwRttMKdC5aE?= =?us-ascii?Q?3xtQOuVmapWmETBR26SXKkYa7PRkTEOiDUVUr0Xo1/rh2Bq/lRtcdTLD9nfI?= =?us-ascii?Q?fDNybx0NdIS/PxaIVzFGpUxffHwvg0kgYQMhSxIaXrrGFo+Xb0bD9Zz519Dc?= =?us-ascii?Q?8H/nM4DGWC4UcuzJgCNv2FN5In2pz+nbFc6c/z+e3h4SFCO/unHlR5Qqpg/y?= =?us-ascii?Q?yxChdj+ScXUfsBalkVeh0Hblk0k3600S+bctLVz3Du9tBsd4v9LakZBgIqp1?= =?us-ascii?Q?OP9z0cvbZFtJ7Qxffr3qMWH4oqlSiVV/WoUz4Xc1dkDclaXLqptdlOraGzeY?= =?us-ascii?Q?NHVvl9KhtuBIPDEmJJewiRn6GrQBvl45EN8+OfRvS57XYuJ08uoDI9i9GNiU?= =?us-ascii?Q?OIqOdhQWSxPACGn64clMb/3TLh9p+w0//8O9gW0XlK6DICW3BbJsbdLbmq1I?= =?us-ascii?Q?m7RpHnM4sU1x8iTR5JMBOmv8StX+fh0/XHmCSMiQGYkgcSIJWCK49HGxDmsP?= =?us-ascii?Q?TJ/i22t4uZnLslUzqLmz7LMA9UvMmRrDYoJS7jpg0aEFFaJbGAOBsaJ31NpC?= =?us-ascii?Q?Qs7KtZyHPItECJ2kx9T49VesqVLoaw89d1re6RZV6teKcNsdZVM4O+Ik3CQe?= =?us-ascii?Q?Upp0+WZM9E77Ao8ACl1sum72gO2ZVxoqXgEsnSzpDV7jHv6uPk86T1kcNsXG?= =?us-ascii?Q?mN8ZdZzK6GsApW1YDjrUp8ND10ZHilp0pJob39Pa0Po=3D?= x-ms-exchange-transport-forked: True Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: MWHPR11MB1645.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: b337982f-bbeb-4bc4-8d98-08d894dcfe02 X-MS-Exchange-CrossTenant-originalarrivaltime: 30 Nov 2020 03:06:54.7261 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: ZdaiBpTK0ze/CPhsGm7NhYEVMdKhEQ23Obj6vQa4AnATMjE6LwTjidO6DI6AkLSsqLn0p7fTjIHyJwAtP3Xckg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW3PR11MB4569 X-OriginatorOrg: intel.com > From: Paul Durrant > Sent: Friday, November 20, 2020 9:25 PM >=20 > From: Paul Durrant >=20 > This makes the code a little easier to read and also makes it more consis= tent > with iremap_entry. >=20 > Also take the opportunity to tidy up the implementation of > device_in_domain(). >=20 > Signed-off-by: Paul Durrant Reviewed-by: > --- > Cc: Kevin Tian >=20 > v10: > - Small tweaks requested by Jan > - Remove macros in favour of direct field access > - Add missing barrier >=20 > v4: > - New in v4 > --- > xen/drivers/passthrough/vtd/iommu.c | 9 +++++---- > xen/drivers/passthrough/vtd/iommu.h | 25 ++++++++++++------------- > xen/drivers/passthrough/vtd/utils.c | 6 +++--- > xen/drivers/passthrough/vtd/x86/ats.c | 27 +++++++++++++++------------ > 4 files changed, 35 insertions(+), 32 deletions(-) >=20 > diff --git a/xen/drivers/passthrough/vtd/iommu.c > b/xen/drivers/passthrough/vtd/iommu.c > index d136fe36883b..1a038541f0a3 100644 > --- a/xen/drivers/passthrough/vtd/iommu.c > +++ b/xen/drivers/passthrough/vtd/iommu.c > @@ -237,7 +237,7 @@ static u64 bus_to_context_maddr(struct vtd_iommu > *iommu, u8 bus) > ASSERT(spin_is_locked(&iommu->lock)); > root_entries =3D (struct root_entry *)map_vtd_domain_page(iommu- > >root_maddr); > root =3D &root_entries[bus]; > - if ( !root_present(*root) ) > + if ( !root->p ) > { > maddr =3D alloc_pgtable_maddr(1, iommu->node); > if ( maddr =3D=3D 0 ) > @@ -245,11 +245,12 @@ static u64 bus_to_context_maddr(struct > vtd_iommu *iommu, u8 bus) > unmap_vtd_domain_page(root_entries); > return 0; > } > - set_root_value(*root, maddr); > - set_root_present(*root); > + root->ctp =3D paddr_to_pfn(maddr); > + smp_wmb(); > + root->p =3D true; > iommu_sync_cache(root, sizeof(struct root_entry)); > } > - maddr =3D (u64) get_context_addr(*root); > + maddr =3D pfn_to_paddr(root->ctp); > unmap_vtd_domain_page(root_entries); > return maddr; > } > diff --git a/xen/drivers/passthrough/vtd/iommu.h > b/xen/drivers/passthrough/vtd/iommu.h > index 216791b3d634..b14628eec260 100644 > --- a/xen/drivers/passthrough/vtd/iommu.h > +++ b/xen/drivers/passthrough/vtd/iommu.h > @@ -184,21 +184,20 @@ > #define dma_frcd_source_id(c) (c & 0xffff) > #define dma_frcd_page_addr(d) (d & (((u64)-1) << 12)) /* low 64 bit */ >=20 > -/* > - * 0: Present > - * 1-11: Reserved > - * 12-63: Context Ptr (12 - (haw-1)) > - * 64-127: Reserved > - */ > struct root_entry { > - u64 val; > - u64 rsvd1; > + union { > + struct { uint64_t lo, hi; }; > + struct { > + /* 0 - 63 */ > + bool p:1; > + unsigned int reserved0:11; > + uint64_t ctp:52; > + > + /* 64 - 127 */ > + uint64_t reserved1; > + }; > + }; > }; > -#define root_present(root) ((root).val & 1) > -#define set_root_present(root) do {(root).val |=3D 1;} while(0) > -#define get_context_addr(root) ((root).val & PAGE_MASK_4K) > -#define set_root_value(root, value) \ > - do {(root).val |=3D ((value) & PAGE_MASK_4K);} while(0) >=20 > struct context_entry { > u64 lo; > diff --git a/xen/drivers/passthrough/vtd/utils.c > b/xen/drivers/passthrough/vtd/utils.c > index 4febcf506d8a..5f25a86a535c 100644 > --- a/xen/drivers/passthrough/vtd/utils.c > +++ b/xen/drivers/passthrough/vtd/utils.c > @@ -112,15 +112,15 @@ void print_vtd_entries(struct vtd_iommu *iommu, > int bus, int devfn, u64 gmfn) > return; > } >=20 > - printk(" root_entry[%02x] =3D %"PRIx64"\n", bus, root_entry[bus].= val); > - if ( !root_present(root_entry[bus]) ) > + printk(" root_entry[%02x] =3D %"PRIx64"\n", bus, root_entry[bus].= lo); > + if ( !root_entry[bus].p ) > { > unmap_vtd_domain_page(root_entry); > printk(" root_entry[%02x] not present\n", bus); > return; > } >=20 > - val =3D root_entry[bus].val; > + val =3D pfn_to_paddr(root_entry[bus].ctp); > unmap_vtd_domain_page(root_entry); > ctxt_entry =3D map_vtd_domain_page(val); > if ( ctxt_entry =3D=3D NULL ) > diff --git a/xen/drivers/passthrough/vtd/x86/ats.c > b/xen/drivers/passthrough/vtd/x86/ats.c > index 04d702b1d6b1..fec969ef75bb 100644 > --- a/xen/drivers/passthrough/vtd/x86/ats.c > +++ b/xen/drivers/passthrough/vtd/x86/ats.c > @@ -74,8 +74,8 @@ int ats_device(const struct pci_dev *pdev, const struct > acpi_drhd_unit *drhd) > static bool device_in_domain(const struct vtd_iommu *iommu, > const struct pci_dev *pdev, uint16_t did) > { > - struct root_entry *root_entry; > - struct context_entry *ctxt_entry =3D NULL; > + struct root_entry *root_entry, *root_entries; > + struct context_entry *context_entry, *context_entries =3D NULL; > unsigned int tt; > bool found =3D false; >=20 > @@ -85,25 +85,28 @@ static bool device_in_domain(const struct > vtd_iommu *iommu, > return false; > } >=20 > - root_entry =3D map_vtd_domain_page(iommu->root_maddr); > - if ( !root_present(root_entry[pdev->bus]) ) > + root_entries =3D (struct root_entry *)map_vtd_domain_page(iommu- > >root_maddr); > + root_entry =3D &root_entries[pdev->bus]; > + if ( !root_entry->p ) > goto out; >=20 > - ctxt_entry =3D map_vtd_domain_page(root_entry[pdev->bus].val); > - if ( context_domain_id(ctxt_entry[pdev->devfn]) !=3D did ) > + context_entries =3D map_vtd_domain_page(root_entry->ctp); > + context_entry =3D &context_entries[pdev->devfn]; > + if ( context_domain_id(*context_entry) !=3D did ) > goto out; >=20 > - tt =3D context_translation_type(ctxt_entry[pdev->devfn]); > + tt =3D context_translation_type(*context_entry); > if ( tt !=3D CONTEXT_TT_DEV_IOTLB ) > goto out; >=20 > found =3D true; > -out: > - if ( root_entry ) > - unmap_vtd_domain_page(root_entry); >=20 > - if ( ctxt_entry ) > - unmap_vtd_domain_page(ctxt_entry); > + out: > + if ( root_entries ) > + unmap_vtd_domain_page(root_entries); > + > + if ( context_entries ) > + unmap_vtd_domain_page(context_entries); >=20 > return found; > } > -- > 2.20.1