From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 83585C433E7 for ; Tue, 13 Oct 2020 23:06:13 +0000 (UTC) Received: from dpdk.org (dpdk.org [92.243.14.124]) by mail.kernel.org (Postfix) with ESMTP id 8E2FA22202 for ; Tue, 13 Oct 2020 23:06:12 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=intel.onmicrosoft.com header.i=@intel.onmicrosoft.com header.b="xCnMR71k" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8E2FA22202 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=intel.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=dev-bounces@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id ECF841DBCB; Wed, 14 Oct 2020 01:06:09 +0200 (CEST) Received: from mga18.intel.com (mga18.intel.com [134.134.136.126]) by dpdk.org (Postfix) with ESMTP id 463351DACB for ; Wed, 14 Oct 2020 01:06:07 +0200 (CEST) IronPort-SDR: nu9MpbJvZ5d7GHBpI8MvKC7ndPFF9Q4/Y4YgUxhohca0IpZwyeyirL5Au568YmU0s1Lxvh0YcY bxdzmodN/bVQ== X-IronPort-AV: E=McAfee;i="6000,8403,9773"; a="153833821" X-IronPort-AV: E=Sophos;i="5.77,372,1596524400"; d="scan'208";a="153833821" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga007.jf.intel.com ([10.7.209.58]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Oct 2020 16:06:05 -0700 IronPort-SDR: wLUKiaUlS1Z96IJQOdIlja2WqPE9/8/smTwUjzTbsQuCnCd6GlEDbm+U3Wcs0/xD6CSK/WPUzD DsZ0u0E6FqiA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.77,372,1596524400"; d="scan'208";a="357189123" Received: from orsmsx606.amr.corp.intel.com ([10.22.229.19]) by orsmga007.jf.intel.com with ESMTP; 13 Oct 2020 16:06:04 -0700 Received: from orsmsx601.amr.corp.intel.com (10.22.229.14) by ORSMSX606.amr.corp.intel.com (10.22.229.19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.1713.5; Tue, 13 Oct 2020 16:06:04 -0700 Received: from ORSEDG602.ED.cps.intel.com (10.7.248.7) by orsmsx601.amr.corp.intel.com (10.22.229.14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.1713.5 via Frontend Transport; Tue, 13 Oct 2020 16:06:04 -0700 Received: from NAM10-MW2-obe.outbound.protection.outlook.com (104.47.55.104) by edgegateway.intel.com (134.134.137.103) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.1713.5; Tue, 13 Oct 2020 16:06:02 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=kxcq8QQGIuCK4kLjD+GeX6sTtHTR37GOMrdwVg9+kUQwmkHdyJhnWIWZ8PNphGqPGM3QACe31NNMuwvIrMy6RchU/D0fu7Rqj2KnUv4syMIMOnxFqQGdWZ5mQoFUiry3fjL0bmtflj4wR3kCrywwTt5J0IMdorKus3Tx7zy5ne+v6T4agc+ZC4VA7eXh2MCw0d91wseTswR8VpZ0yF46Cvyvg+P09Xrm9A4ojnwpha/q+aboEtzISqHx7CvJ4D5gSaUdq6UR5lVnwcjODhhWu+ggPSYaRsNOWW5IMks4ze/0px8sCTQq7Ey9s8KJ33RRB4zMxnI9sZDXXwdqjH7K/Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=UJptWB8LlvLjwGhjlFLlIj9s6KXpy/zmFiszeEzissE=; b=hdmJUl6/z7o61EUUo1RBBUnzy8Ilga0GpUuCaCkuAD/Fd8wsTFypNY++k9SM2NeDnRMx/gHQ+BEmfTHEfWhZHatRCiWKRcvDP9ZGqmwfM/jHDlNorvjfCAiV3qU5qGZK0q/H3jhco3r6orlaWDwgYhzzC+yvImj3Na3PysEFOmmi2469X9qpISJlDX8XHYv2FKBIrc7PSJz0YKbbS1dzPuqvKdQViWyc2R4Lmi5aaOymioAtyWwmTxwkifEAdxwrv1x4OLbH4QLBBpHjdgNmw0oQkDL9+O9+FTbo80Ak0PMQoCR7a1qV8L0cQKEe0r/j/BQwd94yAgMaihX6vof0Og== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel.onmicrosoft.com; s=selector2-intel-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=UJptWB8LlvLjwGhjlFLlIj9s6KXpy/zmFiszeEzissE=; b=xCnMR71kZtL34sYacZXS3YmJHyyt00iJIX/fkmR7+PBv9hIjTdnWJGwvdnJmpLhiQjoVHoEWqSUw30P4DJtwlnmyS4DqCFuEP+O8q3IX0LsO6bSgVGbJnjmpSojKPOWxv+3eE0UeTkWmiNxHUqbUYCDVPBHwfbjJyDghtxWT9II= Received: from BYAPR11MB3303.namprd11.prod.outlook.com (2603:10b6:a03:18::15) by BYAPR11MB3670.namprd11.prod.outlook.com (2603:10b6:a03:f8::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3455.24; Tue, 13 Oct 2020 23:05:59 +0000 Received: from BYAPR11MB3303.namprd11.prod.outlook.com ([fe80::e1c2:7611:6087:665f]) by BYAPR11MB3303.namprd11.prod.outlook.com ([fe80::e1c2:7611:6087:665f%7]) with mapi id 15.20.3455.031; Tue, 13 Oct 2020 23:05:59 +0000 From: "Harris, James R" To: "Niu, Yawei" , =?utf-8?B?R2HDq3RhbiBSaXZldA==?= CC: "dev@dpdk.org" , "spdk@lists.01.org" , "Vanda, Sydney M" Thread-Topic: [dpdk-dev] FW: [SPDK] Re: Potential defect in pci_unplug() Thread-Index: AQHWlf4aWfDE3Ik0A0WcjDzhW0kr0KmPgUmAgAWGPoCAAMH2AA== Date: Tue, 13 Oct 2020 23:05:58 +0000 Message-ID: <993D1C06-858E-4A07-9E65-2001EFF49D0B@intel.com> References: <748FDEF7-6BB0-4D27-B630-4C991D06B6F8@intel.com> <98D9624D-F475-425B-9B06-F13CD9C4884C@intel.com> <20201009160959.dmqhwtqktbyagxbk@u256.net> <378B66E2-6A70-446B-9AC4-622B165C45DB@intel.com> In-Reply-To: <378B66E2-6A70-446B-9AC4-622B165C45DB@intel.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: user-agent: Microsoft-MacOutlook/16.41.20091302 authentication-results: intel.com; dkim=none (message not signed) header.d=none;intel.com; dmarc=none action=none header.from=intel.com; x-originating-ip: [70.176.111.225] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 6aa3f3ea-83c7-435a-983f-08d86fcc8c58 x-ms-traffictypediagnostic: BYAPR11MB3670: x-ld-processed: 46c98d88-e344-4ed4-8496-4ed7712e255d,ExtAddr x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:565; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: Xo+UdlGprQYyM2RLfx/OyDKQXJNNVibkbfDfZQuWrrQL9h6OWfBJ3bbVj8HsfgLOJ0kW9fiipPDy3WCNwh/DiwKS+qXGLbi85W1x0S8l8lRKCtIJLVxqc7GXyMAnkL/PVl4XH6R7iJqdl027uBWteXvb5+UAlz39Zu1NUp9SlmsIhKmPMHHcqN0/oMZoCOW8YRe8/n/P0ywwmY04HpwQyRKlmucB6QCbSjGj8mxlVfeYRH3T20lokFVGqJQaq6CAV0qQH7spglA8doJJiF5UvPynCBOL6oZNQmvUBeDU+ou3OHvOwOlToap/dALMNCU1lXqWUlJe9vgY3Se4UdNQJoEdmuI9OnycieieMy5GAtkFV3rt//H+MKsQx4CR1/hA5/WpDApahFjTmcqQRi8eIg== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BYAPR11MB3303.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(136003)(366004)(376002)(346002)(396003)(39860400002)(71200400001)(4326008)(53546011)(86362001)(66556008)(6506007)(66946007)(2906002)(6512007)(2616005)(6486002)(76116006)(64756008)(8676002)(26005)(66476007)(66446008)(8936002)(107886003)(186003)(5660300002)(36756003)(478600001)(83080400001)(966005)(316002)(110136005)(33656002)(83380400001)(54906003)(66574015); DIR:OUT; SFP:1102; x-ms-exchange-antispam-messagedata: +/HkuWH1iIPSTeefahhwmouGKIvdPUDMj0NRC0SV/Rrqc74OdIqdeyy2P400PG0265QiujGawVoy2Uo6oLtPOdElNv8lJ/O0fNfY53MDo5Giplm48p0C1VnTtLjZ0ShSarYuJneS1hbjAyFKyWJFjTH77GF8pnoO9KGpYC3IF4bRUmJTBqdqqdVaRPLpSfD1KI+Kxb/V+wmBUZ8RV7QhwaokccO6WcWudD5F6Z02ZAJSjyj1g2E7MTrESnQZJ1cw5oKxl3IgHpiQrABdEbNzKhcVGA0NHJZolgcd7yLVnE8j1kyinzxOudikq/k53MJqLCnguZOkf7oaEz2Xt3kaDUZzr2thgHBOhRwdOP3OTZYYOnfuYuXFhZNlqYu/PUcLJC12sCwUoECLQtPwC9HgX4jlMA3OVMbWUR6Wnq/MuyPDNdDdmzip2wA0O/3uXQXf69LcS89hL58+TMft/D3zpHWK700YvDPZnAx9fMKAN4M03xmHgIf0u/lPcYVvvpUIDFSgsxAFhpPOCxlljPKJiIO0iJJywCw7Y8qo4nCPRDQMFkD2qcLzGRr4nZ4V+aASryZ1j1POoDrVALIwSBbDjIarMZck353H4Lr8f+m7RizjeGbFf5UHslYN8MMLGe4ljBcoBqDVVhg948eiBUN1sw== Content-Type: text/plain; charset="utf-8" Content-ID: <81223D9FF8301448BEA0355D45190D50@namprd11.prod.outlook.com> Content-Transfer-Encoding: base64 MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: BYAPR11MB3303.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 6aa3f3ea-83c7-435a-983f-08d86fcc8c58 X-MS-Exchange-CrossTenant-originalarrivaltime: 13 Oct 2020 23:05:59.0273 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: xL//cU0Ux0EvEpLSFuKXMtvTTNImcj7k22JEan8oOVZP0JykqeWYLlxqjcdA0SpA0Nwvn2Nls9xRztC4YJMF1bq/3M+8GhH1DZklTdpAdrA= X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR11MB3670 X-OriginatorOrg: intel.com Subject: Re: [dpdk-dev] FW: [SPDK] Re: Potential defect in pci_unplug() X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" DQrvu79PbiAxMC8xMi8yMCwgOTozMSBQTSwgIk5pdSwgWWF3ZWkiIDx5YXdlaS5uaXVAaW50ZWwu Y29tPiB3cm90ZToNCg0KICAgIFRoYW5rcyBhIGxvdCBmb3IgdGhlIHJlcGx5LCBHYWV0YW4hDQoN CiAgICBTbyBzZWVtcyB3ZSBoYXZlIHRvIGV4cGxpY2l0bHkgdXNlIGZhaWxzYWZlIFBNRCB0byBk ZWFsIHdpdGggdGhpcyByZS1wbHVnIGNhc2U/IEknbSBub3QgcXVpdGUgc3VyZSBpZiB0aGF0J2xs IGNvb3BlcmF0ZSB3aXRoIGN1cnJlbnQgU1BESyBob3RwbHVnIGNvZGUgc2VhbWxlc3NseS4NCg0K ICAgIEkgdGhpbmsgaXQgd291bGQgYmUgZ3JlYXQgaWYgU1BESy9EUERLIGNhbiBmaXggaXQgaW50 ZXJuYWxseSAocmVjb25zdHJ1Y3QgdGhlIGRldmFyZ3Mgb24gcmUtcGx1ZyBzb21laG93PykgYW5k IHByb3ZpZGUgYSB0cmFuc3BhcmVudCBob3RwbHVnIGNhcGFiaWxpdHkgdG8gYXBwcy4NCg0KICAg IFRoYW5rcw0KICAgIC1OaXUNCg0KWWVzLCB0aGFua3MgR2HDq3Rhbi4gIEl0IG1ha2VzIHNlbnNl IHRoZSBEUERLIG9ubHkgaG9ub3JzIHRoZSBQQ0kgd2hpdGVsaXN0IGZvciB0aGUgc3BlY2lmaWMg ZGV2aWNlcyB0aGF0IHdlcmUgcHJlc2VudCBhdCB0aG9zZSBCREZzIGF0IGluaXQgdGltZS4NCg0K Tml1IC0gdGhlIGZhaWxzYWZlIFBNRCBpc24ndCBuZWVkZWQgaGVyZS4gIEkgdGhpbmsgeW91IGNh biBnZXQgd2hhdCB5b3UgbmVlZCBieSBhdm9pZGluZyB0aGUgd2hpdGVsaXN0IGNvbXBsZXRlbHku ICBMZXQncyBjb250aW51ZSB0aGF0IHBhcnQgb2YgdGhlIGRpc2N1c3Npb24gb24gdGhlIFNQREsg bWFpbGluZyBsaXN0Lg0KDQpUaGFua3MsDQoNCi1KaW0NCiANCg0KICAgIE9uIDIwMjAvMTAvMTAs IDEyOjEwIEFNLCAiR2HDq3RhbiBSaXZldCIgPGdyaXZlQHUyNTYubmV0PiB3cm90ZToNCg0KICAg ICAgICBIZWxsbywNCg0KICAgICAgICBPbiAyOS8wOS8yMCAwMToxNSArMDAwMCwgTml1LCBZYXdl aSB3cm90ZToNCiAgICAgICAgPg0KICAgICAgICA+DQogICAgICAgID4gT24gMjAyMC85LzI4LCAx MTo0NCBQTSwgIkhhcnJpcywgSmFtZXMgUiIgPGphbWVzLnIuaGFycmlzQGludGVsLmNvbT4gd3Jv dGU6DQogICAgICAgID4NCiAgICAgICAgPiAgICAgSGkgTml1LA0KICAgICAgICA+DQogICAgICAg ID4gICAgIEkgYWdyZWUsIHRoaXMgZG9lc24ndCBsb29rIHJpZ2h0LiAgQ291bGQgeW91IGZpbGUg YW4gU1BESyBpc3N1ZSBmb3IgdGhpcyB0byBtYWtlIHN1cmUgd2UgdHJhY2sgaXQ/ICBBbmQgdGhl biB0cnkgc2VuZGluZyBhbiBlLW1haWwgdG8gdGhlIERQREsgbWFpbGluZyBsaXN0PyAgSSdtIG9w ZW4gdG8gc3VibWl0dGluZyBhIHBhdGNoIHRvIG91ciBEUERLIHN1Ym1vZHVsZSBzaG9ydC10ZXJt LCBidXQgb25seSBpZiB3ZSBnZXQgYWdyZWVtZW50IHdpdGggRFBESyBjb21tdW5pdHkgdGhhdCB0 aGlzIGZpeCBpcyBjb3JyZWN0Lg0KICAgICAgICA+DQogICAgICAgID4gICAgIFRoYW5rcywNCiAg ICAgICAgPg0KICAgICAgICA+ICAgICBKaW0NCiAgICAgICAgPg0KICAgICAgICA+DQogICAgICAg ID4gICAgIE9uIDkvMjgvMjAsIDEyOjE3IEFNLCAiTml1LCBZYXdlaSIgPHlhd2VpLm5pdUBpbnRl bC5jb20+IHdyb3RlOg0KICAgICAgICA+DQogICAgICAgID4gICAgICAgICBIaSwNCiAgICAgICAg Pg0KICAgICAgICA+ICAgICAgICAgSW4gdGhlIHBjaV91bnBsdWcoKSAoZHBkay9kcml2ZXJzL2J1 cy9wY2kvcGNpX2NvbW1vbi5jKSwgd2h5IGRvIHdlIGNhbGwgcnRlX2RldmFyZ3NfcmVtb3ZlKCkg dG8gcmVtb3ZlIHRoZSBzYXZlZCBkZXZpY2UgYXJncz8gVGhhdCBsb29rcyBhIGRlZmVjdCB0byBt ZSwgc2luY2UgdGhhdOKAmWxsIHJlc3VsdCBpbiB0aGUgaG90IHJlbW92ZWQgZGV2aWNlIGZhaWxl ZCB0byBiZSBkZXRlY3RlZCB3aGVuIGl04oCZcyBwbHVnZ2VkIGJhY2sgKHdoZW4gd2hpdGUgbGlz dCBvcHRpb24gd2FzIHByb3ZpZGVkKSwgdGhlIHNpdHVhdGlvbiBpcyBkZXNjcmliZWQgZm9sbG93 aW5nOg0KICAgICAgICA+DQogICAgICAgID4gICAgICAgICAgIDEuICBBcHAgc3RhcnRzIHdpdGgg d2hpdGUgbGlzdCBvcHRpb24gcHJvdmlkZWQsIGxldOKAmXMgc3VwcG9zZSB0aGUgZGV2aWNlIGlu IHdoaXRlIGxpc3QgaXM6IDAwMDA6ODE6MDAuMDsNCiAgICAgICAgPiAgICAgICAgICAgMi4gIHJ0 ZV9kZXZhcmdzX2FkZCgpIGlzIGNhbGxlZCB0byBhZGQgdGhpcyBkZXZpY2UgYXJnIG9uIHJ0ZV9l YWxfaW5pdCgpOw0KICAgICAgICA+ICAgICAgICAgICAzLiAgSXNzdWUg4oCcZWNobyAxID4gL3N5 cy9idXMvcGNpL2RldmljZXMvMDAwMFw6ODFcOjAwLjAvcmVtb3Zl4oCdIHRvIGhvdCByZW1vdmUg dGhlIGRldmljZTsNCiAgICAgICAgPiAgICAgICAgICAgNC4gIHBjaV91bnBsdWcoKSBpcyBjYWxs ZWQgdG8gcmVtb3ZlIHRoZSBkZXZpY2UsIGFuZCBydGVfZGV2YXJnc19yZW1vdmUoKSBpcyBjYWxs ZWQsIHNvIHRoaXMgZGV2aWNlIGFyZyBmb3Igd2hpdGUgbGlzdCBpcyByZW1vdmUgdG9vOw0KICAg ICAgICA+ICAgICAgICAgICA1LiAgSXNzdWUg4oCcZWNobyAxID4gL3N5cy9idXMvcGNpL3Jlc2Nh buKAnSB0aGVuIOKAnFBDSV9XSElURUxJU1Q94oCdMDAwMDo4MTowMC4w4oCdIHNwZGsvc2NyaXB0 L3NldHVwLnNo4oCdIHRvIGRvIGhvdCBwbHVnOw0KICAgICAgICA+ICAgICAgICAgICA2LiAgSW4g cGNpX3NjYW5fb25lKCksIG5ldyBkZXYgaXMgZ2VuZXJhdGVkLCBob3dldmVyLCB0aGUgZGV2LT5k ZXZpY2UuZGV2YXJncyBpcyBzZXQgTlVMTCBzaW5jZSB0aGUgZGV2aWNlIGFyZ3MgaGFzIGJlZW4g cmVtb3ZlZCBvbiBkZXZpY2UgdW5wbHVnOw0KICAgICAgICA+ICAgICAgICAgICA3LiAgcnRlX3Bj aV9wcm9iZSgpIGRvZXMgd2hpdGUgbGlzdCBzY2FuLCBob3dldmVyLCBpdCB1bmV4cGVjdGVkbHkg c2tpcHMgdGhlIGRldmljZSBiZWNhdXNlIHRoZSBkZXZhcmdzIG9mIHRoZSBkZXZpY2UgaXMgTlVM TCBub3c7DQogICAgICAgID4NCiAgICAgICAgPiAgICAgICAgIEkgZG9u4oCZdCBmdWxseSB1bmRl cnN0YW5kIHRoZSBEUERLIGNvZGUsIGJ1dCB0aGlzIHJ0ZV9kZXZhcmdzX3JlbW92ZSgpIGluIHBj aV91bnBsdWcoKSBkb2VzbuKAmXQgbWFrZSBzZW5zZSB0byBtZSAod2hlbiBJIGNvbW1lbnRlZCBp dCBvdXQsIHNlZW1zIGV2ZXJ5dGhpbmcgd2lsbCB3b3JrIGFzIGV4cGVjdGVkKS4gSXMgdGhpcyBh IGdsaXRjaCBvciBJIG1pc3NlZCBhbnl0aGluZz8NCg0KICAgICAgICBUaGlzIGlzIG5vdCBhIGds aXRjaCwgdGhlIGJlaGF2aW9yIGlzIGludGVuZGVkLg0KICAgICAgICBXaGVuIGEgZGV2aWNlIGlz IHVucGx1Z2dlZCBmcm9tIHRoZSBzeXN0ZW0sIHRoZXJlIGlzIG5vIGV4cGVjdGF0aW9uIHRoYXQN CiAgICAgICAgaXQgd2lsbCByZWFwcGVhciBhdCB0aGUgc2FtZSBQQ0kgYWRkcmVzcy4NCg0KICAg ICAgICBXaGVuIGFuIGFwcGxpY2F0aW9uIHJlY2VpdmVzIHRoZSBSTVYgZXZlbnQgZm9yIHRoZSBk ZXZpY2UsIGl0IGlzIGFsZXJ0ZWQNCiAgICAgICAgdGhhdCB0aGUgZGV2aWNlIHdhcyByZW1vdmVk LiBJdCBpcyBpdHMgcmVzcG9uc2liaWxpdHkgdGhlbiB0byBkZXRlY3QNCiAgICAgICAgd2hlbiBh IGRldmljZSBpdCB3YW50cyB0byB1c2UgaXMgYWRkZWQgYmFjay4gSXQgY2FuIGRvIHNvIHZpYSB1 ZGV2IG9uDQogICAgICAgIGxpbnV4IGZvciBleGFtcGxlLCBidXQgaXQgaXMgdGlnaHRseSBjb3Vw bGVkIHdpdGggdGhlIG9wZXJhdGluZyBzeXN0ZW0NCiAgICAgICAgb3IgZXZlbiB0aGUgZGlzdHJp YnV0aW9uLiBCU0QgYW5kIHdpbmRvd3Mgd29uJ3Qgd29yayB0aGF0IHdheS4NCg0KICAgICAgICBU aGlzIGlzIHRoZSByZWFzb24gdGhpcyByZXNwb25zaWJpbGl0eSB3YXMgbGVmdCB0byB0aGUgYXBw bGljYXRpb24uDQoNCiAgICAgICAgVGhlIGZhaWxzYWZlIFBNRCBhdHRlbXB0cyB0byBvZmZlciB0 cmFuc3BhcmVudCBob3RwbHVnIGNhcGFiaWxpdHkgdG8NCiAgICAgICAgZXhpc3RpbmcgYXBwbGlj YXRpb25zLiBJdCBoYWQgdG8gZGVhbCB3aXRoIHRoaXMgZXhhY3QgaXNzdWUuIFRoZSBzb2x1dGlv bg0KICAgICAgICB1c2VkIHRoZXJlIGhhcyBiZWVuIHRvIHVzZSBlaXRoZXI6DQoNCiAgICAgICAg ICAqIHRoZSBleGVjKCkgc3ViLWRldmljZSB0eXBlLCB0aGF0IHRha2VzIGEgc2hlbGwgY29tbWFu ZCBhcw0KICAgICAgICAgICAgcGFyYW1ldGVyLCB3aGljaCBzaG91bGQgcmV0dXJuIGEgdmFsaWQg ZGV2aWNlIGFmdGVyIHN1Y2Nlc3NmdWwNCiAgICAgICAgICAgIGV4ZWN1dGlvbi4gSW50ZW50aW9u IGlzIGZvciB0aGUgdXNlciB0byBkZWNsYXJlIGEgc2NyaXB0IHRoYXQgd2lsbA0KICAgICAgICAg ICAgZGV0ZWN0IGRldmljZSBob3RwbHVnIG9uIHRoZSBzeXN0ZW0gdXNpbmcgb3RoZXIgbWF0Y2hl cyB0aGFuIFBDSQ0KICAgICAgICAgICAgYWRkcmVzcyAoYXMgaXQgaXMgdW5yZWxpYWJsZSksIHN1 Y2ggYXMgTUFDIGFkZHJlc3MuDQoNCiAgICAgICAgICAqIHRoZSBmZCgpIHN1Yi1kZXZpY2UgdHlw ZSwgd2hpY2ggaXMgYSBwaXBlIHdoZXJlIGFub3RoZXIgcGllY2Ugb2YgY29kZQ0KICAgICAgICAg ICAgd2lsbCBzZW5kIGRldmljZSBJRCBpbnRvLiBUaGlzIGlzIHdoYXQgdmRldl9uZXR2c2MgZG9l cyBjdXJyZW50bHk6DQogICAgICAgICAgICBpdCBkZXRlY3RzIG9uIGl0cyBvd24gdmFsaWQgTmV0 VlNDIGRldmljZXMgYW5kIHdoZW4gb25lIGlzIHN1aXRhYmxlLA0KICAgICAgICAgICAgaXQgc2Vu ZHMgaXRzIFBDSSBpZCBpbiB0aGlzIHBpcGUgdG8gZmFpbHNhZmUgZHJpdmVyLg0KDQogICAgICAg IFNlZSBodHRwczovL2RvYy5kcGRrLm9yZy9ndWlkZXMvbmljcy9mYWlsX3NhZmUuaHRtbA0KICAg ICAgICAgICAgc2VjdGlvbiA1NS4zLjEuIEZhaWwtc2FmZSBjb21tYW5kIGxpbmUgcGFyYW1ldGVy cyBmb3IgZG9jdW1lbnRhdGlvbi4NCg0KICAgICAgICBXaGVuIGEgbmV3IGRldmljZSBpcyBob3Rw bHVnZ2VkIGludG8gRFBESywgaXQgaXMgZGVzY3JpYmVkIGJ5IGFuDQogICAgICAgIHJ0ZV9kZXZh cmdzLiBUaGlzIG5ldyBkZXZhcmdzIGlzIHRoZSBvbmUgdGhhdCB3aWxsIGJlIHVzZWQgZm9yDQog ICAgICAgIGlkZW50aWZpY2F0aW9uIGR1cmluZyBzY2FuIGFuZCBwcm9iZS4gQXMgaXQgcmVwbGFj ZXMgdGhlIG9sZCwgdGhlIG9sZA0KICAgICAgICBvbmUgbmVlZHMgdG8gYmUgcmVtb3ZlZCBhcyBp dCBpcyBzdGFsZS4NCg0KICAgICAgICBSZWdhcmRzLA0KICAgICAgICAtLQ0KICAgICAgICBHYcOr dGFuDQoNCg0KDQo= From mboxrd@z Thu Jan 1 00:00:00 1970 Content-Type: multipart/mixed; boundary="===============8444715883128778679==" MIME-Version: 1.0 From: Harris, James R Subject: [SPDK] Re: [dpdk-dev] FW: Re: Potential defect in pci_unplug() Date: Tue, 13 Oct 2020 23:05:58 +0000 Message-ID: <993D1C06-858E-4A07-9E65-2001EFF49D0B@intel.com> In-Reply-To: 378B66E2-6A70-446B-9AC4-622B165C45DB@intel.com List-ID: To: spdk@lists.01.org --===============8444715883128778679== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable =EF=BB=BFOn 10/12/20, 9:31 PM, "Niu, Yawei" wrote: Thanks a lot for the reply, Gaetan! So seems we have to explicitly use failsafe PMD to deal with this re-pl= ug case? I'm not quite sure if that'll cooperate with current SPDK hotplug = code seamlessly. I think it would be great if SPDK/DPDK can fix it internally (reconstru= ct the devargs on re-plug somehow?) and provide a transparent hotplug capab= ility to apps. Thanks -Niu Yes, thanks Ga=C3=ABtan. It makes sense the DPDK only honors the PCI white= list for the specific devices that were present at those BDFs at init time. Niu - the failsafe PMD isn't needed here. I think you can get what you nee= d by avoiding the whitelist completely. Let's continue that part of the di= scussion on the SPDK mailing list. Thanks, -Jim = On 2020/10/10, 12:10 AM, "Ga=C3=ABtan Rivet" wrote: Hello, On 29/09/20 01:15 +0000, Niu, Yawei wrote: > > > On 2020/9/28, 11:44 PM, "Harris, James R" wrote: > > Hi Niu, > > I agree, this doesn't look right. Could you file an SPDK iss= ue for this to make sure we track it? And then try sending an e-mail to th= e DPDK mailing list? I'm open to submitting a patch to our DPDK submodule = short-term, but only if we get agreement with DPDK community that this fix = is correct. > > Thanks, > > Jim > > > On 9/28/20, 12:17 AM, "Niu, Yawei" wr= ote: > > Hi, > > In the pci_unplug() (dpdk/drivers/bus/pci/pci_common.c), = why do we call rte_devargs_remove() to remove the saved device args? That l= ooks a defect to me, since that=E2=80=99ll result in the hot removed device= failed to be detected when it=E2=80=99s plugged back (when white list opti= on was provided), the situation is described following: > > 1. App starts with white list option provided, let=E2= =80=99s suppose the device in white list is: 0000:81:00.0; > 2. rte_devargs_add() is called to add this device arg = on rte_eal_init(); > 3. Issue =E2=80=9Cecho 1 > /sys/bus/pci/devices/0000\:= 81\:00.0/remove=E2=80=9D to hot remove the device; > 4. pci_unplug() is called to remove the device, and rt= e_devargs_remove() is called, so this device arg for white list is remove t= oo; > 5. Issue =E2=80=9Cecho 1 > /sys/bus/pci/rescan=E2=80= =9D then =E2=80=9CPCI_WHITELIST=3D=E2=80=9D0000:81:00.0=E2=80=9D spdk/scrip= t/setup.sh=E2=80=9D to do hot plug; > 6. In pci_scan_one(), new dev is generated, however, t= he dev->device.devargs is set NULL since the device args has been removed o= n device unplug; > 7. rte_pci_probe() does white list scan, however, it u= nexpectedly skips the device because the devargs of the device is NULL now; > > I don=E2=80=99t fully understand the DPDK code, but this = rte_devargs_remove() in pci_unplug() doesn=E2=80=99t make sense to me (when= I commented it out, seems everything will work as expected). Is this a gli= tch or I missed anything? This is not a glitch, the behavior is intended. When a device is unplugged from the system, there is no expectation= that it will reappear at the same PCI address. When an application receives the RMV event for the device, it is al= erted that the device was removed. It is its responsibility then to detect when a device it wants to use is added back. It can do so via udev = on linux for example, but it is tightly coupled with the operating sys= tem or even the distribution. BSD and windows won't work that way. This is the reason this responsibility was left to the application. The failsafe PMD attempts to offer transparent hotplug capability to existing applications. It had to deal with this exact issue. The so= lution used there has been to use either: * the exec() sub-device type, that takes a shell command as parameter, which should return a valid device after successful execution. Intention is for the user to declare a script that w= ill detect device hotplug on the system using other matches than PCI address (as it is unreliable), such as MAC address. * the fd() sub-device type, which is a pipe where another piece o= f code will send device ID into. This is what vdev_netvsc does current= ly: it detects on its own valid NetVSC devices and when one is suit= able, it sends its PCI id in this pipe to failsafe driver. See https://doc.dpdk.org/guides/nics/fail_safe.html section 55.3.1. Fail-safe command line parameters for documenta= tion. When a new device is hotplugged into DPDK, it is described by an rte_devargs. This new devargs is the one that will be used for identification during scan and probe. As it replaces the old, the o= ld one needs to be removed as it is stale. Regards, -- Ga=C3=ABtan --===============8444715883128778679==--