From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.8 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI, SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B0C9AC3A5A6 for ; Tue, 27 Aug 2019 07:36:32 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 72CA622CF8 for ; Tue, 27 Aug 2019 07:36:32 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=Mellanox.com header.i=@Mellanox.com header.b="SFPRB7Hk" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728335AbfH0Hga (ORCPT ); Tue, 27 Aug 2019 03:36:30 -0400 Received: from mail-eopbgr140081.outbound.protection.outlook.com ([40.107.14.81]:23169 "EHLO EUR01-VE1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1725850AbfH0Hga (ORCPT ); Tue, 27 Aug 2019 03:36:30 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=JcVyD5PWOw0Mpp4O/YY+8RHir6FrKYUXOdIbWhLDO34pfK61aJxydGHFtERZj34ztauAJaovO1XGl8K4ekkZd+gBkNmO+A7I+N/cUqF11P4FLHgfi37ArEmSsJRSCZSlCStdpPvBwxPfWMu/hGQratylW2mTj/eL9KGiRkkAs+BAw4X1ZYJ+qi6+8BxcQNnzqYCtwtuXvWqettLpkzA79CkMrjGHCW4WScCN4YKluRDlbwKfnA+jwCoM/zdQIgRAu/6gNwkRMjb4b3AS60FSwi8Br2nQgguaEcevzWhlCknTnQ/vjY2ARyFFdm7E5YrxOmbFrgx8ZwrmUH6ay1NIhQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=i3+xkHhOvA11fcRlRrXQe6HsuGG1tNz62skr+VSzxGQ=; b=RMU9f/S/Yc7enOBu4UwhfhAHhmPTG4aFwkorrtxvrJTPSZ6cqXk7JSTkVjLtbQ2TrYXexqvpn0wqrqqxY+hoNOvov7vH9+hRnRccMaNJfbZhNjLasSaLiOjLQIMAAxg/sbL1uafOkI1BgBuLocjEsdB02xI7J/4NkIeLN9NEDPcZZcNEINQWuyRgfEnOfiwpJ4FKRvm4ikHzUkG1sGDCCwUUmFDL+fX5KT5eVfwi6PosCDw2lBOvfx1nE9pE3B8lKd+v+jP+ctoaPeU6smC+OipBc8L5w45ZuN5O5ALRFLkI2slsXNGlm7Nmi6wl7nljh4BMJMWiY2Ygm1y1VusRVQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=mellanox.com; dmarc=pass action=none header.from=mellanox.com; dkim=pass header.d=mellanox.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=i3+xkHhOvA11fcRlRrXQe6HsuGG1tNz62skr+VSzxGQ=; b=SFPRB7HkPl9G24Hc/AZ1MQvQqUTnlG2tHg1R/pxhcgzpCeWS82ufv9UUaCU+zElDOFuwnZ2ril2SEfezE/10dZZKueH4juAton4Jr4Y1FMD1KlujrtXxgMapIsUPBh7papzPH93mtcYQbAZ2LIqye+7rM0NaJib/6nd4O47LhNk= Received: from AM6PR05MB5879.eurprd05.prod.outlook.com (20.179.0.76) by AM6PR05MB5126.eurprd05.prod.outlook.com (20.177.197.18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2178.18; Tue, 27 Aug 2019 07:36:19 +0000 Received: from AM6PR05MB5879.eurprd05.prod.outlook.com ([fe80::f9d8:38bc:4a18:f7a7]) by AM6PR05MB5879.eurprd05.prod.outlook.com ([fe80::f9d8:38bc:4a18:f7a7%5]) with mapi id 15.20.2178.023; Tue, 27 Aug 2019 07:36:19 +0000 From: Maxim Mikityanskiy To: Kevin Laatz CC: "netdev@vger.kernel.org" , "ast@kernel.org" , "daniel@iogearbox.net" , "bjorn.topel@intel.com" , "magnus.karlsson@intel.com" , "jakub.kicinski@netronome.com" , "jonathan.lemon@gmail.com" , Saeed Mahameed , "stephen@networkplumber.org" , "bruce.richardson@intel.com" , "ciara.loftus@intel.com" , "bpf@vger.kernel.org" , "intel-wired-lan@lists.osuosl.org" Subject: Re: [PATCH bpf-next v5 03/11] xsk: add support to allow unaligned chunk placement Thread-Topic: [PATCH bpf-next v5 03/11] xsk: add support to allow unaligned chunk placement Thread-Index: AQHVWNBt8Buhoq8L5ESScadzHmrTmacOopQA Date: Tue, 27 Aug 2019 07:36:18 +0000 Message-ID: <4a6ac11e-c8e6-e72b-ede2-f5cd96dcd5cd@mellanox.com> References: <20190730085400.10376-1-kevin.laatz@intel.com> <20190822014427.49800-1-kevin.laatz@intel.com> <20190822014427.49800-4-kevin.laatz@intel.com> In-Reply-To: <20190822014427.49800-4-kevin.laatz@intel.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: HE1PR0902CA0041.eurprd09.prod.outlook.com (2603:10a6:7:15::30) To AM6PR05MB5879.eurprd05.prod.outlook.com (2603:10a6:20b:a2::12) authentication-results: spf=none (sender IP is ) smtp.mailfrom=maximmi@mellanox.com; x-ms-exchange-messagesentrepresentingtype: 1 x-originating-ip: [95.67.35.250] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 12e4c5d8-4aef-4932-e875-08d72ac13ff6 x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: BCL:0;PCL:0;RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600166)(711020)(4605104)(1401327)(4618075)(2017052603328)(7193020);SRVR:AM6PR05MB5126; x-ms-traffictypediagnostic: AM6PR05MB5126: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:8273; x-forefront-prvs: 0142F22657 x-forefront-antispam-report: SFV:NSPM;SFS:(10009020)(4636009)(39860400002)(366004)(396003)(346002)(376002)(136003)(199004)(189003)(305945005)(8936002)(386003)(6916009)(71200400001)(86362001)(102836004)(3846002)(71190400001)(6506007)(229853002)(4326008)(99286004)(316002)(26005)(6486002)(66066001)(31696002)(53546011)(14454004)(2906002)(478600001)(14444005)(6116002)(256004)(446003)(66946007)(66556008)(66446008)(64756008)(2616005)(11346002)(486006)(476003)(6436002)(53936002)(6246003)(6512007)(53946003)(186003)(8676002)(81166006)(54906003)(76176011)(7736002)(36756003)(25786009)(52116002)(81156014)(7416002)(31686004)(30864003)(66476007)(5660300002)(579004);DIR:OUT;SFP:1101;SCL:1;SRVR:AM6PR05MB5126;H:AM6PR05MB5879.eurprd05.prod.outlook.com;FPR:;SPF:None;LANG:en;PTR:InfoNoRecords;A:1;MX:1; received-spf: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: igLb++wHmHk13x9yCm6KCFvMICM0eJnar1tCCH+NekfdzWxUc0m4BkkreQ6CDCexeUzto9Am2CL24nSlhRQSQk2EEn55htuy87CgjTe8KU5euZOSHVH/a3mrobH37DB+VfatSe9XpHt6PDD4Omy37cTUKGQ7xrQWcMLup8pe9BMSbYxChgDUJEDotLhzKoVAgWzf3TB44GmtnZHpKlc7vuzMZ/c7o7uAtkCAtnvoINIFtiAm6Pbg8/IR+z1vmheY/K3TRxUt37F2KHU/oR4c7479eXRDnCdhmqSmIIeGLwscfBqkkln6RwzH7ImeX5uf78qSKD0Kgt3iHj7EnsHWfLzEEo0ywXUXFJ+CnQbXfCMWumLDZZFbjvmbV+ED9fjox/7l9zU/kkcd7llcCSoyCCXdjNjzTKwbg8mR3x76Tvg= Content-Type: text/plain; charset="utf-8" Content-ID: <2785D5BDCF089046A22CD13CD8CEFF13@eurprd05.prod.outlook.com> Content-Transfer-Encoding: base64 MIME-Version: 1.0 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: 12e4c5d8-4aef-4932-e875-08d72ac13ff6 X-MS-Exchange-CrossTenant-originalarrivaltime: 27 Aug 2019 07:36:18.9483 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: 4jX8ZxNwh4eXwBS9QzqdnlvjB/Q+m2r0LbKtGNNcK2cPVRmmYQuIjNiF0FTF7K9oEHHaZ4YqYnN1IJ+Be2qrow== X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR05MB5126 Sender: bpf-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org T24gMjAxOS0wOC0yMiAwNDo0NCwgS2V2aW4gTGFhdHogd3JvdGU6DQo+IEN1cnJlbnRseSwgYWRk cmVzc2VzIGFyZSBjaHVuayBzaXplIGFsaWduZWQuIFRoaXMgbWVhbnMsIHdlIGFyZSB2ZXJ5DQo+ IHJlc3RyaWN0ZWQgaW4gdGVybXMgb2Ygd2hlcmUgd2UgY2FuIHBsYWNlIGNodW5rIHdpdGhpbiB0 aGUgdW1lbS4gRm9yDQo+IGV4YW1wbGUsIGlmIHdlIGhhdmUgYSBjaHVuayBzaXplIG9mIDJrLCB0 aGVuIG91ciBjaHVua3MgY2FuIG9ubHkgYmUgcGxhY2VkDQo+IGF0IDAsMmssNGssNmssOGsuLi4g YW5kIHNvIG9uIChpZS4gZXZlcnkgMmsgc3RhcnRpbmcgZnJvbSAwKS4NCj4gDQo+IFRoaXMgcGF0 Y2ggaW50cm9kdWNlcyB0aGUgYWJpbGl0eSB0byB1c2UgdW5hbGlnbmVkIGNodW5rcy4gV2l0aCB0 aGVzZQ0KPiBjaGFuZ2VzLCB3ZSBhcmUgbm8gbG9uZ2VyIGJvdW5kIHRvIGhhdmluZyB0byBwbGFj ZSBjaHVua3MgYXQgYSAyayAob3INCj4gd2hhdGV2ZXIgeW91ciBjaHVuayBzaXplIGlzKSBpbnRl cnZhbC4gU2luY2Ugd2UgYXJlIG5vIGxvbmdlciBkZWFsaW5nIHdpdGgNCj4gYWxpZ25lZCBjaHVu a3MsIHRoZXkgY2FuIG5vdyBjcm9zcyBwYWdlIGJvdW5kYXJpZXMuIENoZWNrcyBmb3IgcGFnZQ0K PiBjb250aWd1aXR5IGhhdmUgYmVlbiBhZGRlZCBpbiBvcmRlciB0byBrZWVwIHRyYWNrIG9mIHdo aWNoIHBhZ2VzIGFyZQ0KPiBmb2xsb3dlZCBieSBhIHBoeXNpY2FsbHkgY29udGlndW91cyBwYWdl Lg0KPiANCj4gU2lnbmVkLW9mZi1ieTogS2V2aW4gTGFhdHogPGtldmluLmxhYXR6QGludGVsLmNv bT4NCj4gU2lnbmVkLW9mZi1ieTogQ2lhcmEgTG9mdHVzIDxjaWFyYS5sb2Z0dXNAaW50ZWwuY29t Pg0KPiBTaWduZWQtb2ZmLWJ5OiBCcnVjZSBSaWNoYXJkc29uIDxicnVjZS5yaWNoYXJkc29uQGlu dGVsLmNvbT4NCj4gDQo+IC0tLQ0KPiB2MjoNCj4gICAgLSBBZGQgY2hlY2tzIGZvciB0aGUgZmxh Z3MgY29taW5nIGZyb20gdXNlcnNwYWNlDQo+ICAgIC0gRml4IGhvdyB3ZSBnZXQgY2h1bmtfc2l6 ZSBpbiB4c2tfZGlhZy5jDQo+ICAgIC0gQWRkIGRlZmluZXMgZm9yIG1hc2tpbmcgdGhlIG5ldyBk ZXNjcmlwdG9yIGZvcm1hdA0KPiAgICAtIE1vZGlmaWVkIHRoZSByeCBmdW5jdGlvbnMgdG8gdXNl IG5ldyBkZXNjcmlwdG9yIGZvcm1hdA0KPiAgICAtIE1vZGlmaWVkIHRoZSB0eCBmdW5jdGlvbnMg dG8gdXNlIG5ldyBkZXNjcmlwdG9yIGZvcm1hdA0KPiANCj4gdjM6DQo+ICAgIC0gQWRkIGhlbHBl ciBmdW5jdGlvbiB0byBkbyBhZGRyZXNzL29mZnNldCBtYXNraW5nL2FkZGl0aW9uDQo+IA0KPiB2 NDoNCj4gICAgLSBmaXhlZCBwYWdlX3N0YXJ0IGNhbGN1bGF0aW9uIGluIF9feHNrX3Jjdl9tZW1j cHkoKS4NCj4gICAgLSBtb3ZlIG9mZnNldCBoYW5kbGluZyB0byB0aGUgeGRwX3VtZW1fZ2V0Xyog ZnVuY3Rpb25zDQo+ICAgIC0gbW9kaWZpZWQgdGhlIGxlbiBmaWVsZCBpbiB4ZHBfdW1lbV9yZWcg c3RydWN0LiBXZSBub3cgdXNlIDE2IGJpdHMgZnJvbQ0KPiAgICAgIHRoaXMgZm9yIHRoZSBmbGFn cyBmaWVsZC4NCj4gICAgLSByZW1vdmVkIG5leHRfcGdfY29udGlnIGZpZWxkIGZyb20geGRwX3Vt ZW1fcGFnZSBzdHJ1Y3QuIFVzaW5nIGxvdyAxMg0KPiAgICAgIGJpdHMgb2YgYWRkciB0byBzdG9y ZSBmbGFncyBpbnN0ZWFkLg0KPiAgICAtIG90aGVyIG1pbm9yIGNoYW5nZXMgYmFzZWQgb24gcmV2 aWV3IGNvbW1lbnRzDQo+IA0KPiB2NToNCj4gICAgLSBBZGRlZCBhY2Nlc3NvcnMgZm9yIGdldHRp bmcgYWRkciBhbmQgb2Zmc2V0DQo+ICAgIC0gQWRkZWQgaGVscGVyIGZ1bmN0aW9uIHRvIGFkZCBv ZmZzZXQgdG8gYWRkcg0KPiAgICAtIEZpeGVkIG9mZnNldCBoYW5kbGluZyBpbiB4c2tfcmN2DQo+ ICAgIC0gUmVtb3ZlZCBiaXRmaWVsZHMgZnJvbSB4ZHBfdW1lbV9yZWcNCj4gICAgLSBBZGRlZCBz dHJ1Y3Qgc2l6ZSBjaGVja2luZyBmb3IgeGRwX3VtZW1fcmVnIGluIHhza19zZXRzb2Nrb3B0IHRv IGhhbmRsZQ0KPiAgICAgIGRpZmZlcmVudCB2ZXJzaW9ucyBvZiB0aGUgc3RydWN0Lg0KPiAgICAt IGZpeCBjb25mbGljdHMgYWZ0ZXIgJ2JwZi1hZi14ZHAtd2FrZXVwJyB3YXMgbWVyZ2VkLg0KPiAt LS0NCj4gICBpbmNsdWRlL25ldC94ZHBfc29jay5oICAgICAgfCA3NSArKysrKysrKysrKysrKysr KysrKysrKysrKystLQ0KPiAgIGluY2x1ZGUvdWFwaS9saW51eC9pZl94ZHAuaCB8ICA5ICsrKysN Cj4gICBuZXQveGRwL3hkcF91bWVtLmMgICAgICAgICAgfCAxOSArKysrKystLQ0KPiAgIG5ldC94 ZHAveHNrLmMgICAgICAgICAgICAgICB8IDk2ICsrKysrKysrKysrKysrKysrKysrKysrKysrKysr LS0tLS0tLS0NCj4gICBuZXQveGRwL3hza19kaWFnLmMgICAgICAgICAgfCAgMiArLQ0KPiAgIG5l dC94ZHAveHNrX3F1ZXVlLmggICAgICAgICB8IDY4ICsrKysrKysrKysrKysrKysrKysrKystLS0t DQo+ICAgNiBmaWxlcyBjaGFuZ2VkLCAyMzIgaW5zZXJ0aW9ucygrKSwgMzcgZGVsZXRpb25zKC0p DQo+IA0KPiBkaWZmIC0tZ2l0IGEvaW5jbHVkZS9uZXQveGRwX3NvY2suaCBiL2luY2x1ZGUvbmV0 L3hkcF9zb2NrLmgNCj4gaW5kZXggZjAyM2I5OTQwZDY0Li5jOTM5OGNlNzk2MGYgMTAwNjQ0DQo+ IC0tLSBhL2luY2x1ZGUvbmV0L3hkcF9zb2NrLmgNCj4gKysrIGIvaW5jbHVkZS9uZXQveGRwX3Nv Y2suaA0KPiBAQCAtMTYsNiArMTYsMTMgQEANCj4gICBzdHJ1Y3QgbmV0X2RldmljZTsNCj4gICBz dHJ1Y3QgeHNrX3F1ZXVlOw0KPiAgIA0KPiArLyogTWFza3MgZm9yIHhkcF91bWVtX3BhZ2UgZmxh Z3MuDQo+ICsgKiBUaGUgbG93IDEyLWJpdHMgb2YgdGhlIGFkZHIgd2lsbCBiZSAwIHNpbmNlIHRo aXMgaXMgdGhlIHBhZ2UgYWRkcmVzcywgc28gd2UNCj4gKyAqIGNhbiB1c2UgdGhlbSBmb3IgZmxh Z3MuDQo+ICsgKi8NCj4gKyNkZWZpbmUgWFNLX05FWFRfUEdfQ09OVElHX1NISUZUIDANCj4gKyNk ZWZpbmUgWFNLX05FWFRfUEdfQ09OVElHX01BU0sgKDFVTEwgPDwgWFNLX05FWFRfUEdfQ09OVElH X1NISUZUKQ0KPiArDQo+ICAgc3RydWN0IHhkcF91bWVtX3BhZ2Ugew0KPiAgIAl2b2lkICphZGRy Ow0KPiAgIAlkbWFfYWRkcl90IGRtYTsNCj4gQEAgLTI3LDggKzM0LDEyIEBAIHN0cnVjdCB4ZHBf dW1lbV9mcV9yZXVzZSB7DQo+ICAgCXU2NCBoYW5kbGVzW107DQo+ICAgfTsNCj4gICANCj4gLS8q IEZsYWdzIGZvciB0aGUgdW1lbSBmbGFncyBmaWVsZC4gKi8NCj4gLSNkZWZpbmUgWERQX1VNRU1f VVNFU19ORUVEX1dBS0VVUCAoMSA8PCAwKQ0KPiArLyogRmxhZ3MgZm9yIHRoZSB1bWVtIGZsYWdz IGZpZWxkLg0KPiArICoNCj4gKyAqIFRoZSBORUVEX1dBS0VVUCBmbGFnIGlzIDEgZHVlIHRvIHRo ZSByZXVzZSBvZiB0aGUgZmxhZ3MgZmllbGQgZm9yIHB1YmxpYw0KPiArICogZmxhZ3MuIFNlZSBp bmx1ZGUvdWFwaS9pbmNsdWRlL2xpbnV4L2lmX3hkcC5oLg0KPiArICovDQo+ICsjZGVmaW5lIFhE UF9VTUVNX1VTRVNfTkVFRF9XQUtFVVAgKDEgPDwgMSkNCj4gICANCj4gICBzdHJ1Y3QgeGRwX3Vt ZW0gew0KPiAgIAlzdHJ1Y3QgeHNrX3F1ZXVlICpmcTsNCj4gQEAgLTEyNCwxNCArMTM1LDM2IEBA IHZvaWQgeHNrX21hcF90cnlfc29ja19kZWxldGUoc3RydWN0IHhza19tYXAgKm1hcCwgc3RydWN0 IHhkcF9zb2NrICp4cywNCj4gICBpbnQgeHNrX21hcF9pbmMoc3RydWN0IHhza19tYXAgKm1hcCk7 DQo+ICAgdm9pZCB4c2tfbWFwX3B1dChzdHJ1Y3QgeHNrX21hcCAqbWFwKTsNCj4gICANCj4gK3N0 YXRpYyBpbmxpbmUgdTY0IHhza191bWVtX2V4dHJhY3RfYWRkcih1NjQgYWRkcikNCj4gK3sNCj4g KwlyZXR1cm4gYWRkciAmIFhTS19VTkFMSUdORURfQlVGX0FERFJfTUFTSzsNCj4gK30NCj4gKw0K PiArc3RhdGljIGlubGluZSB1NjQgeHNrX3VtZW1fZXh0cmFjdF9vZmZzZXQodTY0IGFkZHIpDQo+ ICt7DQo+ICsJcmV0dXJuIGFkZHIgPj4gWFNLX1VOQUxJR05FRF9CVUZfT0ZGU0VUX1NISUZUOw0K PiArfQ0KPiArDQo+ICtzdGF0aWMgaW5saW5lIHU2NCB4c2tfdW1lbV9hZGRfb2Zmc2V0X3RvX2Fk ZHIodTY0IGFkZHIpDQo+ICt7DQo+ICsJcmV0dXJuIHhza191bWVtX2V4dHJhY3RfYWRkcihhZGRy KSArIHhza191bWVtX2V4dHJhY3Rfb2Zmc2V0KGFkZHIpOw0KPiArfQ0KPiArDQo+ICAgc3RhdGlj IGlubGluZSBjaGFyICp4ZHBfdW1lbV9nZXRfZGF0YShzdHJ1Y3QgeGRwX3VtZW0gKnVtZW0sIHU2 NCBhZGRyKQ0KPiAgIHsNCj4gLQlyZXR1cm4gdW1lbS0+cGFnZXNbYWRkciA+PiBQQUdFX1NISUZU XS5hZGRyICsgKGFkZHIgJiAoUEFHRV9TSVpFIC0gMSkpOw0KPiArCXVuc2lnbmVkIGxvbmcgcGFn ZV9hZGRyOw0KPiArDQo+ICsJYWRkciA9IHhza191bWVtX2FkZF9vZmZzZXRfdG9fYWRkcihhZGRy KTsNCj4gKwlwYWdlX2FkZHIgPSAodW5zaWduZWQgbG9uZyl1bWVtLT5wYWdlc1thZGRyID4+IFBB R0VfU0hJRlRdLmFkZHI7DQo+ICsNCj4gKwlyZXR1cm4gKGNoYXIgKikocGFnZV9hZGRyICYgUEFH RV9NQVNLKSArIChhZGRyICYgflBBR0VfTUFTSyk7DQo+ICAgfQ0KPiAgIA0KPiAgIHN0YXRpYyBp bmxpbmUgZG1hX2FkZHJfdCB4ZHBfdW1lbV9nZXRfZG1hKHN0cnVjdCB4ZHBfdW1lbSAqdW1lbSwg dTY0IGFkZHIpDQo+ICAgew0KPiAtCXJldHVybiB1bWVtLT5wYWdlc1thZGRyID4+IFBBR0VfU0hJ RlRdLmRtYSArIChhZGRyICYgKFBBR0VfU0laRSAtIDEpKTsNCj4gKwlhZGRyID0geHNrX3VtZW1f YWRkX29mZnNldF90b19hZGRyKGFkZHIpOw0KPiArDQo+ICsJcmV0dXJuIHVtZW0tPnBhZ2VzW2Fk ZHIgPj4gUEFHRV9TSElGVF0uZG1hICsgKGFkZHIgJiB+UEFHRV9NQVNLKTsNCj4gICB9DQo+ICAg DQo+ICAgLyogUmV1c2UtcXVldWUgYXdhcmUgdmVyc2lvbiBvZiBGSUxMIHF1ZXVlIGhlbHBlcnMg Ki8NCj4gQEAgLTE3Miw2ICsyMDUsMTkgQEAgc3RhdGljIGlubGluZSB2b2lkIHhza191bWVtX2Zx X3JldXNlKHN0cnVjdCB4ZHBfdW1lbSAqdW1lbSwgdTY0IGFkZHIpDQo+ICAgDQo+ICAgCXJxLT5o YW5kbGVzW3JxLT5sZW5ndGgrK10gPSBhZGRyOw0KPiAgIH0NCj4gKw0KPiArLyogSGFuZGxlIHRo ZSBvZmZzZXQgYXBwcm9wcmlhdGVseSBkZXBlbmRpbmcgb24gYWxpZ25lZCBvciB1bmFsaWduZWQg bW9kZS4NCj4gKyAqIEZvciB1bmFsaWduZWQgbW9kZSwgd2Ugc3RvcmUgdGhlIG9mZnNldCBpbiB0 aGUgdXBwZXIgMTYtYml0cyBvZiB0aGUgYWRkcmVzcy4NCj4gKyAqIEZvciBhbGlnbmVkIG1vZGUs IHdlIHNpbXBseSBhZGQgdGhlIG9mZnNldCB0byB0aGUgYWRkcmVzcy4NCj4gKyAqLw0KPiArc3Rh dGljIGlubGluZSB1NjQgeHNrX3VtZW1fYWRqdXN0X29mZnNldChzdHJ1Y3QgeGRwX3VtZW0gKnVt ZW0sIHU2NCBhZGRyZXNzLA0KPiArCQkJCQkgdTY0IG9mZnNldCkNCj4gK3sNCj4gKwlpZiAodW1l bS0+ZmxhZ3MgJiBYRFBfVU1FTV9VTkFMSUdORURfQ0hVTktfRkxBRykNCj4gKwkJcmV0dXJuIGFk ZHJlc3MgKyAob2Zmc2V0IDw8IFhTS19VTkFMSUdORURfQlVGX09GRlNFVF9TSElGVCk7DQo+ICsJ ZWxzZQ0KPiArCQlyZXR1cm4gYWRkcmVzcyArIG9mZnNldDsNCj4gK30NCj4gICAjZWxzZQ0KPiAg IHN0YXRpYyBpbmxpbmUgaW50IHhza19nZW5lcmljX3JjdihzdHJ1Y3QgeGRwX3NvY2sgKnhzLCBz dHJ1Y3QgeGRwX2J1ZmYgKnhkcCkNCj4gICB7DQo+IEBAIC0yNDEsNiArMjg3LDIxIEBAIHN0YXRp YyBpbmxpbmUgc3RydWN0IHhkcF91bWVtICp4ZHBfZ2V0X3VtZW1fZnJvbV9xaWQoc3RydWN0IG5l dF9kZXZpY2UgKmRldiwNCj4gICAJcmV0dXJuIE5VTEw7DQo+ICAgfQ0KPiAgIA0KPiArc3RhdGlj IGlubGluZSB1NjQgeHNrX3VtZW1fZXh0cmFjdF9hZGRyKHU2NCBhZGRyKQ0KPiArew0KPiArCXJl dHVybiAwOw0KPiArfQ0KPiArDQo+ICtzdGF0aWMgaW5saW5lIHU2NCB4c2tfdW1lbV9leHRyYWN0 X29mZnNldCh1NjQgYWRkcikNCj4gK3sNCj4gKwlyZXR1cm4gMDsNCj4gK30NCj4gKw0KPiArc3Rh dGljIGlubGluZSB1NjQgeHNrX3VtZW1fYWRkX29mZnNldF90b19hZGRyKHU2NCBhZGRyKQ0KPiAr ew0KPiArCXJldHVybiAwOw0KPiArfQ0KPiArDQo+ICAgc3RhdGljIGlubGluZSBjaGFyICp4ZHBf dW1lbV9nZXRfZGF0YShzdHJ1Y3QgeGRwX3VtZW0gKnVtZW0sIHU2NCBhZGRyKQ0KPiAgIHsNCj4g ICAJcmV0dXJuIE5VTEw7DQo+IEBAIC0yOTAsNiArMzUxLDEyIEBAIHN0YXRpYyBpbmxpbmUgYm9v bCB4c2tfdW1lbV91c2VzX25lZWRfd2FrZXVwKHN0cnVjdCB4ZHBfdW1lbSAqdW1lbSkNCj4gICAJ cmV0dXJuIGZhbHNlOw0KPiAgIH0NCj4gICANCj4gK3N0YXRpYyBpbmxpbmUgdTY0IHhza191bWVt X2FkanVzdF9vZmZzZXQoc3RydWN0IHhkcF91bWVtICp1bWVtLCB1NjQgaGFuZGxlLA0KPiArCQkJ CQkgdTY0IG9mZnNldCkNCj4gK3sNCj4gKwlyZXR1cm4gMDsNCj4gK30NCj4gKw0KPiAgICNlbmRp ZiAvKiBDT05GSUdfWERQX1NPQ0tFVFMgKi8NCj4gICANCj4gICAjZW5kaWYgLyogX0xJTlVYX1hE UF9TT0NLX0ggKi8NCj4gZGlmZiAtLWdpdCBhL2luY2x1ZGUvdWFwaS9saW51eC9pZl94ZHAuaCBi L2luY2x1ZGUvdWFwaS9saW51eC9pZl94ZHAuaA0KPiBpbmRleCA2MmI4MGQ1N2I3MmEuLmJlMzI4 YzU5Mzg5ZCAxMDA2NDQNCj4gLS0tIGEvaW5jbHVkZS91YXBpL2xpbnV4L2lmX3hkcC5oDQo+ICsr KyBiL2luY2x1ZGUvdWFwaS9saW51eC9pZl94ZHAuaA0KPiBAQCAtMjYsNiArMjYsOSBAQA0KPiAg ICAqLw0KPiAgICNkZWZpbmUgWERQX1VTRV9ORUVEX1dBS0VVUCAoMSA8PCAzKQ0KPiAgIA0KPiAr LyogRmxhZ3MgZm9yIHhza191bWVtX2NvbmZpZyBmbGFncyAqLw0KPiArI2RlZmluZSBYRFBfVU1F TV9VTkFMSUdORURfQ0hVTktfRkxBRyAoMSA8PCAwKQ0KPiArDQo+ICAgc3RydWN0IHNvY2thZGRy X3hkcCB7DQo+ICAgCV9fdTE2IHN4ZHBfZmFtaWx5Ow0KPiAgIAlfX3UxNiBzeGRwX2ZsYWdzOw0K PiBAQCAtNjYsNiArNjksNyBAQCBzdHJ1Y3QgeGRwX3VtZW1fcmVnIHsNCj4gICAJX191NjQgbGVu OyAvKiBMZW5ndGggb2YgcGFja2V0IGRhdGEgYXJlYSAqLw0KPiAgIAlfX3UzMiBjaHVua19zaXpl Ow0KPiAgIAlfX3UzMiBoZWFkcm9vbTsNCj4gKwlfX3UzMiBmbGFnczsNCj4gICB9Ow0KPiAgIA0K PiAgIHN0cnVjdCB4ZHBfc3RhdGlzdGljcyB7DQo+IEBAIC04Nyw2ICs5MSwxMSBAQCBzdHJ1Y3Qg eGRwX29wdGlvbnMgew0KPiAgICNkZWZpbmUgWERQX1VNRU1fUEdPRkZfRklMTF9SSU5HCTB4MTAw MDAwMDAwVUxMDQo+ICAgI2RlZmluZSBYRFBfVU1FTV9QR09GRl9DT01QTEVUSU9OX1JJTkcJMHgx ODAwMDAwMDBVTEwNCj4gICANCj4gKy8qIE1hc2tzIGZvciB1bmFsaWduZWQgY2h1bmtzIG1vZGUg Ki8NCj4gKyNkZWZpbmUgWFNLX1VOQUxJR05FRF9CVUZfT0ZGU0VUX1NISUZUIDQ4DQo+ICsjZGVm aW5lIFhTS19VTkFMSUdORURfQlVGX0FERFJfTUFTSyBcDQo+ICsJKCgxVUxMIDw8IFhTS19VTkFM SUdORURfQlVGX09GRlNFVF9TSElGVCkgLSAxKQ0KPiArDQo+ICAgLyogUngvVHggZGVzY3JpcHRv ciAqLw0KPiAgIHN0cnVjdCB4ZHBfZGVzYyB7DQo+ICAgCV9fdTY0IGFkZHI7DQo+IGRpZmYgLS1n aXQgYS9uZXQveGRwL3hkcF91bWVtLmMgYi9uZXQveGRwL3hkcF91bWVtLmMNCj4gaW5kZXggMmQ2 NTc3OTI4MmExLi5lOTk3YjI2M2EwZGQgMTAwNjQ0DQo+IC0tLSBhL25ldC94ZHAveGRwX3VtZW0u Yw0KPiArKysgYi9uZXQveGRwL3hkcF91bWVtLmMNCj4gQEAgLTM0MCw2ICszNDAsNyBAQCBzdGF0 aWMgaW50IHhkcF91bWVtX2FjY291bnRfcGFnZXMoc3RydWN0IHhkcF91bWVtICp1bWVtKQ0KPiAg IA0KPiAgIHN0YXRpYyBpbnQgeGRwX3VtZW1fcmVnKHN0cnVjdCB4ZHBfdW1lbSAqdW1lbSwgc3Ry dWN0IHhkcF91bWVtX3JlZyAqbXIpDQo+ICAgew0KPiArCWJvb2wgdW5hbGlnbmVkX2NodW5rcyA9 IG1yLT5mbGFncyAmIFhEUF9VTUVNX1VOQUxJR05FRF9DSFVOS19GTEFHOw0KPiAgIAl1MzIgY2h1 bmtfc2l6ZSA9IG1yLT5jaHVua19zaXplLCBoZWFkcm9vbSA9IG1yLT5oZWFkcm9vbTsNCj4gICAJ dW5zaWduZWQgaW50IGNodW5rcywgY2h1bmtzX3Blcl9wYWdlOw0KPiAgIAl1NjQgYWRkciA9IG1y LT5hZGRyLCBzaXplID0gbXItPmxlbjsNCj4gQEAgLTM1NSw3ICszNTYsMTEgQEAgc3RhdGljIGlu dCB4ZHBfdW1lbV9yZWcoc3RydWN0IHhkcF91bWVtICp1bWVtLCBzdHJ1Y3QgeGRwX3VtZW1fcmVn ICptcikNCj4gICAJCXJldHVybiAtRUlOVkFMOw0KPiAgIAl9DQo+ICAgDQo+IC0JaWYgKCFpc19w b3dlcl9vZl8yKGNodW5rX3NpemUpKQ0KPiArCWlmIChtci0+ZmxhZ3MgJiB+KFhEUF9VTUVNX1VO QUxJR05FRF9DSFVOS19GTEFHIHwNCj4gKwkJCVhEUF9VTUVNX1VTRVNfTkVFRF9XQUtFVVApKQ0K PiArCQlyZXR1cm4gLUVJTlZBTDsNCj4gKw0KPiArCWlmICghdW5hbGlnbmVkX2NodW5rcyAmJiAh aXNfcG93ZXJfb2ZfMihjaHVua19zaXplKSkNCj4gICAJCXJldHVybiAtRUlOVkFMOw0KPiAgIA0K PiAgIAlpZiAoIVBBR0VfQUxJR05FRChhZGRyKSkgew0KPiBAQCAtMzcyLDkgKzM3NywxMSBAQCBz dGF0aWMgaW50IHhkcF91bWVtX3JlZyhzdHJ1Y3QgeGRwX3VtZW0gKnVtZW0sIHN0cnVjdCB4ZHBf dW1lbV9yZWcgKm1yKQ0KPiAgIAlpZiAoY2h1bmtzID09IDApDQo+ICAgCQlyZXR1cm4gLUVJTlZB TDsNCj4gICANCj4gLQljaHVua3NfcGVyX3BhZ2UgPSBQQUdFX1NJWkUgLyBjaHVua19zaXplOw0K PiAtCWlmIChjaHVua3MgPCBjaHVua3NfcGVyX3BhZ2UgfHwgY2h1bmtzICUgY2h1bmtzX3Blcl9w YWdlKQ0KPiAtCQlyZXR1cm4gLUVJTlZBTDsNCj4gKwlpZiAoIXVuYWxpZ25lZF9jaHVua3MpIHsN Cj4gKwkJY2h1bmtzX3Blcl9wYWdlID0gUEFHRV9TSVpFIC8gY2h1bmtfc2l6ZTsNCj4gKwkJaWYg KGNodW5rcyA8IGNodW5rc19wZXJfcGFnZSB8fCBjaHVua3MgJSBjaHVua3NfcGVyX3BhZ2UpDQo+ ICsJCQlyZXR1cm4gLUVJTlZBTDsNCj4gKwl9DQo+ICAgDQo+ICAgCWhlYWRyb29tID0gQUxJR04o aGVhZHJvb20sIDY0KTsNCj4gICANCj4gQEAgLTM4MywxMyArMzkwLDE1IEBAIHN0YXRpYyBpbnQg eGRwX3VtZW1fcmVnKHN0cnVjdCB4ZHBfdW1lbSAqdW1lbSwgc3RydWN0IHhkcF91bWVtX3JlZyAq bXIpDQo+ICAgCQlyZXR1cm4gLUVJTlZBTDsNCj4gICANCj4gICAJdW1lbS0+YWRkcmVzcyA9ICh1 bnNpZ25lZCBsb25nKWFkZHI7DQo+IC0JdW1lbS0+Y2h1bmtfbWFzayA9IH4oKHU2NCljaHVua19z aXplIC0gMSk7DQo+ICsJdW1lbS0+Y2h1bmtfbWFzayA9IHVuYWxpZ25lZF9jaHVua3MgPyBYU0tf VU5BTElHTkVEX0JVRl9BRERSX01BU0sNCj4gKwkJCQkJICAgIDogfigodTY0KWNodW5rX3NpemUg LSAxKTsNCj4gICAJdW1lbS0+c2l6ZSA9IHNpemU7DQo+ICAgCXVtZW0tPmhlYWRyb29tID0gaGVh ZHJvb207DQo+ICAgCXVtZW0tPmNodW5rX3NpemVfbm9ociA9IGNodW5rX3NpemUgLSBoZWFkcm9v bTsNCj4gICAJdW1lbS0+bnBncyA9IHNpemUgLyBQQUdFX1NJWkU7DQo+ICAgCXVtZW0tPnBncyA9 IE5VTEw7DQo+ICAgCXVtZW0tPnVzZXIgPSBOVUxMOw0KPiArCXVtZW0tPmZsYWdzID0gbXItPmZs YWdzOw0KPiAgIAlJTklUX0xJU1RfSEVBRCgmdW1lbS0+eHNrX2xpc3QpOw0KPiAgIAlzcGluX2xv Y2tfaW5pdCgmdW1lbS0+eHNrX2xpc3RfbG9jayk7DQo+ICAgDQo+IGRpZmYgLS1naXQgYS9uZXQv eGRwL3hzay5jIGIvbmV0L3hkcC94c2suYw0KPiBpbmRleCBlZTQ0MjhhODkyZmEuLjkwN2U1ZjEy MzM4ZiAxMDA2NDQNCj4gLS0tIGEvbmV0L3hkcC94c2suYw0KPiArKysgYi9uZXQveGRwL3hzay5j DQo+IEBAIC00NSw3ICs0NSw3IEBAIEVYUE9SVF9TWU1CT0woeHNrX3VtZW1faGFzX2FkZHJzKTsN Cj4gICANCj4gICB1NjQgKnhza191bWVtX3BlZWtfYWRkcihzdHJ1Y3QgeGRwX3VtZW0gKnVtZW0s IHU2NCAqYWRkcikNCj4gICB7DQo+IC0JcmV0dXJuIHhza3FfcGVla19hZGRyKHVtZW0tPmZxLCBh ZGRyKTsNCj4gKwlyZXR1cm4geHNrcV9wZWVrX2FkZHIodW1lbS0+ZnEsIGFkZHIsIHVtZW0pOw0K PiAgIH0NCj4gICBFWFBPUlRfU1lNQk9MKHhza191bWVtX3BlZWtfYWRkcik7DQo+ICAgDQo+IEBA IC0xMTUsMjEgKzExNSw0MyBAQCBib29sIHhza191bWVtX3VzZXNfbmVlZF93YWtldXAoc3RydWN0 IHhkcF91bWVtICp1bWVtKQ0KPiAgIH0NCj4gICBFWFBPUlRfU1lNQk9MKHhza191bWVtX3VzZXNf bmVlZF93YWtldXApOw0KPiAgIA0KPiArLyogSWYgYSBidWZmZXIgY3Jvc3NlcyBhIHBhZ2UgYm91 bmRhcnksIHdlIG5lZWQgdG8gZG8gMiBtZW1jcHkncywgb25lIGZvcg0KPiArICogZWFjaCBwYWdl LiBUaGlzIGlzIG9ubHkgcmVxdWlyZWQgaW4gY29weSBtb2RlLg0KPiArICovDQo+ICtzdGF0aWMg dm9pZCBfX3hza19yY3ZfbWVtY3B5KHN0cnVjdCB4ZHBfdW1lbSAqdW1lbSwgdTY0IGFkZHIsIHZv aWQgKmZyb21fYnVmLA0KPiArCQkJICAgICB1MzIgbGVuLCB1MzIgbWV0YWxlbikNCj4gK3sNCj4g Kwl2b2lkICp0b19idWYgPSB4ZHBfdW1lbV9nZXRfZGF0YSh1bWVtLCBhZGRyKTsNCj4gKw0KPiAr CWFkZHIgPSB4c2tfdW1lbV9hZGRfb2Zmc2V0X3RvX2FkZHIoYWRkcik7DQo+ICsJaWYgKHhza3Ff Y3Jvc3Nlc19ub25fY29udGlnX3BnKHVtZW0sIGFkZHIsIGxlbiArIG1ldGFsZW4pKSB7DQo+ICsJ CXZvaWQgKm5leHRfcGdfYWRkciA9IHVtZW0tPnBhZ2VzWyhhZGRyID4+IFBBR0VfU0hJRlQpICsg MV0uYWRkcjsNCj4gKwkJdTY0IHBhZ2Vfc3RhcnQgPSBhZGRyICYgfihQQUdFX1NJWkUgLSAxKTsN Cj4gKwkJdTY0IGZpcnN0X2xlbiA9IFBBR0VfU0laRSAtIChhZGRyIC0gcGFnZV9zdGFydCk7DQo+ ICsNCj4gKwkJbWVtY3B5KHRvX2J1ZiwgZnJvbV9idWYsIGZpcnN0X2xlbiArIG1ldGFsZW4pOw0K PiArCQltZW1jcHkobmV4dF9wZ19hZGRyLCBmcm9tX2J1ZiArIGZpcnN0X2xlbiwgbGVuIC0gZmly c3RfbGVuKTsNCj4gKw0KPiArCQlyZXR1cm47DQo+ICsJfQ0KPiArDQo+ICsJbWVtY3B5KHRvX2J1 ZiwgZnJvbV9idWYsIGxlbiArIG1ldGFsZW4pOw0KPiArfQ0KPiArDQo+ICAgc3RhdGljIGludCBf X3hza19yY3Yoc3RydWN0IHhkcF9zb2NrICp4cywgc3RydWN0IHhkcF9idWZmICp4ZHAsIHUzMiBs ZW4pDQo+ICAgew0KPiAtCXZvaWQgKnRvX2J1ZiwgKmZyb21fYnVmOw0KPiArCXU2NCBvZmZzZXQg PSB4cy0+dW1lbS0+aGVhZHJvb207DQo+ICsJdTY0IGFkZHIsIG1lbWNweV9hZGRyOw0KPiArCXZv aWQgKmZyb21fYnVmOw0KPiAgIAl1MzIgbWV0YWxlbjsNCj4gLQl1NjQgYWRkcjsNCj4gICAJaW50 IGVycjsNCj4gICANCj4gLQlpZiAoIXhza3FfcGVla19hZGRyKHhzLT51bWVtLT5mcSwgJmFkZHIp IHx8DQo+ICsJaWYgKCF4c2txX3BlZWtfYWRkcih4cy0+dW1lbS0+ZnEsICZhZGRyLCB4cy0+dW1l bSkgfHwNCj4gICAJICAgIGxlbiA+IHhzLT51bWVtLT5jaHVua19zaXplX25vaHIgLSBYRFBfUEFD S0VUX0hFQURST09NKSB7DQo+ICAgCQl4cy0+cnhfZHJvcHBlZCsrOw0KPiAgIAkJcmV0dXJuIC1F Tk9TUEM7DQo+ICAgCX0NCj4gICANCj4gLQlhZGRyICs9IHhzLT51bWVtLT5oZWFkcm9vbTsNCj4g LQ0KPiAgIAlpZiAodW5saWtlbHkoeGRwX2RhdGFfbWV0YV91bnN1cHBvcnRlZCh4ZHApKSkgew0K PiAgIAkJZnJvbV9idWYgPSB4ZHAtPmRhdGE7DQo+ICAgCQltZXRhbGVuID0gMDsNCj4gQEAgLTEz OCw5ICsxNjAsMTEgQEAgc3RhdGljIGludCBfX3hza19yY3Yoc3RydWN0IHhkcF9zb2NrICp4cywg c3RydWN0IHhkcF9idWZmICp4ZHAsIHUzMiBsZW4pDQo+ICAgCQltZXRhbGVuID0geGRwLT5kYXRh IC0geGRwLT5kYXRhX21ldGE7DQo+ICAgCX0NCj4gICANCj4gLQl0b19idWYgPSB4ZHBfdW1lbV9n ZXRfZGF0YSh4cy0+dW1lbSwgYWRkcik7DQo+IC0JbWVtY3B5KHRvX2J1ZiwgZnJvbV9idWYsIGxl biArIG1ldGFsZW4pOw0KPiAtCWFkZHIgKz0gbWV0YWxlbjsNCj4gKwltZW1jcHlfYWRkciA9IHhz a191bWVtX2FkanVzdF9vZmZzZXQoeHMtPnVtZW0sIGFkZHIsIG9mZnNldCk7DQo+ICsJX194c2tf cmN2X21lbWNweSh4cy0+dW1lbSwgbWVtY3B5X2FkZHIsIGZyb21fYnVmLCBsZW4sIG1ldGFsZW4p Ow0KPiArDQo+ICsJb2Zmc2V0ICs9IG1ldGFsZW47DQo+ICsJYWRkciA9IHhza191bWVtX2FkanVz dF9vZmZzZXQoeHMtPnVtZW0sIGFkZHIsIG9mZnNldCk7DQo+ICAgCWVyciA9IHhza3FfcHJvZHVj ZV9iYXRjaF9kZXNjKHhzLT5yeCwgYWRkciwgbGVuKTsNCj4gICAJaWYgKCFlcnIpIHsNCj4gICAJ CXhza3FfZGlzY2FyZF9hZGRyKHhzLT51bWVtLT5mcSk7DQo+IEBAIC0xODUsNiArMjA5LDcgQEAg aW50IHhza19nZW5lcmljX3JjdihzdHJ1Y3QgeGRwX3NvY2sgKnhzLCBzdHJ1Y3QgeGRwX2J1ZmYg KnhkcCkNCj4gICB7DQo+ICAgCXUzMiBtZXRhbGVuID0geGRwLT5kYXRhIC0geGRwLT5kYXRhX21l dGE7DQo+ICAgCXUzMiBsZW4gPSB4ZHAtPmRhdGFfZW5kIC0geGRwLT5kYXRhOw0KPiArCXU2NCBv ZmZzZXQgPSB4cy0+dW1lbS0+aGVhZHJvb207DQo+ICAgCXZvaWQgKmJ1ZmZlcjsNCj4gICAJdTY0 IGFkZHI7DQo+ICAgCWludCBlcnI7DQo+IEBAIC0xOTYsMTcgKzIyMSwxNyBAQCBpbnQgeHNrX2dl bmVyaWNfcmN2KHN0cnVjdCB4ZHBfc29jayAqeHMsIHN0cnVjdCB4ZHBfYnVmZiAqeGRwKQ0KPiAg IAkJZ290byBvdXRfdW5sb2NrOw0KPiAgIAl9DQo+ICAgDQo+IC0JaWYgKCF4c2txX3BlZWtfYWRk cih4cy0+dW1lbS0+ZnEsICZhZGRyKSB8fA0KPiArCWlmICgheHNrcV9wZWVrX2FkZHIoeHMtPnVt ZW0tPmZxLCAmYWRkciwgeHMtPnVtZW0pIHx8DQo+ICAgCSAgICBsZW4gPiB4cy0+dW1lbS0+Y2h1 bmtfc2l6ZV9ub2hyIC0gWERQX1BBQ0tFVF9IRUFEUk9PTSkgew0KPiAgIAkJZXJyID0gLUVOT1NQ QzsNCj4gICAJCWdvdG8gb3V0X2Ryb3A7DQo+ICAgCX0NCj4gICANCj4gLQlhZGRyICs9IHhzLT51 bWVtLT5oZWFkcm9vbTsNCj4gLQ0KPiAtCWJ1ZmZlciA9IHhkcF91bWVtX2dldF9kYXRhKHhzLT51 bWVtLCBhZGRyKTsNCj4gKwlidWZmZXIgPSB4ZHBfdW1lbV9nZXRfZGF0YSh4cy0+dW1lbSwgYWRk ciArIG9mZnNldCk7DQo+ICAgCW1lbWNweShidWZmZXIsIHhkcC0+ZGF0YV9tZXRhLCBsZW4gKyBt ZXRhbGVuKTsNCj4gLQlhZGRyICs9IG1ldGFsZW47DQo+ICsJb2Zmc2V0ICs9IG1ldGFsZW47DQo+ ICsNCj4gKwlhZGRyID0geHNrX3VtZW1fYWRqdXN0X29mZnNldCh4cy0+dW1lbSwgYWRkciwgb2Zm c2V0KTsNCj4gICAJZXJyID0geHNrcV9wcm9kdWNlX2JhdGNoX2Rlc2MoeHMtPnJ4LCBhZGRyLCBs ZW4pOw0KPiAgIAlpZiAoZXJyKQ0KPiAgIAkJZ290byBvdXRfZHJvcDsNCj4gQEAgLTI1MCw3ICsy NzUsNyBAQCBib29sIHhza191bWVtX2NvbnN1bWVfdHgoc3RydWN0IHhkcF91bWVtICp1bWVtLCBz dHJ1Y3QgeGRwX2Rlc2MgKmRlc2MpDQo+ICAgDQo+ICAgCXJjdV9yZWFkX2xvY2soKTsNCj4gICAJ bGlzdF9mb3JfZWFjaF9lbnRyeV9yY3UoeHMsICZ1bWVtLT54c2tfbGlzdCwgbGlzdCkgew0KPiAt CQlpZiAoIXhza3FfcGVla19kZXNjKHhzLT50eCwgZGVzYykpDQo+ICsJCWlmICgheHNrcV9wZWVr X2Rlc2MoeHMtPnR4LCBkZXNjLCB1bWVtKSkNCj4gICAJCQljb250aW51ZTsNCj4gICANCj4gICAJ CWlmICh4c2txX3Byb2R1Y2VfYWRkcl9sYXp5KHVtZW0tPmNxLCBkZXNjLT5hZGRyKSkNCj4gQEAg LTMwNCw3ICszMjksNyBAQCBzdGF0aWMgaW50IHhza19nZW5lcmljX3htaXQoc3RydWN0IHNvY2sg KnNrLCBzdHJ1Y3QgbXNnaGRyICptLA0KPiAgIAlpZiAoeHMtPnF1ZXVlX2lkID49IHhzLT5kZXYt PnJlYWxfbnVtX3R4X3F1ZXVlcykNCj4gICAJCWdvdG8gb3V0Ow0KPiAgIA0KPiAtCXdoaWxlICh4 c2txX3BlZWtfZGVzYyh4cy0+dHgsICZkZXNjKSkgew0KPiArCXdoaWxlICh4c2txX3BlZWtfZGVz Yyh4cy0+dHgsICZkZXNjLCB4cy0+dW1lbSkpIHsNCj4gICAJCWNoYXIgKmJ1ZmZlcjsNCj4gICAJ CXU2NCBhZGRyOw0KPiAgIAkJdTMyIGxlbjsNCj4gQEAgLTMzMyw3ICszNTgsNyBAQCBzdGF0aWMg aW50IHhza19nZW5lcmljX3htaXQoc3RydWN0IHNvY2sgKnNrLCBzdHJ1Y3QgbXNnaGRyICptLA0K PiAgIAkJc2tiLT5kZXYgPSB4cy0+ZGV2Ow0KPiAgIAkJc2tiLT5wcmlvcml0eSA9IHNrLT5za19w cmlvcml0eTsNCj4gICAJCXNrYi0+bWFyayA9IHNrLT5za19tYXJrOw0KPiAtCQlza2Jfc2hpbmZv KHNrYiktPmRlc3RydWN0b3JfYXJnID0gKHZvaWQgKikobG9uZylhZGRyOw0KPiArCQlza2Jfc2hp bmZvKHNrYiktPmRlc3RydWN0b3JfYXJnID0gKHZvaWQgKikobG9uZylkZXNjLmFkZHI7DQo+ICAg CQlza2ItPmRlc3RydWN0b3IgPSB4c2tfZGVzdHJ1Y3Rfc2tiOw0KPiAgIA0KPiAgIAkJZXJyID0g ZGV2X2RpcmVjdF94bWl0KHNrYiwgeHMtPnF1ZXVlX2lkKTsNCj4gQEAgLTUyNiw2ICs1NTEsMjQg QEAgc3RhdGljIHN0cnVjdCBzb2NrZXQgKnhza19sb29rdXBfeHNrX2Zyb21fZmQoaW50IGZkKQ0K PiAgIAlyZXR1cm4gc29jazsNCj4gICB9DQo+ICAgDQo+ICsvKiBDaGVjayBpZiB1bWVtIHBhZ2Vz IGFyZSBjb250aWd1b3VzLg0KPiArICogSWYgemVyby1jb3B5IG1vZGUsIHVzZSB0aGUgRE1BIGFk ZHJlc3MgdG8gZG8gdGhlIHBhZ2UgY29udGlndWl0eSBjaGVjaw0KPiArICogRm9yIGFsbCBvdGhl ciBtb2RlcyB3ZSB1c2UgYWRkciAoa2VybmVsIHZpcnR1YWwgYWRkcmVzcykNCj4gKyAqIFN0b3Jl IHRoZSByZXN1bHQgaW4gdGhlIGxvdyBiaXRzIG9mIGFkZHIuDQo+ICsgKi8NCj4gK3N0YXRpYyB2 b2lkIHhza19jaGVja19wYWdlX2NvbnRpZ3VpdHkoc3RydWN0IHhkcF91bWVtICp1bWVtLCB1MzIg ZmxhZ3MpDQo+ICt7DQo+ICsJc3RydWN0IHhkcF91bWVtX3BhZ2UgKnBncyA9IHVtZW0tPnBhZ2Vz Ow0KPiArCWludCBpLCBpc19jb250aWc7DQo+ICsNCj4gKwlmb3IgKGkgPSAwOyBpIDwgdW1lbS0+ bnBncyAtIDE7IGkrKykgew0KPiArCQlpc19jb250aWcgPSAoZmxhZ3MgJiBYRFBfWkVST0NPUFkp ID8NCj4gKwkJCShwZ3NbaV0uZG1hICsgUEFHRV9TSVpFID09IHBnc1tpICsgMV0uZG1hKSA6DQo+ ICsJCQkocGdzW2ldLmFkZHIgKyBQQUdFX1NJWkUgPT0gcGdzW2kgKyAxXS5hZGRyKTsNCj4gKwkJ cGdzW2ldLmFkZHIgKz0gaXNfY29udGlnIDw8IFhTS19ORVhUX1BHX0NPTlRJR19TSElGVDsNCj4g Kwl9DQo+ICt9DQo+ICsNCj4gICBzdGF0aWMgaW50IHhza19iaW5kKHN0cnVjdCBzb2NrZXQgKnNv Y2ssIHN0cnVjdCBzb2NrYWRkciAqYWRkciwgaW50IGFkZHJfbGVuKQ0KPiAgIHsNCj4gICAJc3Ry dWN0IHNvY2thZGRyX3hkcCAqc3hkcCA9IChzdHJ1Y3Qgc29ja2FkZHJfeGRwICopYWRkcjsNCj4g QEAgLTYxNiw2ICs2NTksOCBAQCBzdGF0aWMgaW50IHhza19iaW5kKHN0cnVjdCBzb2NrZXQgKnNv Y2ssIHN0cnVjdCBzb2NrYWRkciAqYWRkciwgaW50IGFkZHJfbGVuKQ0KPiAgIAkJZXJyID0geGRw X3VtZW1fYXNzaWduX2Rldih4cy0+dW1lbSwgZGV2LCBxaWQsIGZsYWdzKTsNCj4gICAJCWlmIChl cnIpDQo+ICAgCQkJZ290byBvdXRfdW5sb2NrOw0KPiArDQo+ICsJCXhza19jaGVja19wYWdlX2Nv bnRpZ3VpdHkoeHMtPnVtZW0sIGZsYWdzKTsNCj4gICAJfQ0KPiAgIA0KPiAgIAl4cy0+ZGV2ID0g ZGV2Ow0KPiBAQCAtNjM2LDYgKzY4MSwxMyBAQCBzdGF0aWMgaW50IHhza19iaW5kKHN0cnVjdCBz b2NrZXQgKnNvY2ssIHN0cnVjdCBzb2NrYWRkciAqYWRkciwgaW50IGFkZHJfbGVuKQ0KPiAgIAly ZXR1cm4gZXJyOw0KPiAgIH0NCj4gICANCj4gK3N0cnVjdCB4ZHBfdW1lbV9yZWdfdjEgew0KPiAr CV9fdTY0IGFkZHI7IC8qIFN0YXJ0IG9mIHBhY2tldCBkYXRhIGFyZWEgKi8NCj4gKwlfX3U2NCBs ZW47IC8qIExlbmd0aCBvZiBwYWNrZXQgZGF0YSBhcmVhICovDQo+ICsJX191MzIgY2h1bmtfc2l6 ZTsNCj4gKwlfX3UzMiBoZWFkcm9vbTsNCj4gK307DQo+ICsNCj4gICBzdGF0aWMgaW50IHhza19z ZXRzb2Nrb3B0KHN0cnVjdCBzb2NrZXQgKnNvY2ssIGludCBsZXZlbCwgaW50IG9wdG5hbWUsDQo+ ICAgCQkJICBjaGFyIF9fdXNlciAqb3B0dmFsLCB1bnNpZ25lZCBpbnQgb3B0bGVuKQ0KPiAgIHsN Cj4gQEAgLTY3MywxMCArNzI1LDE2IEBAIHN0YXRpYyBpbnQgeHNrX3NldHNvY2tvcHQoc3RydWN0 IHNvY2tldCAqc29jaywgaW50IGxldmVsLCBpbnQgb3B0bmFtZSwNCj4gICAJfQ0KPiAgIAljYXNl IFhEUF9VTUVNX1JFRzoNCj4gICAJew0KPiAtCQlzdHJ1Y3QgeGRwX3VtZW1fcmVnIG1yOw0KPiAr CQlzaXplX3QgbXJfc2l6ZSA9IHNpemVvZihzdHJ1Y3QgeGRwX3VtZW1fcmVnKTsNCj4gKwkJc3Ry dWN0IHhkcF91bWVtX3JlZyBtciA9IHt9Ow0KPiAgIAkJc3RydWN0IHhkcF91bWVtICp1bWVtOw0K PiAgIA0KPiAtCQlpZiAoY29weV9mcm9tX3VzZXIoJm1yLCBvcHR2YWwsIHNpemVvZihtcikpKQ0K PiArCQlpZiAob3B0bGVuIDwgc2l6ZW9mKHN0cnVjdCB4ZHBfdW1lbV9yZWdfdjEpKQ0KPiArCQkJ cmV0dXJuIC1FSU5WQUw7DQo+ICsJCWVsc2UgaWYgKG9wdGxlbiA8IHNpemVvZihtcikpDQo+ICsJ CQltcl9zaXplID0gc2l6ZW9mKHN0cnVjdCB4ZHBfdW1lbV9yZWdfdjEpOw0KPiArDQo+ICsJCWlm IChjb3B5X2Zyb21fdXNlcigmbXIsIG9wdHZhbCwgbXJfc2l6ZSkpDQo+ICAgCQkJcmV0dXJuIC1F RkFVTFQ7DQo+ICAgDQo+ICAgCQltdXRleF9sb2NrKCZ4cy0+bXV0ZXgpOw0KPiBkaWZmIC0tZ2l0 IGEvbmV0L3hkcC94c2tfZGlhZy5jIGIvbmV0L3hkcC94c2tfZGlhZy5jDQo+IGluZGV4IGQ1ZTA2 YzhlMGNiZi4uOTk4NmE3NTlmZTA2IDEwMDY0NA0KPiAtLS0gYS9uZXQveGRwL3hza19kaWFnLmMN Cj4gKysrIGIvbmV0L3hkcC94c2tfZGlhZy5jDQo+IEBAIC01Niw3ICs1Niw3IEBAIHN0YXRpYyBp bnQgeHNrX2RpYWdfcHV0X3VtZW0oY29uc3Qgc3RydWN0IHhkcF9zb2NrICp4cywgc3RydWN0IHNr X2J1ZmYgKm5sc2tiKQ0KPiAgIAlkdS5pZCA9IHVtZW0tPmlkOw0KPiAgIAlkdS5zaXplID0gdW1l bS0+c2l6ZTsNCj4gICAJZHUubnVtX3BhZ2VzID0gdW1lbS0+bnBnczsNCj4gLQlkdS5jaHVua19z aXplID0gKF9fdTMyKSh+dW1lbS0+Y2h1bmtfbWFzayArIDEpOw0KPiArCWR1LmNodW5rX3NpemUg PSB1bWVtLT5jaHVua19zaXplX25vaHIgKyB1bWVtLT5oZWFkcm9vbTsNCj4gICAJZHUuaGVhZHJv b20gPSB1bWVtLT5oZWFkcm9vbTsNCj4gICAJZHUuaWZpbmRleCA9IHVtZW0tPmRldiA/IHVtZW0t PmRldi0+aWZpbmRleCA6IDA7DQo+ICAgCWR1LnF1ZXVlX2lkID0gdW1lbS0+cXVldWVfaWQ7DQo+ IGRpZmYgLS1naXQgYS9uZXQveGRwL3hza19xdWV1ZS5oIGIvbmV0L3hkcC94c2tfcXVldWUuaA0K PiBpbmRleCBkZDllOTg1YzI0NjEuLjZjNjdjOWQwMjk0ZiAxMDA2NDQNCj4gLS0tIGEvbmV0L3hk cC94c2tfcXVldWUuaA0KPiArKysgYi9uZXQveGRwL3hza19xdWV1ZS5oDQo+IEBAIC0xMzQsNiAr MTM0LDE3IEBAIHN0YXRpYyBpbmxpbmUgYm9vbCB4c2txX2hhc19hZGRycyhzdHJ1Y3QgeHNrX3F1 ZXVlICpxLCB1MzIgY250KQ0KPiAgIA0KPiAgIC8qIFVNRU0gcXVldWUgKi8NCj4gICANCj4gK3N0 YXRpYyBpbmxpbmUgYm9vbCB4c2txX2Nyb3NzZXNfbm9uX2NvbnRpZ19wZyhzdHJ1Y3QgeGRwX3Vt ZW0gKnVtZW0sIHU2NCBhZGRyLA0KPiArCQkJCQkgICAgICB1NjQgbGVuZ3RoKQ0KPiArew0KPiAr CWJvb2wgY3Jvc3NfcGcgPSAoYWRkciAmIChQQUdFX1NJWkUgLSAxKSkgKyBsZW5ndGggPiBQQUdF X1NJWkU7DQo+ICsJYm9vbCBuZXh0X3BnX2NvbnRpZyA9DQo+ICsJCSh1bnNpZ25lZCBsb25nKXVt ZW0tPnBhZ2VzWyhhZGRyID4+IFBBR0VfU0hJRlQpXS5hZGRyICYNCj4gKwkJCVhTS19ORVhUX1BH X0NPTlRJR19NQVNLOw0KPiArDQo+ICsJcmV0dXJuIGNyb3NzX3BnICYmICFuZXh0X3BnX2NvbnRp ZzsNCj4gK30NCj4gKw0KPiAgIHN0YXRpYyBpbmxpbmUgYm9vbCB4c2txX2lzX3ZhbGlkX2FkZHIo c3RydWN0IHhza19xdWV1ZSAqcSwgdTY0IGFkZHIpDQo+ICAgew0KPiAgIAlpZiAoYWRkciA+PSBx LT5zaXplKSB7DQo+IEBAIC0xNDQsMjMgKzE1NSw0OSBAQCBzdGF0aWMgaW5saW5lIGJvb2wgeHNr cV9pc192YWxpZF9hZGRyKHN0cnVjdCB4c2tfcXVldWUgKnEsIHU2NCBhZGRyKQ0KPiAgIAlyZXR1 cm4gdHJ1ZTsNCj4gICB9DQo+ICAgDQo+IC1zdGF0aWMgaW5saW5lIHU2NCAqeHNrcV92YWxpZGF0 ZV9hZGRyKHN0cnVjdCB4c2tfcXVldWUgKnEsIHU2NCAqYWRkcikNCj4gK3N0YXRpYyBpbmxpbmUg Ym9vbCB4c2txX2lzX3ZhbGlkX2FkZHJfdW5hbGlnbmVkKHN0cnVjdCB4c2tfcXVldWUgKnEsIHU2 NCBhZGRyLA0KPiArCQkJCQkJdTY0IGxlbmd0aCwNCj4gKwkJCQkJCXN0cnVjdCB4ZHBfdW1lbSAq dW1lbSkNCj4gK3sNCj4gKwlhZGRyID0geHNrX3VtZW1fYWRkX29mZnNldF90b19hZGRyKGFkZHIp Ow0KPiArCWlmIChhZGRyID49IHEtPnNpemUgfHwNCg0KSSBrbm93IEkgYWxyZWFkeSBhc2tlZCBh Ym91dCBpdCwgYnV0IEkgZmVlbCB3ZSBuZWVkIHRvIGNsYXJpZnkgdGhpbmdzIA0KaGVyZSBmdXJ0 aGVyLg0KDQpUaGUgYGFkZHIgPj0gcS0+c2l6ZWAgY2hlY2sgaXMgZ29vZCwgaXQgY2hlY2tzIHRo YXQgdGhlIGFkZHJlc3MgZG9lc24ndCANCm92ZXJmbG93IHRoZSBVTUVNLiBIb3dldmVyLCB0aGUg bmV3IGVuY29kaW5nIG9mIFVNRU0gaGFuZGxlcyBjb25zaXN0cyBvZiANCnR3byBwYXJ0czogYmFz ZSBhZGRyZXNzIG9mIHRoZSBmcmFtZSAobG93IGJpdHMpIGFuZCBvZmZzZXQgKGhpZ2ggYml0cyks IA0KdGhhdCBhcmUgYWRkZWQgdG9nZXRoZXIgYnkgeHNrX3VtZW1fYWRkX29mZnNldF90b19hZGRy LiBJdCdzIHBvc3NpYmxlIHRvIA0KY3JhZnQgYW4gYWRkcmVzcyB0aGF0IGhhcyB0b28gYmlnIGJh c2UgYWRkcmVzcyBvZiB0aGUgZnJhbWUgKGFsbCANCjB4ZmYuLi5mKSwgc29tZSBub24temVybyBv ZmZzZXQsIHNvIGFmdGVyIGFkZGluZyB0aGVtIHRvZ2V0aGVyIHdlIHBhc3MgDQpgYWRkciA8IHEt PnNpemVgLCBidXQgdGhlIGJhc2UgYWRkcmVzcyBleGNlZWRzIHRoZSBVTUVNLiBJbiBteSBvcGlu aW9uLCANCndlIHNob3VsZCBub3QgYWxsb3cgc3VjaCBhZGRyZXNzZXMsIGJlY2F1c2UgdGhlIGJh c2UgYWRkcmVzcyBvZiB0aGUgDQpmcmFtZSBpcyBvdXQgb2YgYm91bmRzLCBzbyB3ZSBuZWVkIG9u ZSBtb3JlIGNoZWNrIGhlcmUuIFdoYXQgZG8geW91IA0KdGhpbmsgYWJvdXQgdGhpcyBwb2ludD8N Cg0KPiArCSAgICB4c2txX2Nyb3NzZXNfbm9uX2NvbnRpZ19wZyh1bWVtLCBhZGRyLCBsZW5ndGgp KSB7DQo+ICsJCXEtPmludmFsaWRfZGVzY3MrKzsNCj4gKwkJcmV0dXJuIGZhbHNlOw0KPiArCX0N Cj4gKw0KPiArCXJldHVybiB0cnVlOw0KPiArfQ0KPiArDQo+ICtzdGF0aWMgaW5saW5lIHU2NCAq eHNrcV92YWxpZGF0ZV9hZGRyKHN0cnVjdCB4c2tfcXVldWUgKnEsIHU2NCAqYWRkciwNCj4gKwkJ CQkgICAgICBzdHJ1Y3QgeGRwX3VtZW0gKnVtZW0pDQo+ICAgew0KPiAgIAl3aGlsZSAocS0+Y29u c190YWlsICE9IHEtPmNvbnNfaGVhZCkgew0KPiAgIAkJc3RydWN0IHhkcF91bWVtX3JpbmcgKnJp bmcgPSAoc3RydWN0IHhkcF91bWVtX3JpbmcgKilxLT5yaW5nOw0KPiAgIAkJdW5zaWduZWQgaW50 IGlkeCA9IHEtPmNvbnNfdGFpbCAmIHEtPnJpbmdfbWFzazsNCj4gICANCj4gICAJCSphZGRyID0g UkVBRF9PTkNFKHJpbmctPmRlc2NbaWR4XSkgJiBxLT5jaHVua19tYXNrOw0KPiArDQo+ICsJCWlm ICh1bWVtLT5mbGFncyAmIFhEUF9VTUVNX1VOQUxJR05FRF9DSFVOS19GTEFHKSB7DQo+ICsJCQlp ZiAoeHNrcV9pc192YWxpZF9hZGRyX3VuYWxpZ25lZChxLCAqYWRkciwNCj4gKwkJCQkJCQkgdW1l bS0+Y2h1bmtfc2l6ZV9ub2hyLA0KPiArCQkJCQkJCSB1bWVtKSkNCj4gKwkJCQlyZXR1cm4gYWRk cjsNCj4gKwkJCWdvdG8gb3V0Ow0KPiArCQl9DQo+ICsNCj4gICAJCWlmICh4c2txX2lzX3ZhbGlk X2FkZHIocSwgKmFkZHIpKQ0KPiAgIAkJCXJldHVybiBhZGRyOw0KPiAgIA0KPiArb3V0Og0KPiAg IAkJcS0+Y29uc190YWlsKys7DQo+ICAgCX0NCj4gICANCj4gICAJcmV0dXJuIE5VTEw7DQo+ICAg fQ0KPiAgIA0KPiAtc3RhdGljIGlubGluZSB1NjQgKnhza3FfcGVla19hZGRyKHN0cnVjdCB4c2tf cXVldWUgKnEsIHU2NCAqYWRkcikNCj4gK3N0YXRpYyBpbmxpbmUgdTY0ICp4c2txX3BlZWtfYWRk cihzdHJ1Y3QgeHNrX3F1ZXVlICpxLCB1NjQgKmFkZHIsDQo+ICsJCQkJICBzdHJ1Y3QgeGRwX3Vt ZW0gKnVtZW0pDQo+ICAgew0KPiAgIAlpZiAocS0+Y29uc190YWlsID09IHEtPmNvbnNfaGVhZCkg ew0KPiAgIAkJc21wX21iKCk7IC8qIEQsIG1hdGNoZXMgQSAqLw0KPiBAQCAtMTcxLDcgKzIwOCw3 IEBAIHN0YXRpYyBpbmxpbmUgdTY0ICp4c2txX3BlZWtfYWRkcihzdHJ1Y3QgeHNrX3F1ZXVlICpx LCB1NjQgKmFkZHIpDQo+ICAgCQlzbXBfcm1iKCk7DQo+ICAgCX0NCj4gICANCj4gLQlyZXR1cm4g eHNrcV92YWxpZGF0ZV9hZGRyKHEsIGFkZHIpOw0KPiArCXJldHVybiB4c2txX3ZhbGlkYXRlX2Fk ZHIocSwgYWRkciwgdW1lbSk7DQo+ICAgfQ0KPiAgIA0KPiAgIHN0YXRpYyBpbmxpbmUgdm9pZCB4 c2txX2Rpc2NhcmRfYWRkcihzdHJ1Y3QgeHNrX3F1ZXVlICpxKQ0KPiBAQCAtMjMwLDggKzI2Nywy MSBAQCBzdGF0aWMgaW5saW5lIGludCB4c2txX3Jlc2VydmVfYWRkcihzdHJ1Y3QgeHNrX3F1ZXVl ICpxKQ0KPiAgIA0KPiAgIC8qIFJ4L1R4IHF1ZXVlICovDQo+ICAgDQo+IC1zdGF0aWMgaW5saW5l IGJvb2wgeHNrcV9pc192YWxpZF9kZXNjKHN0cnVjdCB4c2tfcXVldWUgKnEsIHN0cnVjdCB4ZHBf ZGVzYyAqZCkNCj4gK3N0YXRpYyBpbmxpbmUgYm9vbCB4c2txX2lzX3ZhbGlkX2Rlc2Moc3RydWN0 IHhza19xdWV1ZSAqcSwgc3RydWN0IHhkcF9kZXNjICpkLA0KPiArCQkJCSAgICAgIHN0cnVjdCB4 ZHBfdW1lbSAqdW1lbSkNCj4gICB7DQo+ICsJaWYgKHVtZW0tPmZsYWdzICYgWERQX1VNRU1fVU5B TElHTkVEX0NIVU5LX0ZMQUcpIHsNCj4gKwkJaWYgKCF4c2txX2lzX3ZhbGlkX2FkZHJfdW5hbGln bmVkKHEsIGQtPmFkZHIsIGQtPmxlbiwgdW1lbSkpDQo+ICsJCQlyZXR1cm4gZmFsc2U7DQo+ICsN Cj4gKwkJaWYgKGQtPmxlbiA+IHVtZW0tPmNodW5rX3NpemVfbm9ociB8fCBkLT5vcHRpb25zKSB7 DQo+ICsJCQlxLT5pbnZhbGlkX2Rlc2NzKys7DQo+ICsJCQlyZXR1cm4gZmFsc2U7DQo+ICsJCX0N Cj4gKw0KPiArCQlyZXR1cm4gdHJ1ZTsNCj4gKwl9DQo+ICsNCj4gICAJaWYgKCF4c2txX2lzX3Zh bGlkX2FkZHIocSwgZC0+YWRkcikpDQo+ICAgCQlyZXR1cm4gZmFsc2U7DQo+ICAgDQo+IEBAIC0y NDUsMTQgKzI5NSwxNSBAQCBzdGF0aWMgaW5saW5lIGJvb2wgeHNrcV9pc192YWxpZF9kZXNjKHN0 cnVjdCB4c2tfcXVldWUgKnEsIHN0cnVjdCB4ZHBfZGVzYyAqZCkNCj4gICB9DQo+ICAgDQo+ICAg c3RhdGljIGlubGluZSBzdHJ1Y3QgeGRwX2Rlc2MgKnhza3FfdmFsaWRhdGVfZGVzYyhzdHJ1Y3Qg eHNrX3F1ZXVlICpxLA0KPiAtCQkJCQkJICBzdHJ1Y3QgeGRwX2Rlc2MgKmRlc2MpDQo+ICsJCQkJ CQkgIHN0cnVjdCB4ZHBfZGVzYyAqZGVzYywNCj4gKwkJCQkJCSAgc3RydWN0IHhkcF91bWVtICp1 bWVtKQ0KPiAgIHsNCj4gICAJd2hpbGUgKHEtPmNvbnNfdGFpbCAhPSBxLT5jb25zX2hlYWQpIHsN Cj4gICAJCXN0cnVjdCB4ZHBfcnh0eF9yaW5nICpyaW5nID0gKHN0cnVjdCB4ZHBfcnh0eF9yaW5n ICopcS0+cmluZzsNCj4gICAJCXVuc2lnbmVkIGludCBpZHggPSBxLT5jb25zX3RhaWwgJiBxLT5y aW5nX21hc2s7DQo+ICAgDQo+ICAgCQkqZGVzYyA9IFJFQURfT05DRShyaW5nLT5kZXNjW2lkeF0p Ow0KPiAtCQlpZiAoeHNrcV9pc192YWxpZF9kZXNjKHEsIGRlc2MpKQ0KPiArCQlpZiAoeHNrcV9p c192YWxpZF9kZXNjKHEsIGRlc2MsIHVtZW0pKQ0KPiAgIAkJCXJldHVybiBkZXNjOw0KPiAgIA0K PiAgIAkJcS0+Y29uc190YWlsKys7DQo+IEBAIC0yNjIsNyArMzEzLDggQEAgc3RhdGljIGlubGlu ZSBzdHJ1Y3QgeGRwX2Rlc2MgKnhza3FfdmFsaWRhdGVfZGVzYyhzdHJ1Y3QgeHNrX3F1ZXVlICpx LA0KPiAgIH0NCj4gICANCj4gICBzdGF0aWMgaW5saW5lIHN0cnVjdCB4ZHBfZGVzYyAqeHNrcV9w ZWVrX2Rlc2Moc3RydWN0IHhza19xdWV1ZSAqcSwNCj4gLQkJCQkJICAgICAgc3RydWN0IHhkcF9k ZXNjICpkZXNjKQ0KPiArCQkJCQkgICAgICBzdHJ1Y3QgeGRwX2Rlc2MgKmRlc2MsDQo+ICsJCQkJ CSAgICAgIHN0cnVjdCB4ZHBfdW1lbSAqdW1lbSkNCj4gICB7DQo+ICAgCWlmIChxLT5jb25zX3Rh aWwgPT0gcS0+Y29uc19oZWFkKSB7DQo+ICAgCQlzbXBfbWIoKTsgLyogRCwgbWF0Y2hlcyBBICov DQo+IEBAIC0yNzMsNyArMzI1LDcgQEAgc3RhdGljIGlubGluZSBzdHJ1Y3QgeGRwX2Rlc2MgKnhz a3FfcGVla19kZXNjKHN0cnVjdCB4c2tfcXVldWUgKnEsDQo+ICAgCQlzbXBfcm1iKCk7IC8qIEMs IG1hdGNoZXMgQiAqLw0KPiAgIAl9DQo+ICAgDQo+IC0JcmV0dXJuIHhza3FfdmFsaWRhdGVfZGVz YyhxLCBkZXNjKTsNCj4gKwlyZXR1cm4geHNrcV92YWxpZGF0ZV9kZXNjKHEsIGRlc2MsIHVtZW0p Ow0KPiAgIH0NCj4gICANCj4gICBzdGF0aWMgaW5saW5lIHZvaWQgeHNrcV9kaXNjYXJkX2Rlc2Mo c3RydWN0IHhza19xdWV1ZSAqcSkNCj4gDQoNCg== From mboxrd@z Thu Jan 1 00:00:00 1970 From: Maxim Mikityanskiy Date: Tue, 27 Aug 2019 07:36:18 +0000 Subject: [Intel-wired-lan] [PATCH bpf-next v5 03/11] xsk: add support to allow unaligned chunk placement In-Reply-To: <20190822014427.49800-4-kevin.laatz@intel.com> References: <20190730085400.10376-1-kevin.laatz@intel.com> <20190822014427.49800-1-kevin.laatz@intel.com> <20190822014427.49800-4-kevin.laatz@intel.com> Message-ID: <4a6ac11e-c8e6-e72b-ede2-f5cd96dcd5cd@mellanox.com> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit To: intel-wired-lan@osuosl.org List-ID: On 2019-08-22 04:44, Kevin Laatz wrote: > Currently, addresses are chunk size aligned. This means, we are very > restricted in terms of where we can place chunk within the umem. For > example, if we have a chunk size of 2k, then our chunks can only be placed > at 0,2k,4k,6k,8k... and so on (ie. every 2k starting from 0). > > This patch introduces the ability to use unaligned chunks. With these > changes, we are no longer bound to having to place chunks at a 2k (or > whatever your chunk size is) interval. Since we are no longer dealing with > aligned chunks, they can now cross page boundaries. Checks for page > contiguity have been added in order to keep track of which pages are > followed by a physically contiguous page. > > Signed-off-by: Kevin Laatz > Signed-off-by: Ciara Loftus > Signed-off-by: Bruce Richardson > > --- > v2: > - Add checks for the flags coming from userspace > - Fix how we get chunk_size in xsk_diag.c > - Add defines for masking the new descriptor format > - Modified the rx functions to use new descriptor format > - Modified the tx functions to use new descriptor format > > v3: > - Add helper function to do address/offset masking/addition > > v4: > - fixed page_start calculation in __xsk_rcv_memcpy(). > - move offset handling to the xdp_umem_get_* functions > - modified the len field in xdp_umem_reg struct. We now use 16 bits from > this for the flags field. > - removed next_pg_contig field from xdp_umem_page struct. Using low 12 > bits of addr to store flags instead. > - other minor changes based on review comments > > v5: > - Added accessors for getting addr and offset > - Added helper function to add offset to addr > - Fixed offset handling in xsk_rcv > - Removed bitfields from xdp_umem_reg > - Added struct size checking for xdp_umem_reg in xsk_setsockopt to handle > different versions of the struct. > - fix conflicts after 'bpf-af-xdp-wakeup' was merged. > --- > include/net/xdp_sock.h | 75 +++++++++++++++++++++++++++-- > include/uapi/linux/if_xdp.h | 9 ++++ > net/xdp/xdp_umem.c | 19 ++++++-- > net/xdp/xsk.c | 96 +++++++++++++++++++++++++++++-------- > net/xdp/xsk_diag.c | 2 +- > net/xdp/xsk_queue.h | 68 ++++++++++++++++++++++---- > 6 files changed, 232 insertions(+), 37 deletions(-) > > diff --git a/include/net/xdp_sock.h b/include/net/xdp_sock.h > index f023b9940d64..c9398ce7960f 100644 > --- a/include/net/xdp_sock.h > +++ b/include/net/xdp_sock.h > @@ -16,6 +16,13 @@ > struct net_device; > struct xsk_queue; > > +/* Masks for xdp_umem_page flags. > + * The low 12-bits of the addr will be 0 since this is the page address, so we > + * can use them for flags. > + */ > +#define XSK_NEXT_PG_CONTIG_SHIFT 0 > +#define XSK_NEXT_PG_CONTIG_MASK (1ULL << XSK_NEXT_PG_CONTIG_SHIFT) > + > struct xdp_umem_page { > void *addr; > dma_addr_t dma; > @@ -27,8 +34,12 @@ struct xdp_umem_fq_reuse { > u64 handles[]; > }; > > -/* Flags for the umem flags field. */ > -#define XDP_UMEM_USES_NEED_WAKEUP (1 << 0) > +/* Flags for the umem flags field. > + * > + * The NEED_WAKEUP flag is 1 due to the reuse of the flags field for public > + * flags. See inlude/uapi/include/linux/if_xdp.h. > + */ > +#define XDP_UMEM_USES_NEED_WAKEUP (1 << 1) > > struct xdp_umem { > struct xsk_queue *fq; > @@ -124,14 +135,36 @@ void xsk_map_try_sock_delete(struct xsk_map *map, struct xdp_sock *xs, > int xsk_map_inc(struct xsk_map *map); > void xsk_map_put(struct xsk_map *map); > > +static inline u64 xsk_umem_extract_addr(u64 addr) > +{ > + return addr & XSK_UNALIGNED_BUF_ADDR_MASK; > +} > + > +static inline u64 xsk_umem_extract_offset(u64 addr) > +{ > + return addr >> XSK_UNALIGNED_BUF_OFFSET_SHIFT; > +} > + > +static inline u64 xsk_umem_add_offset_to_addr(u64 addr) > +{ > + return xsk_umem_extract_addr(addr) + xsk_umem_extract_offset(addr); > +} > + > static inline char *xdp_umem_get_data(struct xdp_umem *umem, u64 addr) > { > - return umem->pages[addr >> PAGE_SHIFT].addr + (addr & (PAGE_SIZE - 1)); > + unsigned long page_addr; > + > + addr = xsk_umem_add_offset_to_addr(addr); > + page_addr = (unsigned long)umem->pages[addr >> PAGE_SHIFT].addr; > + > + return (char *)(page_addr & PAGE_MASK) + (addr & ~PAGE_MASK); > } > > static inline dma_addr_t xdp_umem_get_dma(struct xdp_umem *umem, u64 addr) > { > - return umem->pages[addr >> PAGE_SHIFT].dma + (addr & (PAGE_SIZE - 1)); > + addr = xsk_umem_add_offset_to_addr(addr); > + > + return umem->pages[addr >> PAGE_SHIFT].dma + (addr & ~PAGE_MASK); > } > > /* Reuse-queue aware version of FILL queue helpers */ > @@ -172,6 +205,19 @@ static inline void xsk_umem_fq_reuse(struct xdp_umem *umem, u64 addr) > > rq->handles[rq->length++] = addr; > } > + > +/* Handle the offset appropriately depending on aligned or unaligned mode. > + * For unaligned mode, we store the offset in the upper 16-bits of the address. > + * For aligned mode, we simply add the offset to the address. > + */ > +static inline u64 xsk_umem_adjust_offset(struct xdp_umem *umem, u64 address, > + u64 offset) > +{ > + if (umem->flags & XDP_UMEM_UNALIGNED_CHUNK_FLAG) > + return address + (offset << XSK_UNALIGNED_BUF_OFFSET_SHIFT); > + else > + return address + offset; > +} > #else > static inline int xsk_generic_rcv(struct xdp_sock *xs, struct xdp_buff *xdp) > { > @@ -241,6 +287,21 @@ static inline struct xdp_umem *xdp_get_umem_from_qid(struct net_device *dev, > return NULL; > } > > +static inline u64 xsk_umem_extract_addr(u64 addr) > +{ > + return 0; > +} > + > +static inline u64 xsk_umem_extract_offset(u64 addr) > +{ > + return 0; > +} > + > +static inline u64 xsk_umem_add_offset_to_addr(u64 addr) > +{ > + return 0; > +} > + > static inline char *xdp_umem_get_data(struct xdp_umem *umem, u64 addr) > { > return NULL; > @@ -290,6 +351,12 @@ static inline bool xsk_umem_uses_need_wakeup(struct xdp_umem *umem) > return false; > } > > +static inline u64 xsk_umem_adjust_offset(struct xdp_umem *umem, u64 handle, > + u64 offset) > +{ > + return 0; > +} > + > #endif /* CONFIG_XDP_SOCKETS */ > > #endif /* _LINUX_XDP_SOCK_H */ > diff --git a/include/uapi/linux/if_xdp.h b/include/uapi/linux/if_xdp.h > index 62b80d57b72a..be328c59389d 100644 > --- a/include/uapi/linux/if_xdp.h > +++ b/include/uapi/linux/if_xdp.h > @@ -26,6 +26,9 @@ > */ > #define XDP_USE_NEED_WAKEUP (1 << 3) > > +/* Flags for xsk_umem_config flags */ > +#define XDP_UMEM_UNALIGNED_CHUNK_FLAG (1 << 0) > + > struct sockaddr_xdp { > __u16 sxdp_family; > __u16 sxdp_flags; > @@ -66,6 +69,7 @@ struct xdp_umem_reg { > __u64 len; /* Length of packet data area */ > __u32 chunk_size; > __u32 headroom; > + __u32 flags; > }; > > struct xdp_statistics { > @@ -87,6 +91,11 @@ struct xdp_options { > #define XDP_UMEM_PGOFF_FILL_RING 0x100000000ULL > #define XDP_UMEM_PGOFF_COMPLETION_RING 0x180000000ULL > > +/* Masks for unaligned chunks mode */ > +#define XSK_UNALIGNED_BUF_OFFSET_SHIFT 48 > +#define XSK_UNALIGNED_BUF_ADDR_MASK \ > + ((1ULL << XSK_UNALIGNED_BUF_OFFSET_SHIFT) - 1) > + > /* Rx/Tx descriptor */ > struct xdp_desc { > __u64 addr; > diff --git a/net/xdp/xdp_umem.c b/net/xdp/xdp_umem.c > index 2d65779282a1..e997b263a0dd 100644 > --- a/net/xdp/xdp_umem.c > +++ b/net/xdp/xdp_umem.c > @@ -340,6 +340,7 @@ static int xdp_umem_account_pages(struct xdp_umem *umem) > > static int xdp_umem_reg(struct xdp_umem *umem, struct xdp_umem_reg *mr) > { > + bool unaligned_chunks = mr->flags & XDP_UMEM_UNALIGNED_CHUNK_FLAG; > u32 chunk_size = mr->chunk_size, headroom = mr->headroom; > unsigned int chunks, chunks_per_page; > u64 addr = mr->addr, size = mr->len; > @@ -355,7 +356,11 @@ static int xdp_umem_reg(struct xdp_umem *umem, struct xdp_umem_reg *mr) > return -EINVAL; > } > > - if (!is_power_of_2(chunk_size)) > + if (mr->flags & ~(XDP_UMEM_UNALIGNED_CHUNK_FLAG | > + XDP_UMEM_USES_NEED_WAKEUP)) > + return -EINVAL; > + > + if (!unaligned_chunks && !is_power_of_2(chunk_size)) > return -EINVAL; > > if (!PAGE_ALIGNED(addr)) { > @@ -372,9 +377,11 @@ static int xdp_umem_reg(struct xdp_umem *umem, struct xdp_umem_reg *mr) > if (chunks == 0) > return -EINVAL; > > - chunks_per_page = PAGE_SIZE / chunk_size; > - if (chunks < chunks_per_page || chunks % chunks_per_page) > - return -EINVAL; > + if (!unaligned_chunks) { > + chunks_per_page = PAGE_SIZE / chunk_size; > + if (chunks < chunks_per_page || chunks % chunks_per_page) > + return -EINVAL; > + } > > headroom = ALIGN(headroom, 64); > > @@ -383,13 +390,15 @@ static int xdp_umem_reg(struct xdp_umem *umem, struct xdp_umem_reg *mr) > return -EINVAL; > > umem->address = (unsigned long)addr; > - umem->chunk_mask = ~((u64)chunk_size - 1); > + umem->chunk_mask = unaligned_chunks ? XSK_UNALIGNED_BUF_ADDR_MASK > + : ~((u64)chunk_size - 1); > umem->size = size; > umem->headroom = headroom; > umem->chunk_size_nohr = chunk_size - headroom; > umem->npgs = size / PAGE_SIZE; > umem->pgs = NULL; > umem->user = NULL; > + umem->flags = mr->flags; > INIT_LIST_HEAD(&umem->xsk_list); > spin_lock_init(&umem->xsk_list_lock); > > diff --git a/net/xdp/xsk.c b/net/xdp/xsk.c > index ee4428a892fa..907e5f12338f 100644 > --- a/net/xdp/xsk.c > +++ b/net/xdp/xsk.c > @@ -45,7 +45,7 @@ EXPORT_SYMBOL(xsk_umem_has_addrs); > > u64 *xsk_umem_peek_addr(struct xdp_umem *umem, u64 *addr) > { > - return xskq_peek_addr(umem->fq, addr); > + return xskq_peek_addr(umem->fq, addr, umem); > } > EXPORT_SYMBOL(xsk_umem_peek_addr); > > @@ -115,21 +115,43 @@ bool xsk_umem_uses_need_wakeup(struct xdp_umem *umem) > } > EXPORT_SYMBOL(xsk_umem_uses_need_wakeup); > > +/* If a buffer crosses a page boundary, we need to do 2 memcpy's, one for > + * each page. This is only required in copy mode. > + */ > +static void __xsk_rcv_memcpy(struct xdp_umem *umem, u64 addr, void *from_buf, > + u32 len, u32 metalen) > +{ > + void *to_buf = xdp_umem_get_data(umem, addr); > + > + addr = xsk_umem_add_offset_to_addr(addr); > + if (xskq_crosses_non_contig_pg(umem, addr, len + metalen)) { > + void *next_pg_addr = umem->pages[(addr >> PAGE_SHIFT) + 1].addr; > + u64 page_start = addr & ~(PAGE_SIZE - 1); > + u64 first_len = PAGE_SIZE - (addr - page_start); > + > + memcpy(to_buf, from_buf, first_len + metalen); > + memcpy(next_pg_addr, from_buf + first_len, len - first_len); > + > + return; > + } > + > + memcpy(to_buf, from_buf, len + metalen); > +} > + > static int __xsk_rcv(struct xdp_sock *xs, struct xdp_buff *xdp, u32 len) > { > - void *to_buf, *from_buf; > + u64 offset = xs->umem->headroom; > + u64 addr, memcpy_addr; > + void *from_buf; > u32 metalen; > - u64 addr; > int err; > > - if (!xskq_peek_addr(xs->umem->fq, &addr) || > + if (!xskq_peek_addr(xs->umem->fq, &addr, xs->umem) || > len > xs->umem->chunk_size_nohr - XDP_PACKET_HEADROOM) { > xs->rx_dropped++; > return -ENOSPC; > } > > - addr += xs->umem->headroom; > - > if (unlikely(xdp_data_meta_unsupported(xdp))) { > from_buf = xdp->data; > metalen = 0; > @@ -138,9 +160,11 @@ static int __xsk_rcv(struct xdp_sock *xs, struct xdp_buff *xdp, u32 len) > metalen = xdp->data - xdp->data_meta; > } > > - to_buf = xdp_umem_get_data(xs->umem, addr); > - memcpy(to_buf, from_buf, len + metalen); > - addr += metalen; > + memcpy_addr = xsk_umem_adjust_offset(xs->umem, addr, offset); > + __xsk_rcv_memcpy(xs->umem, memcpy_addr, from_buf, len, metalen); > + > + offset += metalen; > + addr = xsk_umem_adjust_offset(xs->umem, addr, offset); > err = xskq_produce_batch_desc(xs->rx, addr, len); > if (!err) { > xskq_discard_addr(xs->umem->fq); > @@ -185,6 +209,7 @@ int xsk_generic_rcv(struct xdp_sock *xs, struct xdp_buff *xdp) > { > u32 metalen = xdp->data - xdp->data_meta; > u32 len = xdp->data_end - xdp->data; > + u64 offset = xs->umem->headroom; > void *buffer; > u64 addr; > int err; > @@ -196,17 +221,17 @@ int xsk_generic_rcv(struct xdp_sock *xs, struct xdp_buff *xdp) > goto out_unlock; > } > > - if (!xskq_peek_addr(xs->umem->fq, &addr) || > + if (!xskq_peek_addr(xs->umem->fq, &addr, xs->umem) || > len > xs->umem->chunk_size_nohr - XDP_PACKET_HEADROOM) { > err = -ENOSPC; > goto out_drop; > } > > - addr += xs->umem->headroom; > - > - buffer = xdp_umem_get_data(xs->umem, addr); > + buffer = xdp_umem_get_data(xs->umem, addr + offset); > memcpy(buffer, xdp->data_meta, len + metalen); > - addr += metalen; > + offset += metalen; > + > + addr = xsk_umem_adjust_offset(xs->umem, addr, offset); > err = xskq_produce_batch_desc(xs->rx, addr, len); > if (err) > goto out_drop; > @@ -250,7 +275,7 @@ bool xsk_umem_consume_tx(struct xdp_umem *umem, struct xdp_desc *desc) > > rcu_read_lock(); > list_for_each_entry_rcu(xs, &umem->xsk_list, list) { > - if (!xskq_peek_desc(xs->tx, desc)) > + if (!xskq_peek_desc(xs->tx, desc, umem)) > continue; > > if (xskq_produce_addr_lazy(umem->cq, desc->addr)) > @@ -304,7 +329,7 @@ static int xsk_generic_xmit(struct sock *sk, struct msghdr *m, > if (xs->queue_id >= xs->dev->real_num_tx_queues) > goto out; > > - while (xskq_peek_desc(xs->tx, &desc)) { > + while (xskq_peek_desc(xs->tx, &desc, xs->umem)) { > char *buffer; > u64 addr; > u32 len; > @@ -333,7 +358,7 @@ static int xsk_generic_xmit(struct sock *sk, struct msghdr *m, > skb->dev = xs->dev; > skb->priority = sk->sk_priority; > skb->mark = sk->sk_mark; > - skb_shinfo(skb)->destructor_arg = (void *)(long)addr; > + skb_shinfo(skb)->destructor_arg = (void *)(long)desc.addr; > skb->destructor = xsk_destruct_skb; > > err = dev_direct_xmit(skb, xs->queue_id); > @@ -526,6 +551,24 @@ static struct socket *xsk_lookup_xsk_from_fd(int fd) > return sock; > } > > +/* Check if umem pages are contiguous. > + * If zero-copy mode, use the DMA address to do the page contiguity check > + * For all other modes we use addr (kernel virtual address) > + * Store the result in the low bits of addr. > + */ > +static void xsk_check_page_contiguity(struct xdp_umem *umem, u32 flags) > +{ > + struct xdp_umem_page *pgs = umem->pages; > + int i, is_contig; > + > + for (i = 0; i < umem->npgs - 1; i++) { > + is_contig = (flags & XDP_ZEROCOPY) ? > + (pgs[i].dma + PAGE_SIZE == pgs[i + 1].dma) : > + (pgs[i].addr + PAGE_SIZE == pgs[i + 1].addr); > + pgs[i].addr += is_contig << XSK_NEXT_PG_CONTIG_SHIFT; > + } > +} > + > static int xsk_bind(struct socket *sock, struct sockaddr *addr, int addr_len) > { > struct sockaddr_xdp *sxdp = (struct sockaddr_xdp *)addr; > @@ -616,6 +659,8 @@ static int xsk_bind(struct socket *sock, struct sockaddr *addr, int addr_len) > err = xdp_umem_assign_dev(xs->umem, dev, qid, flags); > if (err) > goto out_unlock; > + > + xsk_check_page_contiguity(xs->umem, flags); > } > > xs->dev = dev; > @@ -636,6 +681,13 @@ static int xsk_bind(struct socket *sock, struct sockaddr *addr, int addr_len) > return err; > } > > +struct xdp_umem_reg_v1 { > + __u64 addr; /* Start of packet data area */ > + __u64 len; /* Length of packet data area */ > + __u32 chunk_size; > + __u32 headroom; > +}; > + > static int xsk_setsockopt(struct socket *sock, int level, int optname, > char __user *optval, unsigned int optlen) > { > @@ -673,10 +725,16 @@ static int xsk_setsockopt(struct socket *sock, int level, int optname, > } > case XDP_UMEM_REG: > { > - struct xdp_umem_reg mr; > + size_t mr_size = sizeof(struct xdp_umem_reg); > + struct xdp_umem_reg mr = {}; > struct xdp_umem *umem; > > - if (copy_from_user(&mr, optval, sizeof(mr))) > + if (optlen < sizeof(struct xdp_umem_reg_v1)) > + return -EINVAL; > + else if (optlen < sizeof(mr)) > + mr_size = sizeof(struct xdp_umem_reg_v1); > + > + if (copy_from_user(&mr, optval, mr_size)) > return -EFAULT; > > mutex_lock(&xs->mutex); > diff --git a/net/xdp/xsk_diag.c b/net/xdp/xsk_diag.c > index d5e06c8e0cbf..9986a759fe06 100644 > --- a/net/xdp/xsk_diag.c > +++ b/net/xdp/xsk_diag.c > @@ -56,7 +56,7 @@ static int xsk_diag_put_umem(const struct xdp_sock *xs, struct sk_buff *nlskb) > du.id = umem->id; > du.size = umem->size; > du.num_pages = umem->npgs; > - du.chunk_size = (__u32)(~umem->chunk_mask + 1); > + du.chunk_size = umem->chunk_size_nohr + umem->headroom; > du.headroom = umem->headroom; > du.ifindex = umem->dev ? umem->dev->ifindex : 0; > du.queue_id = umem->queue_id; > diff --git a/net/xdp/xsk_queue.h b/net/xdp/xsk_queue.h > index dd9e985c2461..6c67c9d0294f 100644 > --- a/net/xdp/xsk_queue.h > +++ b/net/xdp/xsk_queue.h > @@ -134,6 +134,17 @@ static inline bool xskq_has_addrs(struct xsk_queue *q, u32 cnt) > > /* UMEM queue */ > > +static inline bool xskq_crosses_non_contig_pg(struct xdp_umem *umem, u64 addr, > + u64 length) > +{ > + bool cross_pg = (addr & (PAGE_SIZE - 1)) + length > PAGE_SIZE; > + bool next_pg_contig = > + (unsigned long)umem->pages[(addr >> PAGE_SHIFT)].addr & > + XSK_NEXT_PG_CONTIG_MASK; > + > + return cross_pg && !next_pg_contig; > +} > + > static inline bool xskq_is_valid_addr(struct xsk_queue *q, u64 addr) > { > if (addr >= q->size) { > @@ -144,23 +155,49 @@ static inline bool xskq_is_valid_addr(struct xsk_queue *q, u64 addr) > return true; > } > > -static inline u64 *xskq_validate_addr(struct xsk_queue *q, u64 *addr) > +static inline bool xskq_is_valid_addr_unaligned(struct xsk_queue *q, u64 addr, > + u64 length, > + struct xdp_umem *umem) > +{ > + addr = xsk_umem_add_offset_to_addr(addr); > + if (addr >= q->size || I know I already asked about it, but I feel we need to clarify things here further. The `addr >= q->size` check is good, it checks that the address doesn't overflow the UMEM. However, the new encoding of UMEM handles consists of two parts: base address of the frame (low bits) and offset (high bits), that are added together by xsk_umem_add_offset_to_addr. It's possible to craft an address that has too big base address of the frame (all 0xff...f), some non-zero offset, so after adding them together we pass `addr < q->size`, but the base address exceeds the UMEM. In my opinion, we should not allow such addresses, because the base address of the frame is out of bounds, so we need one more check here. What do you think about this point? > + xskq_crosses_non_contig_pg(umem, addr, length)) { > + q->invalid_descs++; > + return false; > + } > + > + return true; > +} > + > +static inline u64 *xskq_validate_addr(struct xsk_queue *q, u64 *addr, > + struct xdp_umem *umem) > { > while (q->cons_tail != q->cons_head) { > struct xdp_umem_ring *ring = (struct xdp_umem_ring *)q->ring; > unsigned int idx = q->cons_tail & q->ring_mask; > > *addr = READ_ONCE(ring->desc[idx]) & q->chunk_mask; > + > + if (umem->flags & XDP_UMEM_UNALIGNED_CHUNK_FLAG) { > + if (xskq_is_valid_addr_unaligned(q, *addr, > + umem->chunk_size_nohr, > + umem)) > + return addr; > + goto out; > + } > + > if (xskq_is_valid_addr(q, *addr)) > return addr; > > +out: > q->cons_tail++; > } > > return NULL; > } > > -static inline u64 *xskq_peek_addr(struct xsk_queue *q, u64 *addr) > +static inline u64 *xskq_peek_addr(struct xsk_queue *q, u64 *addr, > + struct xdp_umem *umem) > { > if (q->cons_tail == q->cons_head) { > smp_mb(); /* D, matches A */ > @@ -171,7 +208,7 @@ static inline u64 *xskq_peek_addr(struct xsk_queue *q, u64 *addr) > smp_rmb(); > } > > - return xskq_validate_addr(q, addr); > + return xskq_validate_addr(q, addr, umem); > } > > static inline void xskq_discard_addr(struct xsk_queue *q) > @@ -230,8 +267,21 @@ static inline int xskq_reserve_addr(struct xsk_queue *q) > > /* Rx/Tx queue */ > > -static inline bool xskq_is_valid_desc(struct xsk_queue *q, struct xdp_desc *d) > +static inline bool xskq_is_valid_desc(struct xsk_queue *q, struct xdp_desc *d, > + struct xdp_umem *umem) > { > + if (umem->flags & XDP_UMEM_UNALIGNED_CHUNK_FLAG) { > + if (!xskq_is_valid_addr_unaligned(q, d->addr, d->len, umem)) > + return false; > + > + if (d->len > umem->chunk_size_nohr || d->options) { > + q->invalid_descs++; > + return false; > + } > + > + return true; > + } > + > if (!xskq_is_valid_addr(q, d->addr)) > return false; > > @@ -245,14 +295,15 @@ static inline bool xskq_is_valid_desc(struct xsk_queue *q, struct xdp_desc *d) > } > > static inline struct xdp_desc *xskq_validate_desc(struct xsk_queue *q, > - struct xdp_desc *desc) > + struct xdp_desc *desc, > + struct xdp_umem *umem) > { > while (q->cons_tail != q->cons_head) { > struct xdp_rxtx_ring *ring = (struct xdp_rxtx_ring *)q->ring; > unsigned int idx = q->cons_tail & q->ring_mask; > > *desc = READ_ONCE(ring->desc[idx]); > - if (xskq_is_valid_desc(q, desc)) > + if (xskq_is_valid_desc(q, desc, umem)) > return desc; > > q->cons_tail++; > @@ -262,7 +313,8 @@ static inline struct xdp_desc *xskq_validate_desc(struct xsk_queue *q, > } > > static inline struct xdp_desc *xskq_peek_desc(struct xsk_queue *q, > - struct xdp_desc *desc) > + struct xdp_desc *desc, > + struct xdp_umem *umem) > { > if (q->cons_tail == q->cons_head) { > smp_mb(); /* D, matches A */ > @@ -273,7 +325,7 @@ static inline struct xdp_desc *xskq_peek_desc(struct xsk_queue *q, > smp_rmb(); /* C, matches B */ > } > > - return xskq_validate_desc(q, desc); > + return xskq_validate_desc(q, desc, umem); > } > > static inline void xskq_discard_desc(struct xsk_queue *q) >