From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2220BC433EF for ; Wed, 23 Feb 2022 00:51:25 +0000 (UTC) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id DB02440DF6; Wed, 23 Feb 2022 01:51:03 +0100 (CET) Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by mails.dpdk.org (Postfix) with ESMTP id 9745E4067B for ; Wed, 23 Feb 2022 01:51:02 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1645577462; x=1677113462; h=message-id:date:subject:from:to:cc:references: in-reply-to:content-transfer-encoding:mime-version; bh=IpiC5fKujNSVaQkJHD0yiNLojjqIBLjqbbmvfBb5ErA=; b=HKWmVGGWuIxJHelfNb32bxEuCMaWCodMPqJTF9uqztPAZWW9+gJD9UfL SDoGo3p9+3fm5jkEMbreYzNkR8DAgthriY74S9/wXQrYJof1r4q9zfbfh 5EaSGOqltyzCEFIGXltlVCk3txhlVfpkk2+7WSTiemsZOig7EsqVd8iLG EAwMQy0GOd6cWTylfTslLxWeu6bTghMKGQFALSoPtHa7VuvIbk9B0VJyB 9BWdNJky7E0PHcAk37erVRC0dHskA8s2ngn1zYCEfdtScT5yhjfIGwKEG um8D2E720ntllCHwF98bEC7/kVGhE5GXpjhMeQwPlIRutXN2IC50tWLHu A==; X-IronPort-AV: E=McAfee;i="6200,9189,10266"; a="231825768" X-IronPort-AV: E=Sophos;i="5.88,389,1635231600"; d="scan'208";a="231825768" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Feb 2022 16:51:01 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.88,389,1635231600"; d="scan'208";a="548004971" Received: from fmsmsx604.amr.corp.intel.com ([10.18.126.84]) by orsmga008.jf.intel.com with ESMTP; 22 Feb 2022 16:51:00 -0800 Received: from fmsmsx611.amr.corp.intel.com (10.18.126.91) by fmsmsx604.amr.corp.intel.com (10.18.126.84) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.20; Tue, 22 Feb 2022 16:51:00 -0800 Received: from fmsmsx606.amr.corp.intel.com (10.18.126.86) by fmsmsx611.amr.corp.intel.com (10.18.126.91) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.20; Tue, 22 Feb 2022 16:51:00 -0800 Received: from fmsedg601.ED.cps.intel.com (10.1.192.135) by fmsmsx606.amr.corp.intel.com (10.18.126.86) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.20 via Frontend Transport; Tue, 22 Feb 2022 16:51:00 -0800 Received: from NAM04-BN8-obe.outbound.protection.outlook.com (104.47.74.44) by edgegateway.intel.com (192.55.55.70) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2308.20; Tue, 22 Feb 2022 16:50:59 -0800 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=TqmJpfCv+ggQ6BprzujUU/NS/aNTzCs+sP2e3QjlWg5yKeOrR7GGnVW2A5Kll38NLlSIxRnntuTrTFFCIDwpyLbFND26exxvKcpVc0N4X1V9MDGlZbuRht1PZgilE6/p993VkK2hzlg8g0ujoQHMyFCTL9GPPUS15ki0/Jo4Qj3YW1NzayfrRmfQUdbxJXtDvI/KdnIshAZgfMpHZYE8KAFk5Ll6WoeJLK7vNbMLhRY7haLUpJnJgJCPio9mfDujmjOy1j9F4GzFHOTEaPLjNpCdvuIip1fABrn6ztPpwlrz5S6NaffTtST6Rt9VMUgY7bhF7MsZNSkQyBYzvwvgKA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=fIdTW5GlhFI2YugB7gntGB9G+1doGg/X5n9GtyF7FLI=; b=RUtD1H1V2voP20/mE4cb5AN9Q9BZZd2GvWsC5SggXo/6qb7KIEDXuLDOHBqNCT5Vu07P+0CgJchksTbG568EcdBK4YeDWQh6bqVU4KrOvBS4eRjR3pux/NmAv7GtLLvUjL+bNBDKH1a18q3I7vO6HL1/BJ9qQuOtq//B3iERVaFEpAGZWBXbs9Zflg+V8KLvPqyIT3BiZ2hhAnaEvzMsA6cRx4cmkh/mRB6sppAoPFQjMcKt9vL2lSps9j18vfNh7pOlAClxswS9dIB4Sulk/XvPQYXR6Nu7ptuCgO/dsqKPxZPIxKoEHbhJ6miuUgniUUBnVFqPZ/qG5MBBeRWfKA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; Received: from PH0PR11MB5000.namprd11.prod.outlook.com (2603:10b6:510:41::19) by MN2PR11MB3709.namprd11.prod.outlook.com (2603:10b6:208:f3::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4995.26; Wed, 23 Feb 2022 00:50:52 +0000 Received: from PH0PR11MB5000.namprd11.prod.outlook.com ([fe80::98be:5506:5020:28a2]) by PH0PR11MB5000.namprd11.prod.outlook.com ([fe80::98be:5506:5020:28a2%4]) with mapi id 15.20.4995.027; Wed, 23 Feb 2022 00:50:52 +0000 Message-ID: <3a3fccd1-af2a-dee4-475c-879385f4b2e5@intel.com> Date: Wed, 23 Feb 2022 00:50:46 +0000 Subject: Re: [PATCH v2 10/21] net/ena: proxy AQ calls to primary process Content-Language: en-US From: Ferruh Yigit To: Michal Krawczyk , CC: , , Stanislaw Kardach , Dawid Gorecki References: <20220222160634.24489-1-mk@semihalf.com> <20220222181146.28882-1-mk@semihalf.com> <20220222181146.28882-11-mk@semihalf.com> X-User: ferruhy In-Reply-To: Content-Type: text/plain; charset="UTF-8"; format=flowed Content-Transfer-Encoding: 8bit X-ClientProxiedBy: LO4P123CA0171.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:18a::14) To PH0PR11MB5000.namprd11.prod.outlook.com (2603:10b6:510:41::19) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 29e7adb8-5016-4100-89a3-08d9f6668ab2 X-MS-TrafficTypeDiagnostic: MN2PR11MB3709:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: DmsrjusbFHzLm36OQl2Phaf/xIsA9ffxqCnNnal9sENs1zeOT1tzfWEfzz9IYCujNJtHQwyaKFvZEVwhVcFQei2Mp36BTHo4R0Qjh3axtEY4aHij6VJevsu30gENKcwoJ37Re9AAZXqayEplR2WKY7vNV9FKDWnacMra+hbyNYuKIHavuGea3oaNMLYfaTnJBTew/OiIvVE+sRsJmpSfYkdQtD6kJcpusRfdOaO4MJ6ESgFz25vNvaB3aML3F+Hc44FxrrryNMRd3raBPyj6/5BkqFl+EuL1g1aWlCcDfiLPCPVneFN8Gag6bMrbUm4V9L2NyD4smtlXsokNQi6zi4gXbawKaG6ojuaqZE6iZbN5s1U8ZeGOCSa8Tj93f150Ii4qCJFIGtjxuyc9Mdi3S9xVGBsMbFbVkQ1zfIduHj3sX0OWZEw6j0N81LTQQAfGqsppEYkSL7uhUQhkZS58wTteaxjTb5cjbVShfEoz6qSpxJRyAZd5Hq0b57FdSQTNpAJ9ihsww6qCS9kgzt6Eb7vbo4aBbKYbk0xcYYvqAKVEi8te3cKAkzlzAwByx8GKPoLB/8wll0L/7kUzBQ83IXG0jU/QDcDxCl36d3N/O5044enq9GNMw1ctpaDwmRGOEiEYDNqXI3eqY0W2fGOVDCJ3DGeB+HvOSdez5iXg++fPCxQNAAZ3fF89WGLyoPgrEv5gBPOUaux6RQ60mZqhPBbajs2XwapFDoMViwJVdfCXuGNsY6SR6RW/JSryJlCSSRU8XRVI8RSbuM7qiQ/6H1Jhl5dKLwPk/XJZ6xvm91w= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:PH0PR11MB5000.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230001)(366004)(54906003)(36756003)(83380400001)(53546011)(6666004)(2906002)(6506007)(31686004)(8936002)(44832011)(5660300002)(66946007)(8676002)(186003)(6486002)(966005)(2616005)(4326008)(66556008)(66476007)(38100700002)(6512007)(31696002)(82960400001)(86362001)(26005)(316002)(508600001)(45980500001); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?d3BxZ2xmSEtwdnVidHpmRERjY1p4MDZhTjA1azFlc3JneEJJNm9QUlh2cEpp?= =?utf-8?B?eDlheFJiMlFRMWQyc0laTng2OUpnSXhiR1QzWXlPQ1JmbmNDbGJLMFBIUTdG?= =?utf-8?B?QzBWUUpYWnhoVWpEMXhZRnEvQVU1UE5qeWowRFJPL2M4VkFpYW92bFNaNlhS?= =?utf-8?B?SThteFRveVd0RkdBTUtWeHRocXlkbjJ0dkVuMFQ2OTc5T3MrcUdqc2pjSWVL?= =?utf-8?B?VTBRQThWZzZaelMzYktMMWM2dkFpaXE4c0xTUUdmQnBnalhZVlZ4ZUgrcnQw?= =?utf-8?B?V2JkQVh0bmxmZk53eFRud0kzUWhQV1g2eGZ0YTJFSVlETlN1anBqbkgzU0tm?= =?utf-8?B?KzY3alZGNElzMGxJWWpkWEZIV2ZsYmI5bmp2Qi92Vi9kc1hqRit3VDF3WTc2?= =?utf-8?B?RCt6enB2Uk1uN0hGRjIwY01UcVV1NDJZU0hVVVByNUVPams5Y2JseUczaFg0?= =?utf-8?B?bFhKeE1HZjNDNjIyRTI1bjJ6QnloL0hZcHdaZlRnS2RscENsU3hsc1RTT3dU?= =?utf-8?B?S2laZFZVbS92SE5ia3J6VEtobEZqcDFJNTJNcDRhSkcxZVdKNmRxQXc4QVk2?= =?utf-8?B?cHN5RURveXZpTE9TMERyWm8yRStlSG42Q0JHQjRFTml4NnoyM0FIMXhGcHN3?= =?utf-8?B?SXVvb3J3SytPL3lOeHJaY2I0NTdrVnl5QmYxS1g3RUYwK3RlK1pLbjhOZ3JR?= =?utf-8?B?UjFVT25aRUZITStXalBsSFZoYlg0YTZabGVGSE1QdlhQRE9Ub09vdWdlV2Ns?= =?utf-8?B?TGpWNXhzM1NJMVZEcmxrZFNtT2k5MUltKzNHcVAwZzZjOFNITzV3TktJeWVp?= =?utf-8?B?RFBZajZ2NGRDOUc1WjkzUS85Q3ducXhJWWRLdlJVdzR0VkF4aHV3WlRIMzht?= =?utf-8?B?VWdHbXp4YVU4Wm42OHd5aGxHR0RLZk5qZUFQWXhTUERPdm9td2VZa0FqUFVS?= =?utf-8?B?MVoraFVNT0pPU2xkMXVVZkZtd3c4T3VCL1BNMHBMR0FSeDJYVC8xNDhaRHZh?= =?utf-8?B?ajg2eTJBdy9GZlhtcitabFhjVmYyZkltNXhpYmwrVGloVkpWSVZqcGJXOU1a?= =?utf-8?B?bGtuUjlRdnRmRDhsaUZJQi9kSWJidDBwVTNUWnJQcUZYRSsyblNGb0VySi9J?= =?utf-8?B?N0E2ZnJtV25KYVNVcjE5NngyWXVZTFJhWkdKM1h1ZUhoVDkrSVhteUpuN3Ex?= =?utf-8?B?b2ppcnJUTWUyY1g1NWNvc0U3ejU3M3hwTXh4Z0ZqS29ic25CVzIyaDFOMEhs?= =?utf-8?B?Z0VkSDN0YTBGbkI1ZURzdlRUTER3bG84c2pka3JMa1MxQjJoVHkrZDYxRDB6?= =?utf-8?B?UWxJMzFEQWFQMUVmRFBSTS9FL2VmckxYeW1YQ3dKenhXekkvMHgxcVBnc0J0?= =?utf-8?B?K3VYYVhycUVRWmYyY2dRQWNVclNJT0xuVVUvZVorSFYyeTQ4akhLWEV0OS90?= =?utf-8?B?QXFVaEZkblpraWNDQjR3cU1GR01KOG9TSVdXUmJ0T3k0NW10NDNDVWdSYkpz?= =?utf-8?B?Q0RTYzliMXJwcDlXNXB2VGZ6T24wMis4YlRlMVpOMmpmSlR2LzFxdmJBK0ZS?= =?utf-8?B?YTJiLy9PWHZlNDQxSll2YklwRnB0VXZGNmg3YjJkVmlhTmdYaTl4Z1Y4MXJY?= =?utf-8?B?dENhdGJUUXhqMzIzaU1RU3liL1FDQmNMSVJNTDFqUGEwdzUyQ1FhNmRNSmln?= =?utf-8?B?eEE4bW1Zajl5eE51TGh6Z0JKdWJ3TVRnMWp5MXRqY3oyd2srUXdLNUhmckdQ?= =?utf-8?B?SG42cTJvV3d1TnM0VWNtMi9MY2ttd1ltVU1yODdnTExmZFhyWExGK1hTdFlp?= =?utf-8?B?R09aYy9PUDhnS0JRWDlDblN4QTRsNkNMU0VFdWZRaFhpL2MyclNWVjNTeWNp?= =?utf-8?B?YUdVc3V5blNWK2F1ejR2WmR3NURkK21EMU9YNXdPNmJKSG9GQVd0N2lKKzRL?= =?utf-8?B?L2ZBcVFUQXd1NUtDQmRId0JvYzFsUC9EZVBVSXdnU1B6VE9rOENZTE1SMm9q?= =?utf-8?B?bEw5ajgxSGhLN2c2bFlQamVVYkNMVnVJMm5ZNnUxMk5HZUlzdENtbFBBSDNs?= =?utf-8?B?ZFFlbHJCMWpVL3dxcWpxeFpvV3VFd252QWl4MkFJRWhpa0FBZ1pKS0tQYXhS?= =?utf-8?B?UW1WTVRNVVUyNzB6dXpTQjJUUldTbnlqc3JoL0VLaDQwZVFvT2pwSGVMWmdP?= =?utf-8?Q?1NxTtJtquH38dZ0CuGrCUpk=3D?= X-MS-Exchange-CrossTenant-Network-Message-Id: 29e7adb8-5016-4100-89a3-08d9f6668ab2 X-MS-Exchange-CrossTenant-AuthSource: PH0PR11MB5000.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Feb 2022 00:50:52.6813 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: pE5lWtH0WYjgA+LGijyc98qaopJ8ehZPqMX33PaLggQtUIsN/wpZAEVOb8kdxZ+FEKOo8bIz5XlN0u+X8pVgyw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR11MB3709 X-OriginatorOrg: intel.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On 2/22/2022 10:24 PM, Ferruh Yigit wrote: > On 2/22/2022 6:11 PM, Michal Krawczyk wrote: >> From: Stanislaw Kardach >> >> Due to how the ena_com compatibility layer is written, all AQ commands >> triggering functions use stack to save results of AQ and then copy them >> to user given function. >> Therefore to keep the compatibility layer common, introduce ENA_PROXY >> macro. It either calls the wrapped function directly (in primary >> process) or proxies it to the primary via DPDK IPC mechanism. Since all >> proxied calls are taken under a lock share the result data through >> shared memory (in struct ena_adapter) to work around 256B IPC parameter >> size limit. >> >> New proxy calls can be added by >> 1. Adding a new message type at the end of enum ena_mp_req >> 2. Adding new message arguments to the struct ena_mp_body if needed >> 3. Defining proxy request descriptor with ENA_PROXY_DESC. Its arguments >>     include handlers for request preparation and response processing. >>     Any of those may be empty (aside of marking arguments as used). >> 4. Adding request handling logic to ena_mp_primary_handle() >> 5. Replacing proxied function calls with ENA_PROXY(adapter, , ...) >> >> Signed-off-by: Stanislaw Kardach >> Reviewed-by: Michal Krawczyk >> Reviewed-by: Dawid Gorecki >> Reviewed-by: Shai Brandes > > <...> > >> + */ >> +#define ENA_PROXY(a, f, ...)                        \ >> +({                                    \ >> +    struct ena_adapter *_a = (a);                    \ >> +    struct timespec ts = { .tv_sec = ENA_MP_REQ_TMO };        \ >> +    struct ena_mp_body *req, *rsp;                    \ >> +    struct rte_mp_reply mp_rep;                    \ >> +    struct rte_mp_msg mp_req;                    \ >> +    int ret;                            \ >> +                                    \ >> +    if (rte_eal_process_type() == RTE_PROC_PRIMARY) {        \ >> +        ret = f(__VA_ARGS__);                    \ >> +    } else {                            \ >> +        /* Prepare and send request */                \ >> +        req = (struct ena_mp_body *)&mp_req.param;        \ >> +        mp_msg_init(&mp_req, mp_type_ ## f, _a->edev_data->port_id); \ >> +        mp_prep_ ## f(_a, req, ## __VA_ARGS__);            \ >> +                                    \ >> +        ret = rte_mp_request_sync(&mp_req, &mp_rep, &ts);    \ >> +        if (likely(!ret)) {                    \ >> +            RTE_ASSERT(mp_rsp.nb_received == 1);        \ > > 'mp_rsp' is not defined, I assume intention is to use 'mp_rep'. > > Build error can be reproduced by enabling RTE_ASSERT (by setting > 'RTE_ENABLE_ASSERT'). > >> +            rsp = (struct ena_mp_body *)&mp_rep.msgs[0].param; \ >> +            ret = rsp->result;                \ >> +            if (ret == 0) {                    \ >> +                mp_proc_##f(_a, rsp, ## __VA_ARGS__);    \ >> +            } else {                    \ >> +                PMD_DRV_LOG(ERR,            \ >> +                        "%s returned error: %d\n",    \ >> +                        mp_name_ ## f, rsp->result);\ >> +            }                        \ >> +            free(mp_rep.msgs);                \ >> +        } else if (rte_errno == ENOTSUP) {            \ >> +            PMD_DRV_LOG(ERR,                \ >> +                    "No IPC, can't proxy to primary\n");\ >> +            ret = -rte_errno;                \ >> +        } else {                        \ >> +            PMD_DRV_LOG(ERR, "Request %s failed: %s\n",    \ >> +                    mp_name_ ## f,            \ >> +                    rte_strerror(rte_errno));        \ >> +            ret = -ECOMM;                    \ Also 'ECOMM' seems only defined for Linux, so it breaks the FreeBSD build: http://mails.dpdk.org/archives/test-report/2022-February/263131.html >> +        }                            \ >> +    }                                \ >> +    ret;                                \ >> +}) >> > <...>