From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0906CC432BE for ; Wed, 1 Sep 2021 07:50:03 +0000 (UTC) Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by mail.kernel.org (Postfix) with ESMTP id 9468460F11 for ; Wed, 1 Sep 2021 07:50:02 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 9468460F11 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=dpdk.org Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id CD29D4013F; Wed, 1 Sep 2021 09:50:01 +0200 (CEST) Received: from mail-io1-f52.google.com (mail-io1-f52.google.com [209.85.166.52]) by mails.dpdk.org (Postfix) with ESMTP id A47D440041 for ; Wed, 1 Sep 2021 09:50:00 +0200 (CEST) Received: by mail-io1-f52.google.com with SMTP id a15so2881295iot.2 for ; Wed, 01 Sep 2021 00:50:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=HPtEC1ANF0A/oOJTZym/ZBrQIvCVxnVyJatwycwLrbk=; b=QgaGZXVmf31oj5NsxYay9cQKOjOgWbXcWZtqHAlXStATymjZmQLrTWZblREMYfKxA1 bC/+9CR66ol7vNI876sLQfr+nEcMTu+akvyQyKm+HwvJAhOHy9X12bEwQ7q/kmChZu3C Buv3ReEK3ZiTZg3dG5FKSoOrkQm20Rx8Qkak/6YGH1rPXRUaGbEz/9pZbjvMQTY9QbA7 Myy96a3QyYfUHXNAHtQOdmq9qQ4oPRw18POegZ0F1DmC/KmCDeEjtc6OuIthknC96GBp S6kxGjKPUz2TZ3feazfq25UajKINB4W0T4fGWPnX+e+NWeIIe0YoKWH/luiPjDnYpOfc GY5w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=HPtEC1ANF0A/oOJTZym/ZBrQIvCVxnVyJatwycwLrbk=; b=MgEcS3ll3MRexp2eZZkbYs2GoI99waKInbzESzqPfzaYhNtL23KaFOnoHZTsI8X+pO ZJKMDT4Yp+N2IrNBdxzDLX4bd40tAnE/SLKtJgq35tzej88qnxTG32rLePpYfaaxO5bJ ycFmzQ29ACUjiz+nBLpK41aizctK46ugtaNVRjtZI2NaDh5R3UYojeklk2C32n37YlnS LMarWhS7Y09xP1YiArlZxsyEPuM7Z4M4x4pRIY5Uh0IcVxdxgNEwMNu7XDH50GJ1WFih CPe0NQSSOiIDypACetPwy8NZXHTWwjVgnfAJcw38MA5TcnTAXK5dcf5/50U2SzhOx7ay 2tKw== X-Gm-Message-State: AOAM5320mwijiSIHJONAIBWVGQNC373AoK6D/2rjSlbCcILhQ/t7vPFj 8f8jrgEyQZQpUuqOORWnvLOk3Bl3dY5QwAzl/VQ= X-Google-Smtp-Source: ABdhPJzxFnFl95UeU7QIhBJ7K5xz8cWJ2AM+2dgEli1KPCjtySaWflmEbsoyHG52UTJDln7ymZZFOMg1J4jDL5nn49Y= X-Received: by 2002:a02:c055:: with SMTP id u21mr6654320jam.113.1630482599962; Wed, 01 Sep 2021 00:49:59 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Jerin Jacob Date: Wed, 1 Sep 2021 13:19:33 +0530 Message-ID: To: "Kaladi, Ashok K" , Harman Kalra , Nithin Dabilpuram , Ferruh Yigit , Anatoly Burakov , "Richardson, Bruce" , "Ananyev, Konstantin" , Thomas Monjalon , David Marchand Cc: "jerinj@marvell.com" , "Jayatheerthan, Jay" , "Carrillo, Erik G" , "Gujjar, Abhinandan S" , "dev@dpdk.org" , "Ayyadurai, Balasankar" Content-Type: text/plain; charset="UTF-8" Subject: Re: [dpdk-dev] [RFC] Control packet event adapter and FIFO library X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On Wed, Sep 1, 2021 at 11:55 AM Jerin Jacob wrote: > > On Wed, Sep 1, 2021 at 11:12 AM Kaladi, Ashok K > wrote: > > > > Dear dpdk-dev team, > > > > We would like to propose the following RFC for your review. > > > > A user space application may need access to the packets handled by eventdev > > based DPDK application. This application doesn't use mbuf or eventdev based > > DPDK APIs. Presently this is not possible without passing packets through > > DPDK KNI. > > > I think it is an innovative idea it is useful for multiple use cases > not just for eventdev. > > Some feedback on thoughts > > 1) The FIFO library should be generic it should not be specific to eventdev > 2) I think, This FIFO library should be generic and KNI also be a > consumer of this library > 3) I think, FIFO should not be a device instead it should be an > abstact object like rte_mempool * > 4) We need to consider User space app can be another DPDK primary > process or some non DPDK app > 4) I think, we can remove the Linux shared memory dependency instead > of introduce some > scheme of "exporting" memzone from DPDK application to another user > space app or another DPDK primary process. > I see the following reasons: > - It is backed by hugepage so better performance > - Is kernel do any memcpy when using Linux shm calls in kernel space? > > Thoughts? > > May be you can share set of API prototypes without any implementation > for the next level discussion if > others are OK this kind of library. Some thoughts for generic fifo library. // Create SPSC fifo of size struct rte_ipc_queue* rte_ipc_queue_create(const char *name, size_t size, int socket_id, unsigned int flags); void rte_ipc_queue_destroy(struct rte_ipc_queue* iq); // userspace app or another DPDK primary process to get the ipc_queue handle. struct rte_ipc_queue* rte_ipc_queue_lookup(const char *proc_name, const char *ipc_queue_name); // Get the memory of size to fill by producer size_t rte_ipc_queue_enq_peek(struct rte_ipc_queue* iq, void **obj, size_t size); // Commit the head pointer int rte_ipc_queue_enq_commit(struct rte_ipc_queue* iq, void *obj, size_t size); // Get the memory of size to to consumer by consumer size_t rte_ipc_queue_deq_peek(struct rte_ipc_queue* iq, void **obj, size_t size); // Commit the tail pointer int rte_ipc_queue_deq_commit(struct rte_ipc_queue* iq, void *obj, size_t size); int rte_ipc_queue_in_use(struct rte_ipc_queue* iq) int rte_ipc_queue_avilable(struct rte_ipc_queue* iq) except rte_ipc_queue_create() and rte_ipc_queue_destroy() all the remaning rte_ipc_* APIs can be used the other userspace appliction. Another alternative is to have ZeroMQ kind of messaging library for various transports. It may not as fast this, Needs a prototype to verify. > > > > > This RFC introduces control packet event adapter (CP adapter) and FIFO library > > which can be used by user space applications to access the packets > > handled by eventdev based DPDK application. Linux shared memory is used to > > keep mempool (mbufs), packets and FIFOs. This allows copy-free access of > > packets to both the applications. The Rx and Tx FIFOs are used to exchange > > packets between the applications. > > > > '------------------' > > | mempool | > > '-------------' '-----------' |..................| '----------' '-----------' > > | | | | | <--- Tx FIFOs | | | | | > > | User space | | | | <--- Alloc FIFO | | Control | | DPDK | > > | App <----> FIFO Lib <----> ---> Rx FIFOs <----> Packet <----> Eventdev | > > | | | | | ---> Free FIFO | | Adapter | | App | > > | | | | | | | | | | > > '-------------' '-----------' | | '----------' '-----------' > > '------------------' > > > > CP adapter does the mbuf allocation and free for user space application. Mbufs > > allocated for user space application are put into alloc queue by CP adapter. > > The used mbufs are put into free queue by user space application. > > The FIFO lib translates packets between rte_mbuf format and a simplified > > format which will be used by the user application. FIFO lib doesn't use any other > > DPDK features than rte_mbuf structure. The simplified packet format contains only > > essential parameters like address of the payload, packet length, etc. > > > > - user space app send path: When user space application wants to send > > a packet, it takes an mbufs from Alloc FIFO, writes packet to it and puts it > > into Rx queue. CP adapter which polls this queue gets the packet and enqueues > > to eventdev. Then the mbuf is freed by CP adapter. > > > > - use space app receive path: CP adapter puts the packets to Tx FIFO, > > user space application polls the FIFO and gets the packet. After consuming > > the packet it puts the mbuf to be freed to free FIFO. CP adapter regularly > > reads the free FIFO and frees the mbufs in it. > > > > The CP adapter can service multiple devices and queues. Each queue is > > configured with a servicing weight to control the relative frequency with which > > the adapter polls the queue, and the event fields to use when constructing > > packet events. > > > > API Summary > > > > FIFO APIs: > > - FIFO device open/close. > > - API to allocate PDU buffer from alloc queue. > > - API to send a burst of packets to an Rx FIFO identified by FIFO id and device id. > > - API to receive a burst of packets from Tx FIFO identified by FIFO id and device id. > > - API to put the used buffer to Free queue. Use application will call this after > > using the payload. > > > > Control Packet Adapter APIs: > > - Initialization API. > > - Device addition and removal APIs > > - Device configuration APIs. > > - FIFO add and remove APIs. > > - Event enqueue and dequeue APIs. > > > > > > We look forward for your feedback on this proposal. Once we have initial feedback, > > patches will be submitted for review. > > > > > > Thanks & Regards > > Ashok Kaladi > >