From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-0.5 required=3.0 tests=BAYES_00,DKIM_ADSP_CUSTOM_MED, DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1624CC433E0 for ; Wed, 6 Jan 2021 01:15:43 +0000 (UTC) Received: from shelob.surriel.com (shelob.surriel.com [96.67.55.147]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 961B022CB1 for ; Wed, 6 Jan 2021 01:15:42 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 961B022CB1 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kernelnewbies-bounces+kernelnewbies=archiver.kernel.org@kernelnewbies.org Received: from localhost ([::1] helo=shelob.surriel.com) by shelob.surriel.com with esmtp (Exim 4.94) (envelope-from ) id 1kwxQJ-0003im-9I for kernelnewbies@archiver.kernel.org; Tue, 05 Jan 2021 20:15:39 -0500 Received: from mail-ot1-x32a.google.com ([2607:f8b0:4864:20::32a]) by shelob.surriel.com with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.94) (envelope-from ) id 1kwxOh-0001kz-DI for kernelnewbies@kernelnewbies.org; Tue, 05 Jan 2021 20:13:59 -0500 Received: by mail-ot1-x32a.google.com with SMTP id n42so1559756ota.12 for ; Tue, 05 Jan 2021 17:13:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=HX9PWPFloQFdeYgoTvcJRqVEaJRr4iEgM4V1ZoxPN8k=; b=ttKFdbo9MIeb8pUhxR6h72PqTY044HGG3f39WTque+Yjm2SOOq78fePOO1UyURi5li 1cnx4Wi3Y7nnhsnm8l9uUcvt4N6tnzOJIuDW2eSK1k1VUcmgYwdrHOox1A0nCQbZRbku zY2LnoxWTDGW+TI9eRR9lS55hl6UsD/1wuoCqQhgqfkG2duO81/GtqxgPjQtxOzUgfb9 H66ZikilRBtVpCM/yAaLyyfOeVxcmwWp+Xk1kPwshfl35+7jbFY+LlIa6V3t5ZQaxlga R5657265oaPm2rfrMYk4BcK8d78FzEjdUJecM26NgME98EJdDdZBv6O3tLxbxcmGc2BA co7A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=HX9PWPFloQFdeYgoTvcJRqVEaJRr4iEgM4V1ZoxPN8k=; b=QcwRmKxEZ9xdHEluB98LA6q7CFL857ytey7ZZZZtyIQ+Qz6K1oFbqjzPafUC3VaX5E E9f8HYLQh1MNpBBTFujZAGza80dfj/bNMlA2gCjk+Fm8ys2Xbx7ptNPFwrP3QGlzVTwg j9yr82sAkim8Y+j2SpL4uGJwsChBUDHIT//FqbcxV9pDaaECNUF3GXCHpy+uTgO3U64j U2bOz7XvgQZt4yiJzkc/4Yh6horuHnzj3d0DI8a42cxa+gZVunqxbRhe8V8Bc8yLYAlP thCB1r5DTUwYNIcFJPmSVee5chC41AkhY7nCT5PL1okP50U0nvrRB+DGa7p8FP8FodRr B+RA== X-Gm-Message-State: AOAM532VxN6pd9eAa7D7M8pWaZpQYzzjkq4pixB5P+ThVrsVsj20ql0k kxTPJaqluAAfc93iMmUA2iSb6qRXVn9PI1bjODRcOUdP4S8= X-Google-Smtp-Source: ABdhPJwi/kb8ZJYteVD0n2+cq/LXnfWUQn1VV0rLb0yzCksgzRGY2+SCFCfSXU2d0oKge1WPG2uvtf1cNmjXr3UR3qk= X-Received: by 2002:a05:6830:1501:: with SMTP id k1mr1673673otp.12.1609895637482; Tue, 05 Jan 2021 17:13:57 -0800 (PST) MIME-Version: 1.0 From: Joe Doe Date: Tue, 5 Jan 2021 17:13:48 -0800 Message-ID: Subject: Linux has timerfd, signalfd and eventfd. Is it time for a "unionfd"? To: kernelnewbies@kernelnewbies.org X-BeenThere: kernelnewbies@kernelnewbies.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Learn about the Linux kernel List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: kernelnewbies-bounces+kernelnewbies=archiver.kernel.org@kernelnewbies.org One of the architectures that Linux applications use, is to run in a loop that watches a number of FDs (e.g. epoll()) and react to the events from those FDs. Now let's say I am writing a communication library that users can use to send and receive messages to other processes over the network. And let's assume that for reasons, my library is using multiple connections internally. I have no way to construct my library in such a way that it will only export a single FD to the user, and hide the rest of the internal connections. I will either have to expose all the FDs, making it more difficult for the user, or I would have to run the library code in a different thread, and use eventfd to make the interface simpler. Having a "UnionFD" that would get triggered when one of my other FDs is ready for I/O would solve the problem. For example an API of the hypothetical library with the tools we have today might look something like this: int comm_lib_init(void); // Returns the "master" FD int comm_lib_accept(int master_fd); // Returns an FD when a new connection is accepted int comm_lib_send_message(int master_fd, const char *dest, void *data, size_t data_len); // If a new connection was required to send this message, then it will return the FD int comm_lib_recv_message(int master_fd, ...); The user of the library would have to do the bookkeeping for all the FDs that are returned by the above APIs themself. Instead with a "UnionFD", the API would only ever return a single FD and the user would call `comm_lib_send_message` and `comm_lib_recv_message` with this FD. E.g. the application's code would look something like that: comm_fd = comm_lib_init(); epollfd = poll_create1(0); ev.events = EPOLLIN; ev.data.fd = comm_fd; epoll_ctl(epollfd, EPOLL_CTL_ADD, comm_fd, &ev); for (;;) { fds = epoll_wait(epollfd, events, MAX_EVENTS, -1); } Once in the loop, the user doesn't have to worry about FD bookkeeping anymore. Another example would be a much simpler HTTP library. The library would just return a single "master_fd" that the user would plug in their epoll() and just call something like `get_next_request()` when the FD gets triggered. That way we could easily add this HTTP server to an epoll() loop that monitors other FDs as well. No more worrying about accept(), and per-connection FD bookkeeping. I believe a construct like this would help some popular existing libraries as well, like zeromq and the grpc library. Does the above make sense? Or is there already a way to achieve that in Linux that I am not aware of? Thanks, Joe _______________________________________________ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies