Why the edge-triggered mode doesn't work for epoll file descriptor?

Why the edge-triggered mode doesn't work for epoll file descriptor?

[Heiher]

Hello,

I've added a pipe file descriptor (fd1) to an epoll (fd3) with
EPOLLOUT in edge-triggered mode, and then added the fd3 to another
epoll (fd4) with EPOLLIN in edge-triggered too.

Next, waiting for fd4 without timeout. When fd1 to be writable, i
think epoll_wait(fd4, ...)  only return once, because all file
descriptors are added in edge-triggered mode.

But, the actual result is returns many and many times until do once
eopll_wait(fd3, ...).

#include <stdio.h>
#include <unistd.h>
#include <sys/epoll.h>

int
main (int argc, char *argv[])
{
    int efd[2];
    struct epoll_event e;

    efd[0] = epoll_create (1);
    if (efd[0] < 0)
        return -1;

    efd[1] = epoll_create (1);
    if (efd[1] < 0)
        return -2;

    e.events = EPOLLIN | EPOLLET;
    e.data.u64 = 1;
    if (epoll_ctl (efd[0], EPOLL_CTL_ADD, efd[1], &e) < 0)
        return -3;

    e.events = EPOLLOUT | EPOLLET;
    e.data.u64 = 2;
    if (epoll_ctl (efd[1], EPOLL_CTL_ADD, 1, &e) < 0)
        return -4;

    for (;;) {
        struct epoll_event events[16];
        int nfds;

        nfds = epoll_wait (efd[0], events, 16, -1);
        printf ("nfds: %d\n", nfds);
    }

    close (efd[1]);
    close (efd[0]);

    return 0;
}

-- 
Best regards!
Hev
https://hev.cc

Re: Why the edge-triggered mode doesn't work for epoll file descriptor?

[Heiher]

On Sun, Aug 18, 2019 at 12:36 AM Heiher <r@hev.cc> wrote:
>
> Hello,
>
> I've added a pipe file descriptor (fd1) to an epoll (fd3) with
> EPOLLOUT in edge-triggered mode, and then added the fd3 to another
> epoll (fd4) with EPOLLIN in edge-triggered too.
>
> Next, waiting for fd4 without timeout. When fd1 to be writable, i
> think epoll_wait(fd4, ...)  only return once, because all file
> descriptors are added in edge-triggered mode.
>
> But, the actual result is returns many and many times until do once
> eopll_wait(fd3, ...).
>
> #include <stdio.h>
> #include <unistd.h>
> #include <sys/epoll.h>
>
> int
> main (int argc, char *argv[])
> {
>     int efd[2];
>     struct epoll_event e;
>
>     efd[0] = epoll_create (1);
>     if (efd[0] < 0)
>         return -1;
>
>     efd[1] = epoll_create (1);
>     if (efd[1] < 0)
>         return -2;
>
>     e.events = EPOLLIN | EPOLLET;
>     e.data.u64 = 1;
>     if (epoll_ctl (efd[0], EPOLL_CTL_ADD, efd[1], &e) < 0)
>         return -3;
>
>     e.events = EPOLLOUT | EPOLLET;
>     e.data.u64 = 2;
>     if (epoll_ctl (efd[1], EPOLL_CTL_ADD, 1, &e) < 0)
>         return -4;
>
>     for (;;) {
>         struct epoll_event events[16];
>         int nfds;
>
>         nfds = epoll_wait (efd[0], events, 16, -1);
>         printf ("nfds: %d\n", nfds);
>     }
>
>     close (efd[1]);
>     close (efd[0]);
>
>     return 0;
> }
>
> --
> Best regards!
> Hev
> https://hev.cc

Is this behavior correct? any help?

-- 
Best regards!
Hev
https://hev.cc

Re: Why the edge-triggered mode doesn't work for epoll file descriptor?

[Eric Wong]

Heiher <r@hev.cc> wrote:
> Hello,
> 
> I've added a pipe file descriptor (fd1) to an epoll (fd3) with
> EPOLLOUT in edge-triggered mode, and then added the fd3 to another
> epoll (fd4) with EPOLLIN in edge-triggered too.
> 
> Next, waiting for fd4 without timeout. When fd1 to be writable, i
> think epoll_wait(fd4, ...)  only return once, because all file
> descriptors are added in edge-triggered mode.
> 
> But, the actual result is returns many and many times until do once
> eopll_wait(fd3, ...).

It looks like you can trigger a wakeup loop with printf writing
to the terminal (not a pipe), and that write to the terminal
triggering the EPOLLOUT wakeup over and over again.

I don't know TTY stuff at all, but I assume it's intended
for terminals.

You refer to "pipe file descriptor (fd1)", but I can't reproduce
the error when running your code piped to "tee" and using
strace to check epoll_wait returns.

"strace ./foo | tee /dev/null" only shows one epoll_wait returning.

>     e.events = EPOLLIN | EPOLLET;
>     e.data.u64 = 1;
>     if (epoll_ctl (efd[0], EPOLL_CTL_ADD, efd[1], &e) < 0)
>         return -3;
> 
>     e.events = EPOLLOUT | EPOLLET;
>     e.data.u64 = 2;
>     if (epoll_ctl (efd[1], EPOLL_CTL_ADD, 1, &e) < 0)
>         return -4;

Since epfd[1] is waiting for stdout...

>     for (;;) {
>         struct epoll_event events[16];
>         int nfds;
> 
>         nfds = epoll_wait (efd[0], events, 16, -1);
>         printf ("nfds: %d\n", nfds);

Try outputting your message to stderr instead of stdout:

        fprintf(stderr, "nfds: %d\n", nfds);

And then run your program so stdout and stderr point to
different files:

	./foo | tee /dev/null

(so stdout becomes a pipe, and stderr remains your terminal)

Re: Why the edge-triggered mode doesn't work for epoll file descriptor?

[Heiher]

Hello,

Thank you reply.

On Mon, Aug 26, 2019 at 2:22 PM Eric Wong <e@80x24.org> wrote:
>
> Heiher <r@hev.cc> wrote:
> > Hello,
> >
> > I've added a pipe file descriptor (fd1) to an epoll (fd3) with
> > EPOLLOUT in edge-triggered mode, and then added the fd3 to another
> > epoll (fd4) with EPOLLIN in edge-triggered too.
> >
> > Next, waiting for fd4 without timeout. When fd1 to be writable, i
> > think epoll_wait(fd4, ...)  only return once, because all file
> > descriptors are added in edge-triggered mode.
> >
> > But, the actual result is returns many and many times until do once
> > eopll_wait(fd3, ...).
>
> It looks like you can trigger a wakeup loop with printf writing
> to the terminal (not a pipe), and that write to the terminal
> triggering the EPOLLOUT wakeup over and over again.
>
> I don't know TTY stuff at all, but I assume it's intended
> for terminals.
>
> You refer to "pipe file descriptor (fd1)", but I can't reproduce
> the error when running your code piped to "tee" and using
> strace to check epoll_wait returns.
>
> "strace ./foo | tee /dev/null" only shows one epoll_wait returning.
>
> >     e.events = EPOLLIN | EPOLLET;
> >     e.data.u64 = 1;
> >     if (epoll_ctl (efd[0], EPOLL_CTL_ADD, efd[1], &e) < 0)
> >         return -3;
> >
> >     e.events = EPOLLOUT | EPOLLET;
> >     e.data.u64 = 2;
> >     if (epoll_ctl (efd[1], EPOLL_CTL_ADD, 1, &e) < 0)
> >         return -4;
>
> Since epfd[1] is waiting for stdout...
>
> >     for (;;) {
> >         struct epoll_event events[16];
> >         int nfds;
> >
> >         nfds = epoll_wait (efd[0], events, 16, -1);
> >         printf ("nfds: %d\n", nfds);
>
> Try outputting your message to stderr instead of stdout:
>
>         fprintf(stderr, "nfds: %d\n", nfds);
>
> And then run your program so stdout and stderr point to
> different files:
>
>         ./foo | tee /dev/null
>
> (so stdout becomes a pipe, and stderr remains your terminal)

OK, Let's use a set of test cases without external interference to
indicate the problem.

epoll1.c:
#include <stdio.h>
#include <unistd.h>
#include <sys/epoll.h>
#include <sys/socket.h>

int
main (int argc, char *argv[])
{
    int sfd[2];
    int efd[2];
    struct epoll_event e;

    if (socketpair (AF_UNIX, SOCK_STREAM, 0, sfd) < 0)
        return -1;

    efd[0] = epoll_create (1);
    if (efd[0] < 0)
        return -2;

    efd[1] = epoll_create (1);
    if (efd[1] < 0)
        return -3;

    e.events = EPOLLIN | EPOLLET;
    e.data.u64 = 1;
    if (epoll_ctl (efd[0], EPOLL_CTL_ADD, efd[1], &e) < 0)
        return -3;

    e.events = EPOLLIN;
    e.data.u64 = 2;
    if (epoll_ctl (efd[1], EPOLL_CTL_ADD, sfd[0], &e) < 0)
        return -4;

    /**
     * Current structure:
     *     efd[0]:
     *     {
     *         efd[1] (EPOLLIN | EPOLLET):
     *         {
     *             sfd[0] (EPOLLIN)
     *         }
     *     }
     */

    /* Make the sfd[0] is readable */
    if (write (sfd[1], "a", 1) != 1)
        return -5;

    for (;;) {
        struct epoll_event events[16];
        int nfds;

        /**
         * IIRC, the epoll_wait(efd[0]) returns while efd[1] events
changed only,
         *
         * so, the first call should be returned, because sfd[0] from
not readable
         * to readable.
         *
         * and then the calls should be blocked, because not any fd's event
         * changed in efd[1] pool, and efd[1] is working in edge-triggerd mode,
         * so, efd[1]'s event not changed.
         */
        nfds = epoll_wait (efd[0], events, 16, -1);
        printf ("nfds: %d\n", nfds);
    }

    close (efd[1]);
    close (efd[0]);
    close (sfd[0]);
    close (sfd[1]);

    return 0;
}

epoll2.c:
#include <stdio.h>
#include <unistd.h>
#include <sys/epoll.h>
#include <sys/socket.h>

int
main (int argc, char *argv[])
{
    int sfd[2];
    int efd;
    struct epoll_event e;

    if (socketpair (AF_UNIX, SOCK_STREAM, 0, sfd) < 0)
        return -1;

    efd = epoll_create (1);
    if (efd < 0)
        return -2;

    e.events = EPOLLIN | EPOLLET;
    e.data.u64 = 1;
    if (epoll_ctl (efd, EPOLL_CTL_ADD, sfd[0], &e) < 0)
        return -3;

    /**
     * Current structure:
     *     efd:
     *     {
     *         sfd[0] (EPOLLIN | EPOLLET)
     *     }
     */

    /* Make the sfd[0] is readable */
    if (write (sfd[1], "a", 1) != 1)
        return -5;

    for (;;) {
        struct epoll_event events[16];
        int nfds;

        nfds = epoll_wait (efd, events, 16, -1);
        printf ("nfds: %d\n", nfds);
    }

    close (efd);
    close (sfd[0]);
    close (sfd[1]);

    return 0;
}

I don't know why the epoll1 prints many and many times, i think the
epoll2 is correctly behavior.

-- 
Best regards!
Hev
https://hev.cc