[PATCH] prserv: make localhost work

[PATCH] prserv: make localhost work

[Yu, Mingli]

From: Mingli Yu <mingli.yu@windriver.com>

After [1] introduced, the PR server doesn't work in docker when use
below setting like before.
PRSERV_HOST = "localhost:0"

And it's because the localhost is resolved to an ipv6 address ::1 as
the below bitbake-prserv shows.
 bitbake$ bitbake-prserv --start --host=localhost --port=42005
 bitbake$ cat prserv.log
    Traceback (most recent call last):
      File "/OE/nodistro/honister/bitbake/bin/bitbake-prserv", line 55, in <module>
        ret = main()
      File "/OE/nodistro/honister/bitbake/bin/bitbake-prserv", line 46, in main
        ret=prserv.serv.start_daemon(options.dbfile, options.host, options.port,os.path.abspath(options.logfile), options.read_only)
      File "/OE/nodistro/honister/bitbake/lib/prserv/serv.py", line 226, in start_daemon
        run_as_daemon(daemon_main, pidfile, os.path.abspath(logfile))
      File "/OE/nodistro/honister/bitbake/lib/prserv/serv.py", line 202, in run_as_daemon
        func()
      File "/OE/nodistro/honister/bitbake/lib/prserv/serv.py", line 224, in daemon_main
        server.serve_forever()
      File "/OE/nodistro/honister/bitbake/lib/bb/asyncrpc/serv.py", line 233, in serve_forever
        self.start()
      File "/OE/nodistro/honister/bitbake/lib/bb/asyncrpc/serv.py", line 144, in start_tcp
        self.server = self.loop.run_until_complete(server_coro)
      File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
        return future.result()
      File "/usr/lib/python3.8/asyncio/streams.py", line 94, in start_server
        return await loop.create_server(factory, host, port, **kwds)
      File "/usr/lib/python3.8/asyncio/base_events.py", line 1463, in create_server
        raise OSError(err.errno, 'error while attempting '
    OSError: [Errno 99] error while attempting to bind on address ('::1', 42005, 0, 0): cannot assign requested address

So add the extra logic to make the localhost resolved as expected to
make the PR service work especially in docker.

[1] 6a2b23e2 prserv: Replace XML RPC with modern asyncrpc implementation

Signed-off-by: Mingli Yu <mingli.yu@windriver.com>
---
 lib/prserv/serv.py | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/lib/prserv/serv.py b/lib/prserv/serv.py
index 68db3d3e..0a20b927 100644
--- a/lib/prserv/serv.py
+++ b/lib/prserv/serv.py
@@ -126,7 +126,7 @@ class PRServSingleton(object):
 
     def start(self):
         self.prserv = PRServer(self.dbfile)
-        self.prserv.start_tcp_server(self.host, self.port)
+        self.prserv.start_tcp_server(socket.gethostbyname(self.host), self.port)
         self.process = self.prserv.serve_as_process()
 
         if not self.prserv.address:
@@ -221,7 +221,7 @@ def start_daemon(dbfile, host, port, logfile, read_only=False):
     dbfile = os.path.abspath(dbfile)
     def daemon_main():
         server = PRServer(dbfile, read_only=read_only)
-        server.start_tcp_server(host, port)
+        server.start_tcp_server(ip, port)
         server.serve_forever()
 
     run_as_daemon(daemon_main, pidfile, os.path.abspath(logfile))
-- 
2.17.1

Re: [PATCH] prserv: make localhost work

[Scott Murray]

On Tue, 31 Aug 2021, mingli.yu@windriver.com wrote:

> From: Mingli Yu <mingli.yu@windriver.com>
>
> After [1] introduced, the PR server doesn't work in docker when use
> below setting like before.
> PRSERV_HOST = "localhost:0"
>
> And it's because the localhost is resolved to an ipv6 address ::1 as
> the below bitbake-prserv shows.
>  bitbake$ bitbake-prserv --start --host=localhost --port=42005
>  bitbake$ cat prserv.log
>     Traceback (most recent call last):
>       File "/OE/nodistro/honister/bitbake/bin/bitbake-prserv", line 55, in <module>
>         ret = main()
>       File "/OE/nodistro/honister/bitbake/bin/bitbake-prserv", line 46, in main
>         ret=prserv.serv.start_daemon(options.dbfile, options.host, options.port,os.path.abspath(options.logfile), options.read_only)
>       File "/OE/nodistro/honister/bitbake/lib/prserv/serv.py", line 226, in start_daemon
>         run_as_daemon(daemon_main, pidfile, os.path.abspath(logfile))
>       File "/OE/nodistro/honister/bitbake/lib/prserv/serv.py", line 202, in run_as_daemon
>         func()
>       File "/OE/nodistro/honister/bitbake/lib/prserv/serv.py", line 224, in daemon_main
>         server.serve_forever()
>       File "/OE/nodistro/honister/bitbake/lib/bb/asyncrpc/serv.py", line 233, in serve_forever
>         self.start()
>       File "/OE/nodistro/honister/bitbake/lib/bb/asyncrpc/serv.py", line 144, in start_tcp
>         self.server = self.loop.run_until_complete(server_coro)
>       File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
>         return future.result()
>       File "/usr/lib/python3.8/asyncio/streams.py", line 94, in start_server
>         return await loop.create_server(factory, host, port, **kwds)
>       File "/usr/lib/python3.8/asyncio/base_events.py", line 1463, in create_server
>         raise OSError(err.errno, 'error while attempting '
>     OSError: [Errno 99] error while attempting to bind on address ('::1', 42005, 0, 0): cannot assign requested address
>
> So add the extra logic to make the localhost resolved as expected to
> make the PR service work especially in docker.
>
> [1] 6a2b23e2 prserv: Replace XML RPC with modern asyncrpc implementation
>
> Signed-off-by: Mingli Yu <mingli.yu@windriver.com>
> ---
>  lib/prserv/serv.py | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/lib/prserv/serv.py b/lib/prserv/serv.py
> index 68db3d3e..0a20b927 100644
> --- a/lib/prserv/serv.py
> +++ b/lib/prserv/serv.py
> @@ -126,7 +126,7 @@ class PRServSingleton(object):
>
>      def start(self):
>          self.prserv = PRServer(self.dbfile)
> -        self.prserv.start_tcp_server(self.host, self.port)
> +        self.prserv.start_tcp_server(socket.gethostbyname(self.host), self.port)
>          self.process = self.prserv.serve_as_process()
>
>          if not self.prserv.address:
> @@ -221,7 +221,7 @@ def start_daemon(dbfile, host, port, logfile, read_only=False):
>      dbfile = os.path.abspath(dbfile)
>      def daemon_main():
>          server = PRServer(dbfile, read_only=read_only)
> -        server.start_tcp_server(host, port)
> +        server.start_tcp_server(ip, port)
>          server.serve_forever()
>
>      run_as_daemon(daemon_main, pidfile, os.path.abspath(logfile))
>

Richard has picked up Martin's patch in master-next, I think that would
remove the need for this one?

Scott

Re: [PATCH] prserv: make localhost work

[Yu, Mingli]

On 8/31/21 11:01 PM, Scott Murray wrote:
> [Please note: This e-mail is from an EXTERNAL e-mail address]
> 
> On Tue, 31 Aug 2021, mingli.yu@windriver.com wrote:
> 
>> From: Mingli Yu <mingli.yu@windriver.com>
>>
>> After [1] introduced, the PR server doesn't work in docker when use
>> below setting like before.
>> PRSERV_HOST = "localhost:0"
>>
>> And it's because the localhost is resolved to an ipv6 address ::1 as
>> the below bitbake-prserv shows.
>>   bitbake$ bitbake-prserv --start --host=localhost --port=42005
>>   bitbake$ cat prserv.log
>>      Traceback (most recent call last):
>>        File "/OE/nodistro/honister/bitbake/bin/bitbake-prserv", line 55, in <module>
>>          ret = main()
>>        File "/OE/nodistro/honister/bitbake/bin/bitbake-prserv", line 46, in main
>>          ret=prserv.serv.start_daemon(options.dbfile, options.host, options.port,os.path.abspath(options.logfile), options.read_only)
>>        File "/OE/nodistro/honister/bitbake/lib/prserv/serv.py", line 226, in start_daemon
>>          run_as_daemon(daemon_main, pidfile, os.path.abspath(logfile))
>>        File "/OE/nodistro/honister/bitbake/lib/prserv/serv.py", line 202, in run_as_daemon
>>          func()
>>        File "/OE/nodistro/honister/bitbake/lib/prserv/serv.py", line 224, in daemon_main
>>          server.serve_forever()
>>        File "/OE/nodistro/honister/bitbake/lib/bb/asyncrpc/serv.py", line 233, in serve_forever
>>          self.start()
>>        File "/OE/nodistro/honister/bitbake/lib/bb/asyncrpc/serv.py", line 144, in start_tcp
>>          self.server = self.loop.run_until_complete(server_coro)
>>        File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
>>          return future.result()
>>        File "/usr/lib/python3.8/asyncio/streams.py", line 94, in start_server
>>          return await loop.create_server(factory, host, port, **kwds)
>>        File "/usr/lib/python3.8/asyncio/base_events.py", line 1463, in create_server
>>          raise OSError(err.errno, 'error while attempting '
>>      OSError: [Errno 99] error while attempting to bind on address ('::1', 42005, 0, 0): cannot assign requested address
>>
>> So add the extra logic to make the localhost resolved as expected to
>> make the PR service work especially in docker.
>>
>> [1] 6a2b23e2 prserv: Replace XML RPC with modern asyncrpc implementation
>>
>> Signed-off-by: Mingli Yu <mingli.yu@windriver.com>
>> ---
>>   lib/prserv/serv.py | 4 ++--
>>   1 file changed, 2 insertions(+), 2 deletions(-)
>>
>> diff --git a/lib/prserv/serv.py b/lib/prserv/serv.py
>> index 68db3d3e..0a20b927 100644
>> --- a/lib/prserv/serv.py
>> +++ b/lib/prserv/serv.py
>> @@ -126,7 +126,7 @@ class PRServSingleton(object):
>>
>>       def start(self):
>>           self.prserv = PRServer(self.dbfile)
>> -        self.prserv.start_tcp_server(self.host, self.port)
>> +        self.prserv.start_tcp_server(socket.gethostbyname(self.host), self.port)
>>           self.process = self.prserv.serve_as_process()
>>
>>           if not self.prserv.address:
>> @@ -221,7 +221,7 @@ def start_daemon(dbfile, host, port, logfile, read_only=False):
>>       dbfile = os.path.abspath(dbfile)
>>       def daemon_main():
>>           server = PRServer(dbfile, read_only=read_only)
>> -        server.start_tcp_server(host, port)
>> +        server.start_tcp_server(ip, port)
>>           server.serve_forever()
>>
>>       run_as_daemon(daemon_main, pidfile, os.path.abspath(logfile))
>>
> 
> Richard has picked up Martin's patch in master-next, I think that would
> remove the need for this one?

The patch from Martin is just make the error message specific, but 
localhost still does't work in docker with Martin's patch and this patch 
is used to make it work in docker.

Thanks,

> 
> Scott
> 

Re: [bitbake-devel] [PATCH] prserv: make localhost work

[Joshua Watt]

On Tue, Aug 31, 2021 at 8:14 PM Yu, Mingli <mingli.yu@windriver.com> wrote:
>
>
>
> On 8/31/21 11:01 PM, Scott Murray wrote:
> > [Please note: This e-mail is from an EXTERNAL e-mail address]
> >
> > On Tue, 31 Aug 2021, mingli.yu@windriver.com wrote:
> >
> >> From: Mingli Yu <mingli.yu@windriver.com>
> >>
> >> After [1] introduced, the PR server doesn't work in docker when use
> >> below setting like before.
> >> PRSERV_HOST = "localhost:0"
> >>
> >> And it's because the localhost is resolved to an ipv6 address ::1 as
> >> the below bitbake-prserv shows.
> >>   bitbake$ bitbake-prserv --start --host=localhost --port=42005
> >>   bitbake$ cat prserv.log
> >>      Traceback (most recent call last):
> >>        File "/OE/nodistro/honister/bitbake/bin/bitbake-prserv", line 55, in <module>
> >>          ret = main()
> >>        File "/OE/nodistro/honister/bitbake/bin/bitbake-prserv", line 46, in main
> >>          ret=prserv.serv.start_daemon(options.dbfile, options.host, options.port,os.path.abspath(options.logfile), options.read_only)
> >>        File "/OE/nodistro/honister/bitbake/lib/prserv/serv.py", line 226, in start_daemon
> >>          run_as_daemon(daemon_main, pidfile, os.path.abspath(logfile))
> >>        File "/OE/nodistro/honister/bitbake/lib/prserv/serv.py", line 202, in run_as_daemon
> >>          func()
> >>        File "/OE/nodistro/honister/bitbake/lib/prserv/serv.py", line 224, in daemon_main
> >>          server.serve_forever()
> >>        File "/OE/nodistro/honister/bitbake/lib/bb/asyncrpc/serv.py", line 233, in serve_forever
> >>          self.start()
> >>        File "/OE/nodistro/honister/bitbake/lib/bb/asyncrpc/serv.py", line 144, in start_tcp
> >>          self.server = self.loop.run_until_complete(server_coro)
> >>        File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
> >>          return future.result()
> >>        File "/usr/lib/python3.8/asyncio/streams.py", line 94, in start_server
> >>          return await loop.create_server(factory, host, port, **kwds)
> >>        File "/usr/lib/python3.8/asyncio/base_events.py", line 1463, in create_server
> >>          raise OSError(err.errno, 'error while attempting '
> >>      OSError: [Errno 99] error while attempting to bind on address ('::1', 42005, 0, 0): cannot assign requested address
> >>
> >> So add the extra logic to make the localhost resolved as expected to
> >> make the PR service work especially in docker.
> >>
> >> [1] 6a2b23e2 prserv: Replace XML RPC with modern asyncrpc implementation
> >>
> >> Signed-off-by: Mingli Yu <mingli.yu@windriver.com>
> >> ---
> >>   lib/prserv/serv.py | 4 ++--
> >>   1 file changed, 2 insertions(+), 2 deletions(-)
> >>
> >> diff --git a/lib/prserv/serv.py b/lib/prserv/serv.py
> >> index 68db3d3e..0a20b927 100644
> >> --- a/lib/prserv/serv.py
> >> +++ b/lib/prserv/serv.py
> >> @@ -126,7 +126,7 @@ class PRServSingleton(object):
> >>
> >>       def start(self):
> >>           self.prserv = PRServer(self.dbfile)
> >> -        self.prserv.start_tcp_server(self.host, self.port)
> >> +        self.prserv.start_tcp_server(socket.gethostbyname(self.host), self.port)
> >>           self.process = self.prserv.serve_as_process()
> >>
> >>           if not self.prserv.address:
> >> @@ -221,7 +221,7 @@ def start_daemon(dbfile, host, port, logfile, read_only=False):
> >>       dbfile = os.path.abspath(dbfile)
> >>       def daemon_main():
> >>           server = PRServer(dbfile, read_only=read_only)
> >> -        server.start_tcp_server(host, port)
> >> +        server.start_tcp_server(ip, port)
> >>           server.serve_forever()
> >>
> >>       run_as_daemon(daemon_main, pidfile, os.path.abspath(logfile))
> >>
> >
> > Richard has picked up Martin's patch in master-next, I think that would
> > remove the need for this one?
>
> The patch from Martin is just make the error message specific, but
> localhost still does't work in docker with Martin's patch and this patch
> is used to make it work in docker.

I think that was because IPv6 support had been incorrectly configured
for Docker, perhaps that is the same thing happening in your setup?

>
> Thanks,
>
> >
> > Scott
> >
>
> 
>

Re: [bitbake-devel] [PATCH] prserv: make localhost work

[Yu, Mingli]

On 9/1/21 9:31 AM, Joshua Watt wrote:
> [Please note: This e-mail is from an EXTERNAL e-mail address]
> 
> On Tue, Aug 31, 2021 at 8:14 PM Yu, Mingli <mingli.yu@windriver.com> wrote:
>>
>>
>>
>> On 8/31/21 11:01 PM, Scott Murray wrote:
>>> [Please note: This e-mail is from an EXTERNAL e-mail address]
>>>
>>> On Tue, 31 Aug 2021, mingli.yu@windriver.com wrote:
>>>
>>>> From: Mingli Yu <mingli.yu@windriver.com>
>>>>
>>>> After [1] introduced, the PR server doesn't work in docker when use
>>>> below setting like before.
>>>> PRSERV_HOST = "localhost:0"
>>>>
>>>> And it's because the localhost is resolved to an ipv6 address ::1 as
>>>> the below bitbake-prserv shows.
>>>>    bitbake$ bitbake-prserv --start --host=localhost --port=42005
>>>>    bitbake$ cat prserv.log
>>>>       Traceback (most recent call last):
>>>>         File "/OE/nodistro/honister/bitbake/bin/bitbake-prserv", line 55, in <module>
>>>>           ret = main()
>>>>         File "/OE/nodistro/honister/bitbake/bin/bitbake-prserv", line 46, in main
>>>>           ret=prserv.serv.start_daemon(options.dbfile, options.host, options.port,os.path.abspath(options.logfile), options.read_only)
>>>>         File "/OE/nodistro/honister/bitbake/lib/prserv/serv.py", line 226, in start_daemon
>>>>           run_as_daemon(daemon_main, pidfile, os.path.abspath(logfile))
>>>>         File "/OE/nodistro/honister/bitbake/lib/prserv/serv.py", line 202, in run_as_daemon
>>>>           func()
>>>>         File "/OE/nodistro/honister/bitbake/lib/prserv/serv.py", line 224, in daemon_main
>>>>           server.serve_forever()
>>>>         File "/OE/nodistro/honister/bitbake/lib/bb/asyncrpc/serv.py", line 233, in serve_forever
>>>>           self.start()
>>>>         File "/OE/nodistro/honister/bitbake/lib/bb/asyncrpc/serv.py", line 144, in start_tcp
>>>>           self.server = self.loop.run_until_complete(server_coro)
>>>>         File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
>>>>           return future.result()
>>>>         File "/usr/lib/python3.8/asyncio/streams.py", line 94, in start_server
>>>>           return await loop.create_server(factory, host, port, **kwds)
>>>>         File "/usr/lib/python3.8/asyncio/base_events.py", line 1463, in create_server
>>>>           raise OSError(err.errno, 'error while attempting '
>>>>       OSError: [Errno 99] error while attempting to bind on address ('::1', 42005, 0, 0): cannot assign requested address
>>>>
>>>> So add the extra logic to make the localhost resolved as expected to
>>>> make the PR service work especially in docker.
>>>>
>>>> [1] 6a2b23e2 prserv: Replace XML RPC with modern asyncrpc implementation
>>>>
>>>> Signed-off-by: Mingli Yu <mingli.yu@windriver.com>
>>>> ---
>>>>    lib/prserv/serv.py | 4 ++--
>>>>    1 file changed, 2 insertions(+), 2 deletions(-)
>>>>
>>>> diff --git a/lib/prserv/serv.py b/lib/prserv/serv.py
>>>> index 68db3d3e..0a20b927 100644
>>>> --- a/lib/prserv/serv.py
>>>> +++ b/lib/prserv/serv.py
>>>> @@ -126,7 +126,7 @@ class PRServSingleton(object):
>>>>
>>>>        def start(self):
>>>>            self.prserv = PRServer(self.dbfile)
>>>> -        self.prserv.start_tcp_server(self.host, self.port)
>>>> +        self.prserv.start_tcp_server(socket.gethostbyname(self.host), self.port)
>>>>            self.process = self.prserv.serve_as_process()
>>>>
>>>>            if not self.prserv.address:
>>>> @@ -221,7 +221,7 @@ def start_daemon(dbfile, host, port, logfile, read_only=False):
>>>>        dbfile = os.path.abspath(dbfile)
>>>>        def daemon_main():
>>>>            server = PRServer(dbfile, read_only=read_only)
>>>> -        server.start_tcp_server(host, port)
>>>> +        server.start_tcp_server(ip, port)
>>>>            server.serve_forever()
>>>>
>>>>        run_as_daemon(daemon_main, pidfile, os.path.abspath(logfile))
>>>>
>>>
>>> Richard has picked up Martin's patch in master-next, I think that would
>>> remove the need for this one?
>>
>> The patch from Martin is just make the error message specific, but
>> localhost still does't work in docker with Martin's patch and this patch
>> is used to make it work in docker.
> 
> I think that was because IPv6 support had been incorrectly configured
> for Docker, perhaps that is the same thing happening in your setup?

In the same docker, the PR service works with PRSERV_HOST = 
"localhost:0" before, but it begins not work recently after some code 
restructure for prserv part.

BTW, the ipv6 is disabled in the docker of my env.

Thanks,

> 
>>
>> Thanks,
>>
>>>
>>> Scott
>>>
>>
>> 
>>

Re: [bitbake-devel] [PATCH] prserv: make localhost work

[Martin Jansa]

[-- Attachment #1: Type: text/plain, Size: 1027 bytes --]

>
>
> > I think that was because IPv6 support had been incorrectly configured
> > for Docker, perhaps that is the same thing happening in your setup?
>
> In the same docker, the PR service works with PRSERV_HOST =
> "localhost:0" before, but it begins not work recently after some code
> restructure for prserv part.
>
> BTW, the ipv6 is disabled in the docker of my env.
>

Check your /etc/hosts

There is ::1 localhost localhost-ip6 even when ipv6 is disabled in docker's
deamon.json.

And that makes bitbake (asyncio) to try binding to ::1 instead of
127.0.0.1, if you remove "localhost" from ::1 in /etc/hosts in running
container, then it works again OK.

My patch allows you to use 127.0.0.1:0 explicitly and still keep the same
behavior (as autostarted prserv with random port) as when localhost:0 is
being used (and also for consistency).

Your patch is IMHO good addition to that (that the localhost is resolved in
python and then people don't need to use 127.0.0.1 explicitly).

Regards,

[-- Attachment #2: Type: text/html, Size: 1423 bytes --]