bitbake-devel.lists.openembedded.org archive mirror
 help / color / mirror / Atom feed
From: "Scott Murray" <scott.murray@konsulko.com>
To: bitbake-devel@lists.openembedded.org,
	Richard Purdie <richard.purdie@linuxfoundation.org>,
	Joshua Watt <JPEWhacker@gmail.com>,
	Paul Barker <paul@pbarker.dev>
Subject: [PATCH 1/1] prservice: remove connection caching
Date: Thu, 19 Aug 2021 12:50:38 -0400	[thread overview]
Message-ID: <cfd7ee07ef735c930773dd3298b5ce84abff986b.1629387539.git.scott.murray@konsulko.com> (raw)

This patch is a follow on of the the PR server rework in bitbake to add
read-only support.  The shift to using the bb.asyncrpc code in the PR
server and client brings issues with respect to reuse of the same
asyncio loop in different processes.  This patch removes the PR service
connection caching to avoid one source of this problem.  It is believed
that in practice this should have little impact on overall performance.

Signed-off-by: Scott Murray <scott.murray@konsulko.com>
---

NOTE: Do not apply this without the prerequisite bitbake PR server
      rework patches, see the bitbake-devel for the v6 patchset.

 meta/classes/package.bbclass |  5 ++---
 meta/lib/oe/prservice.py     | 25 ++++++++++++-------------
 2 files changed, 14 insertions(+), 16 deletions(-)

diff --git a/meta/classes/package.bbclass b/meta/classes/package.bbclass
index e17f0c797e..c4c5515d5d 100644
--- a/meta/classes/package.bbclass
+++ b/meta/classes/package.bbclass
@@ -714,9 +714,7 @@ python package_get_auto_pr() {
         return
 
     try:
-        conn = d.getVar("__PRSERV_CONN")
-        if conn is None:
-            conn = oe.prservice.prserv_make_conn(d)
+        conn = oe.prservice.prserv_make_conn(d)
         if conn is not None:
             if "AUTOINC" in pkgv:
                 srcpv = bb.fetch2.get_srcrev(d)
@@ -725,6 +723,7 @@ python package_get_auto_pr() {
                 d.setVar("PRSERV_PV_AUTOINC", str(value))
 
             auto_pr = conn.getPR(version, pkgarch, checksum)
+            conn.close()
     except Exception as e:
         bb.fatal("Can NOT get PRAUTO, exception %s" %  str(e))
     if auto_pr is None:
diff --git a/meta/lib/oe/prservice.py b/meta/lib/oe/prservice.py
index 15ce060ff6..339f7aebca 100644
--- a/meta/lib/oe/prservice.py
+++ b/meta/lib/oe/prservice.py
@@ -11,7 +11,6 @@ def prserv_make_conn(d, check = False):
         if check:
             if not conn.ping():
                 raise Exception('service not available')
-        d.setVar("__PRSERV_CONN",conn)
     except Exception as exc:
         bb.fatal("Connecting to PR service %s:%s failed: %s" % (host_params[0], host_params[1], str(exc)))
 
@@ -22,31 +21,29 @@ def prserv_dump_db(d):
         bb.error("Not using network based PR service")
         return None
 
-    conn = d.getVar("__PRSERV_CONN")
+    conn = prserv_make_conn(d)
     if conn is None:
-        conn = prserv_make_conn(d)
-        if conn is None:
-            bb.error("Making connection failed to remote PR service")
-            return None
+        bb.error("Making connection failed to remote PR service")
+        return None
 
     #dump db
     opt_version = d.getVar('PRSERV_DUMPOPT_VERSION')
     opt_pkgarch = d.getVar('PRSERV_DUMPOPT_PKGARCH')
     opt_checksum = d.getVar('PRSERV_DUMPOPT_CHECKSUM')
     opt_col = ("1" == d.getVar('PRSERV_DUMPOPT_COL'))
-    return conn.export(opt_version, opt_pkgarch, opt_checksum, opt_col)
+    d = conn.export(opt_version, opt_pkgarch, opt_checksum, opt_col)
+    conn.close()
+    return d
 
 def prserv_import_db(d, filter_version=None, filter_pkgarch=None, filter_checksum=None):
     if not d.getVar('PRSERV_HOST'):
         bb.error("Not using network based PR service")
         return None
 
-    conn = d.getVar("__PRSERV_CONN")
+    conn = prserv_make_conn(d)
     if conn is None:
-        conn = prserv_make_conn(d)
-        if conn is None:
-            bb.error("Making connection failed to remote PR service")
-            return None
+        bb.error("Making connection failed to remote PR service")
+        return None
     #get the entry values
     imported = []
     prefix = "PRAUTO$"
@@ -70,6 +67,7 @@ def prserv_import_db(d, filter_version=None, filter_pkgarch=None, filter_checksu
                 bb.error("importing(%s,%s,%s,%d) failed. DB may have larger value %d" % (version,pkgarch,checksum,value,ret))
             else:
                imported.append((version,pkgarch,checksum,value))
+    conn.close()
     return imported
 
 def prserv_export_tofile(d, metainfo, datainfo, lockdown, nomax=False):
@@ -125,4 +123,5 @@ def prserv_check_avail(d):
     except TypeError:
         bb.fatal('Undefined/incorrect PRSERV_HOST value. Format: "host:port"')
     else:
-        prserv_make_conn(d, True)
+        conn = prserv_make_conn(d, True)
+        conn.close()
-- 
2.31.1


                 reply	other threads:[~2021-08-19 16:50 UTC|newest]

Thread overview: [no followups] expand[flat|nested]  mbox.gz  Atom feed

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=cfd7ee07ef735c930773dd3298b5ce84abff986b.1629387539.git.scott.murray@konsulko.com \
    --to=scott.murray@konsulko.com \
    --cc=JPEWhacker@gmail.com \
    --cc=bitbake-devel@lists.openembedded.org \
    --cc=paul@pbarker.dev \
    --cc=richard.purdie@linuxfoundation.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).