Unfortunately this seems to completely break SSTATE_MIRROR handling file:// at least in some setups.

All setscene tasks are failing, because there is a symlink in sstate-cache, but when trying to use it, it points to relative path into wrong directory.

e.g.
DEBUG: Executing python function do_package_qa_setscene
DEBUG: Searching for fb/a4/sstate:python3-ifcfg:cortexa72-oe-linux:0.21:r0:cortexa72:3:fba4a512f137dced352b65244a828d6014453f4f3899c45b178a7cbcf8b5d8b2_package_qa.tgz in paths:
    /jenkins/mjansa/build/ros/ros2-rolling-hardknott/sstate-cache
DEBUG: Defaulting to /jenkins/mjansa/build/ros/ros2-rolling-hardknott/sstate-cache/fb/a4/sstate:python3-ifcfg:cortexa72-oe-linux:0.21:r0:cortexa72:3:fba4a512f137dced352b65244a828d6014453f4f3899c45b178a7cbcf8b5d8b2_package_qa.tgz for fb/a4/sstate:python3-ifcfg:cor
texa72-oe-linux:0.21:r0:cortexa72:3:fba4a512f137dced352b65244a828d6014453f4f3899c45b178a7cbcf8b5d8b2_package_qa.tgz
DEBUG: Testing URL file://fb/a4/sstate:python3-ifcfg:cortexa72-oe-linux:0.21:r0:cortexa72:3:fba4a512f137dced352b65244a828d6014453f4f3899c45b178a7cbcf8b5d8b2_package_qa.tgz;downloadfilename=fb/a4/sstate:python3-ifcfg:cortexa72-oe-linux:0.21:r0:cortexa72:3:fba4a512f137dced352b65244a828d6014453f4f3899c45b178a7cbcf8b5d8b2_package_qa.tgz
DEBUG: For url ['file', '', 'fb/a4/sstate:python3-ifcfg:cortexa72-oe-linux:0.21:r0:cortexa72:3:fba4a512f137dced352b65244a828d6014453f4f3899c45b178a7cbcf8b5d8b2_package_qa.tgz', '', '', OrderedDict([('downloadfilename', 'fb/a4/sstate:python3-ifcfg:cortexa72-oe-linux:0.21:r0:cortexa72:3:fba4a512f137dced352b65244a828d6014453f4f3899c45b178a7cbcf8b5d8b2_package_qa.tgz')])] comparing ['file', '', '.*', '', '', OrderedDict()] to ['file', '', '/mnt/mirror-write-ros/jansa/webos/ros-hardknott/sstate-cache/PATH', '', '', OrderedDict()]
DEBUG: For url file://fb/a4/sstate:python3-ifcfg:cortexa72-oe-linux:0.21:r0:cortexa72:3:fba4a512f137dced352b65244a828d6014453f4f3899c45b178a7cbcf8b5d8b2_package_qa.tgz;downloadfilename=fb/a4/sstate:python3-ifcfg:cortexa72-oe-linux:0.21:r0:cortexa72:3:fba4a512f137dced352b65244a828d6014453f4f3899c45b178a7cbcf8b5d8b2_package_qa.tgz returning file:///mnt/mirror-write-ros/jansa/webos/ros-hardknott/sstate-cache/fb/a4/sstate%3Apython3-ifcfg%3Acortexa72-oe-linux%3A0.21%3Ar0%3Acortexa72%3A3%3Afba4a512f137dced352b65244a828d6014453f4f3899c45b178a7cbcf8b5d8b2_package_qa.tgz;downloadfilename=fb/a4/sstate:python3-ifcfg:cortexa72-oe-linux:0.21:r0:cortexa72:3:fba4a512f137dced352b65244a828d6014453f4f3899c45b178a7cbcf8b5d8b2_package_qa.tgz
DEBUG: Trying PREMIRRORS
DEBUG: For url ['file', '', 'fb/a4/sstate:python3-ifcfg:cortexa72-oe-linux:0.21:r0:cortexa72:3:fba4a512f137dced352b65244a828d6014453f4f3899c45b178a7cbcf8b5d8b2_package_qa.tgz', '', '', OrderedDict([('downloadfilename', 'fb/a4/sstate:python3-ifcfg:cortexa72-oe-linux:0.21:r0:cortexa72:3:fba4a512f137dced352b65244a828d6014453f4f3899c45b178a7cbcf8b5d8b2_package_qa.tgz')])] comparing ['file', '', '.*', '', '', OrderedDict()] to ['file', '', '/mnt/mirror-write-ros/jansa/webos/ros-hardknott/sstate-cache/PATH', '', '', OrderedDict()]
DEBUG: For url file://fb/a4/sstate:python3-ifcfg:cortexa72-oe-linux:0.21:r0:cortexa72:3:fba4a512f137dced352b65244a828d6014453f4f3899c45b178a7cbcf8b5d8b2_package_qa.tgz;downloadfilename=fb/a4/sstate:python3-ifcfg:cortexa72-oe-linux:0.21:r0:cortexa72:3:fba4a512f137dced352b65244a828d6014453f4f3899c45b178a7cbcf8b5d8b2_package_qa.tgz returning file:///mnt/mirror-write-ros/jansa/webos/ros-hardknott/sstate-cache/fb/a4/sstate%3Apython3-ifcfg%3Acortexa72-oe-linux%3A0.21%3Ar0%3Acortexa72%3A3%3Afba4a512f137dced352b65244a828d6014453f4f3899c45b178a7cbcf8b5d8b2_package_qa.tgz;downloadfilename=fb/a4/sstate:python3-ifcfg:cortexa72-oe-linux:0.21:r0:cortexa72:3:fba4a512f137dced352b65244a828d6014453f4f3899c45b178a7cbcf8b5d8b2_package_qa.tgz
DEBUG: Searching for fb/a4/sstate:python3-ifcfg:cortexa72-oe-linux:0.21:r0:cortexa72:3:fba4a512f137dced352b65244a828d6014453f4f3899c45b178a7cbcf8b5d8b2_package_qa.tgz.siginfo in paths:
    /jenkins/mjansa/build/ros/ros2-rolling-hardknott/sstate-cache
DEBUG: Defaulting to /jenkins/mjansa/build/ros/ros2-rolling-hardknott/sstate-cache/fb/a4/sstate:python3-ifcfg:cortexa72-oe-linux:0.21:r0:cortexa72:3:fba4a512f137dced352b65244a828d6014453f4f3899c45b178a7cbcf8b5d8b2_package_qa.tgz.siginfo for fb/a4/sstate:python3-ifcfg:cortexa72-oe-linux:0.21:r0:cortexa72:3:fba4a512f137dced352b65244a828d6014453f4f3899c45b178a7cbcf8b5d8b2_package_qa.tgz.siginfo
DEBUG: Testing URL file://fb/a4/sstate:python3-ifcfg:cortexa72-oe-linux:0.21:r0:cortexa72:3:fba4a512f137dced352b65244a828d6014453f4f3899c45b178a7cbcf8b5d8b2_package_qa.tgz.siginfo;downloadfilename=fb/a4/sstate:python3-ifcfg:cortexa72-oe-linux:0.21:r0:cortexa72:3:fba4a512f137dced352b65244a828d6014453f4f3899c45b178a7cbcf8b5d8b2_package_qa.tgz.siginfo
DEBUG: For url ['file', '', 'fb/a4/sstate:python3-ifcfg:cortexa72-oe-linux:0.21:r0:cortexa72:3:fba4a512f137dced352b65244a828d6014453f4f3899c45b178a7cbcf8b5d8b2_package_qa.tgz.siginfo', '', '', OrderedDict([('downloadfilename', 'fb/a4/sstate:python3-ifcfg:cortexa72-oe-linux:0.21:r0:cortexa72:3:fba4a512f137dced352b65244a828d6014453f4f3899c45b178a7cbcf8b5d8b2_package_qa.tgz.siginfo')])] comparing ['file', '', '.*', '', '', OrderedDict()] to ['file', '', '/mnt/mirror-write-ros/jansa/webos/ros-hardknott/sstate-cache/PATH', '', '', OrderedDict()]
DEBUG: For url file://fb/a4/sstate:python3-ifcfg:cortexa72-oe-linux:0.21:r0:cortexa72:3:fba4a512f137dced352b65244a828d6014453f4f3899c45b178a7cbcf8b5d8b2_package_qa.tgz.siginfo;downloadfilename=fb/a4/sstate:python3-ifcfg:cortexa72-oe-linux:0.21:r0:cortexa72:3:fba4a512f137dced352b65244a828d6014453f4f3899c45b178a7cbcf8b5d8b2_package_qa.tgz.siginfo returning file:///mnt/mirror-write-ros/jansa/webos/ros-hardknott/sstate-cache/fb/a4/sstate%3Apython3-ifcfg%3Acortexa72-oe-linux%3A0.21%3Ar0%3Acortexa72%3A3%3Afba4a512f137dced352b65244a828d6014453f4f3899c45b178a7cbcf8b5d8b2_package_qa.tgz.siginfo;downloadfilename=fb/a4/sstate:python3-ifcfg:cortexa72-oe-linux:0.21:r0:cortexa72:3:fba4a512f137dced352b65244a828d6014453f4f3899c45b178a7cbcf8b5d8b2_package_qa.tgz.siginfo
DEBUG: Trying PREMIRRORS
DEBUG: For url ['file', '', 'fb/a4/sstate:python3-ifcfg:cortexa72-oe-linux:0.21:r0:cortexa72:3:fba4a512f137dced352b65244a828d6014453f4f3899c45b178a7cbcf8b5d8b2_package_qa.tgz.siginfo', '', '', OrderedDict([('downloadfilename', 'fb/a4/sstate:python3-ifcfg:cortexa72-oe-linux:0.21:r0:cortexa72:3:fba4a512f137dced352b65244a828d6014453f4f3899c45b178a7cbcf8b5d8b2_package_qa.tgz.siginfo')])] comparing ['file', '', '.*', '', '', OrderedDict()] to ['file', '', '/mnt/mirror-write-ros/jansa/webos/ros-hardknott/sstate-cache/PATH', '', '', OrderedDict()]
DEBUG: For url file://fb/a4/sstate:python3-ifcfg:cortexa72-oe-linux:0.21:r0:cortexa72:3:fba4a512f137dced352b65244a828d6014453f4f3899c45b178a7cbcf8b5d8b2_package_qa.tgz.siginfo;downloadfilename=fb/a4/sstate:python3-ifcfg:cortexa72-oe-linux:0.21:r0:cortexa72:3:fba4a512f137dced352b65244a828d6014453f4f3899c45b178a7cbcf8b5d8b2_package_qa.tgz.siginfo returning file:///mnt/mirror-write-ros/jansa/webos/ros-hardknott/sstate-cache/fb/a4/sstate%3Apython3-ifcfg%3Acortexa72-oe-linux%3A0.21%3Ar0%3Acortexa72%3A3%3Afba4a512f137dced352b65244a828d6014453f4f3899c45b178a7cbcf8b5d8b2_package_qa.tgz.siginfo;downloadfilename=fb/a4/sstate:python3-ifcfg:cortexa72-oe-linux:0.21:r0:cortexa72:3:fba4a512f137dced352b65244a828d6014453f4f3899c45b178a7cbcf8b5d8b2_package_qa.tgz.siginfo
NOTE: Sstate package /jenkins/mjansa/build/ros/ros2-rolling-hardknott/sstate-cache/fb/a4/sstate:python3-ifcfg:cortexa72-oe-linux:0.21:r0:cortexa72:3:fba4a512f137dced352b65244a828d6014453f4f3899c45b178a7cbcf8b5d8b2_package_qa.tgz does not exist
ERROR: No suitable staging package found
DEBUG: Python function do_package_qa_setscene finished

$ ls -lah "/jenkins/mjansa/build/ros/ros2-rolling-hardknott/sstate-cache/fb/a4/sstate:python3-ifcfg:cortexa72-oe-linux:0.21:r0:cortexa72:3:fba4a512f137dced352b65244a828d6014453f4f3899c45b178a7cbcf8b5d8b2_package_qa.tgz"
lrwxrwxrwx 1 mjansa mjansa 229 Nov 27 05:41 /jenkins/mjansa/build/ros/ros2-rolling-hardknott/sstate-cache/fb/a4/sstate:python3-ifcfg:cortexa72-oe-linux:0.21:r0:cortexa72:3:fba4a512f137dced352b65244a828d6014453f4f3899c45b178a7cbcf8b5d8b2_package_qa.tgz -> ../../../../../../../../mnt/mirror-write-ros/jansa/webos/ros-hardknott/sstate-cache/fb/a4/sstate:python3-ifcfg:cortexa72-oe-linux:0.21:r0:cortexa72:3:fba4a512f137dced352b65244a828d6014453f4f3899c45b178a7cbcf8b5d8b2_package_qa.tgz



On Wed, Nov 18, 2020 at 4:36 PM Hombourger, Cedric <Cedric_Hombourger@mentor.com> wrote:
try_mirror_url() creates a symlink named as the original file to make
everything look like files specified in SRC_URI were downloaded from
their original location. The link is however created as an absolute
reference, this makes DL_DIR non-relocatable. This also causes issues
with the Isar project since it bind mounts DL_DIR to /downloads to
perform some of its build tasks in a chrooted environment (rendering
all symbolic links from DL_DIR invalid). Modify ensure_symlink() to
take an optional "relative" argument and have that function use
os.path.relpath() to produce a relative symlink.

Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com>
Cc: Chris Larson <Chris_Larson@mentor.com>
---
 lib/bb/fetch2/__init__.py | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/lib/bb/fetch2/__init__.py b/lib/bb/fetch2/__init__.py
index ca81bd52..e090cd86 100644
--- a/lib/bb/fetch2/__init__.py
+++ b/lib/bb/fetch2/__init__.py
@@ -1026,7 +1026,8 @@ def try_mirror_url(fetch, origud, ud, ld, check = False):
                     origud.method.build_mirror_data(origud, ld)
             return origud.localpath
         # Otherwise the result is a local file:// and we symlink to it
-        ensure_symlink(ud.localpath, origud.localpath)
+        ensure_symlink(ud.localpath, origud.localpath, relative=True)
+
         update_stamp(origud, ld)
         return ud.localpath

@@ -1060,7 +1061,7 @@ def try_mirror_url(fetch, origud, ud, ld, check = False):
             bb.utils.unlockfile(lf)


-def ensure_symlink(target, link_name):
+def ensure_symlink(target, link_name, relative=False):
     if not os.path.exists(link_name):
         if os.path.islink(link_name):
             # Broken symbolic link
@@ -1071,6 +1072,8 @@ def ensure_symlink(target, link_name):
         # same time, in which case we do not want the second task to
         # fail when the link has already been created by the first task.
         try:
+            if relative is True:
+                target = os.path.relpath(target, os.path.dirname(link_name))
             os.symlink(target, link_name)
         except FileExistsError:
             pass
--
2.25.1