[meta-mingw][PATCH] mingw32-common: change mingw sdk to .zip type

[meta-mingw][PATCH] mingw32-common: change mingw sdk to .zip type

[changqing.li]

From: Changqing Li <changqing.li@windriver.com>

Current sdk type is tar.xz, but for mingw sdk, since we
have symlink under the sdk folder, 7zip which used to
extract tar.xz cannot handle it, refer 7zip upstream bug:
https://sourceforge.net/p/sevenzip/discussion/45797/thread/c71d6b96/
so change to .zip type

Also tried to use option --dereference of tar which use means:
"follow symlinks; archive and dump the files they point to"
with this option, when archive it will deal with symlink to hard link,
but windows not support hard link, so drop this solution,
and use .zip for windows.

Signed-off-by: Changqing Li <changqing.li@windriver.com>
---
 conf/machine-sdk/include/mingw32-common.inc | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/conf/machine-sdk/include/mingw32-common.inc b/conf/machine-sdk/include/mingw32-common.inc
index 9011ded..dbad50a 100644
--- a/conf/machine-sdk/include/mingw32-common.inc
+++ b/conf/machine-sdk/include/mingw32-common.inc
@@ -46,3 +46,8 @@ GCCPIE_mingw32 = ""
 # wine and wineserver are required to test MinGW SDKs
 HOSTTOOLS += "${@'wine wineserver' if (bb.utils.contains_any('IMAGE_CLASSES', 'testsdk', True, False, d) or any(x in (d.getVar("BBINCLUDED") or "") for x in ["testsdk.bbclass"])) else ''}"
 
+# mingw sdk use .zip type
+SDK_TAR_TYPE = "zip"
+SDK_TAR_DEPENDS = "zip-native"
+SDK_TAR_CMD ?= "zip -r  ${SDKDEPLOYDIR}/${TOOLCHAIN_OUTPUTNAME}.${SDK_TAR_TYPE} ."
+
-- 
2.7.4

Re: [meta-mingw][PATCH] mingw32-common: change mingw sdk to .zip type

[Joshua Watt]

[-- Attachment #1: Type: text/plain, Size: 1891 bytes --]

On Wed, Jan 16, 2019, 3:37 AM <changqing.li@windriver.com wrote:

> From: Changqing Li <changqing.li@windriver.com>
>
> Current sdk type is tar.xz, but for mingw sdk, since we
> have symlink under the sdk folder, 7zip which used to
> extract tar.xz cannot handle it, refer 7zip upstream bug:
> https://sourceforge.net/p/sevenzip/discussion/45797/thread/c71d6b96/
> so change to .zip type
>
> Also tried to use option --dereference of tar which use means:
> "follow symlinks; archive and dump the files they point to"
> with this option, when archive it will deal with symlink to hard link,
> but windows not support hard link, so drop this solution,
> and use .zip for windows.
>
> Signed-off-by: Changqing Li <changqing.li@windriver.com>
> ---
>  conf/machine-sdk/include/mingw32-common.inc | 5 +++++
>  1 file changed, 5 insertions(+)
>
> diff --git a/conf/machine-sdk/include/mingw32-common.inc
> b/conf/machine-sdk/include/mingw32-common.inc
> index 9011ded..dbad50a 100644
> --- a/conf/machine-sdk/include/mingw32-common.inc
> +++ b/conf/machine-sdk/include/mingw32-common.inc
> @@ -46,3 +46,8 @@ GCCPIE_mingw32 = ""
>  # wine and wineserver are required to test MinGW SDKs
>  HOSTTOOLS += "${@'wine wineserver' if
> (bb.utils.contains_any('IMAGE_CLASSES', 'testsdk', True, False, d) or any(x
> in (d.getVar("BBINCLUDED") or "") for x in ["testsdk.bbclass"])) else ''}"
>
> +# mingw sdk use .zip type
> +SDK_TAR_TYPE = "zip"
> +SDK_TAR_DEPENDS = "zip-native"
> +SDK_TAR_CMD ?= "zip -r
> ${SDKDEPLOYDIR}/${TOOLCHAIN_OUTPUTNAME}.${SDK_TAR_TYPE} ."
> +
>

You will also need to update the QA tests, since right now they expecting a
tar file (lib/oeqa/sdkmingw/testsdk.py)

--

2.7.4
>
> --
> _______________________________________________
> yocto mailing list
> yocto@yoctoproject.org
> https://lists.yoctoproject.org/listinfo/yocto
>

[-- Attachment #2: Type: text/html, Size: 3233 bytes --]

Re: [meta-mingw][PATCH] mingw32-common: change mingw sdk to .zip type

[Burton, Ross]

Just in case you hadn't noticed, that this depends on a corresponding
patch to oe-core, which I've comments on.

Ross

On Wed, 16 Jan 2019 at 13:14, Joshua Watt <jpewhacker@gmail.com> wrote:
>
>
>
> On Wed, Jan 16, 2019, 3:37 AM <changqing.li@windriver.com wrote:
>>
>> From: Changqing Li <changqing.li@windriver.com>
>>
>> Current sdk type is tar.xz, but for mingw sdk, since we
>> have symlink under the sdk folder, 7zip which used to
>> extract tar.xz cannot handle it, refer 7zip upstream bug:
>> https://sourceforge.net/p/sevenzip/discussion/45797/thread/c71d6b96/
>> so change to .zip type
>>
>> Also tried to use option --dereference of tar which use means:
>> "follow symlinks; archive and dump the files they point to"
>> with this option, when archive it will deal with symlink to hard link,
>> but windows not support hard link, so drop this solution,
>> and use .zip for windows.
>>
>> Signed-off-by: Changqing Li <changqing.li@windriver.com>
>> ---
>>  conf/machine-sdk/include/mingw32-common.inc | 5 +++++
>>  1 file changed, 5 insertions(+)
>>
>> diff --git a/conf/machine-sdk/include/mingw32-common.inc b/conf/machine-sdk/include/mingw32-common.inc
>> index 9011ded..dbad50a 100644
>> --- a/conf/machine-sdk/include/mingw32-common.inc
>> +++ b/conf/machine-sdk/include/mingw32-common.inc
>> @@ -46,3 +46,8 @@ GCCPIE_mingw32 = ""
>>  # wine and wineserver are required to test MinGW SDKs
>>  HOSTTOOLS += "${@'wine wineserver' if (bb.utils.contains_any('IMAGE_CLASSES', 'testsdk', True, False, d) or any(x in (d.getVar("BBINCLUDED") or "") for x in ["testsdk.bbclass"])) else ''}"
>>
>> +# mingw sdk use .zip type
>> +SDK_TAR_TYPE = "zip"
>> +SDK_TAR_DEPENDS = "zip-native"
>> +SDK_TAR_CMD ?= "zip -r  ${SDKDEPLOYDIR}/${TOOLCHAIN_OUTPUTNAME}.${SDK_TAR_TYPE} ."
>> +
>
>
> You will also need to update the QA tests, since right now they expecting a tar file (lib/oeqa/sdkmingw/testsdk.py)
>
>> --
>>
>> 2.7.4
>>
>> --
>> _______________________________________________
>> yocto mailing list
>> yocto@yoctoproject.org
>> https://lists.yoctoproject.org/listinfo/yocto
>
> --
> _______________________________________________
> yocto mailing list
> yocto@yoctoproject.org
> https://lists.yoctoproject.org/listinfo/yocto

Re: [meta-mingw][PATCH] mingw32-common: change mingw sdk to .zip type

[Changqing Li]

Ross and Joshua,  thanks for your comments,  I will send a v2 later.

On 1/16/19 9:17 PM, Burton, Ross wrote:
> Just in case you hadn't noticed, that this depends on a corresponding
> patch to oe-core, which I've comments on.
>
> Ross
>
> On Wed, 16 Jan 2019 at 13:14, Joshua Watt <jpewhacker@gmail.com> wrote:
>>
>>
>> On Wed, Jan 16, 2019, 3:37 AM <changqing.li@windriver.com wrote:
>>> From: Changqing Li <changqing.li@windriver.com>
>>>
>>> Current sdk type is tar.xz, but for mingw sdk, since we
>>> have symlink under the sdk folder, 7zip which used to
>>> extract tar.xz cannot handle it, refer 7zip upstream bug:
>>> https://sourceforge.net/p/sevenzip/discussion/45797/thread/c71d6b96/
>>> so change to .zip type
>>>
>>> Also tried to use option --dereference of tar which use means:
>>> "follow symlinks; archive and dump the files they point to"
>>> with this option, when archive it will deal with symlink to hard link,
>>> but windows not support hard link, so drop this solution,
>>> and use .zip for windows.
>>>
>>> Signed-off-by: Changqing Li <changqing.li@windriver.com>
>>> ---
>>>   conf/machine-sdk/include/mingw32-common.inc | 5 +++++
>>>   1 file changed, 5 insertions(+)
>>>
>>> diff --git a/conf/machine-sdk/include/mingw32-common.inc b/conf/machine-sdk/include/mingw32-common.inc
>>> index 9011ded..dbad50a 100644
>>> --- a/conf/machine-sdk/include/mingw32-common.inc
>>> +++ b/conf/machine-sdk/include/mingw32-common.inc
>>> @@ -46,3 +46,8 @@ GCCPIE_mingw32 = ""
>>>   # wine and wineserver are required to test MinGW SDKs
>>>   HOSTTOOLS += "${@'wine wineserver' if (bb.utils.contains_any('IMAGE_CLASSES', 'testsdk', True, False, d) or any(x in (d.getVar("BBINCLUDED") or "") for x in ["testsdk.bbclass"])) else ''}"
>>>
>>> +# mingw sdk use .zip type
>>> +SDK_TAR_TYPE = "zip"
>>> +SDK_TAR_DEPENDS = "zip-native"
>>> +SDK_TAR_CMD ?= "zip -r  ${SDKDEPLOYDIR}/${TOOLCHAIN_OUTPUTNAME}.${SDK_TAR_TYPE} ."
>>> +
>>
>> You will also need to update the QA tests, since right now they expecting a tar file (lib/oeqa/sdkmingw/testsdk.py)
>>
>>> --
>>>
>>> 2.7.4
>>>
>>> --
>>> _______________________________________________
>>> yocto mailing list
>>> yocto@yoctoproject.org
>>> https://lists.yoctoproject.org/listinfo/yocto
>> --
>> _______________________________________________
>> yocto mailing list
>> yocto@yoctoproject.org
>> https://lists.yoctoproject.org/listinfo/yocto

-- 
BRs

Sandy(Li Changqing)

[meta-mingw][PATCH V2] testsdk: enhance extract sdk part

[changqing.li]

change in V2:
1. move zip part to oe-core, and if usr want to use .zip file
   set SDK_ARCHIVE_TYPE to zip in local.conf 
2. correct extract sdk function for qa test 

Changqing Li (1):
  testsdk: enhance extract sdk part

 lib/oeqa/sdkmingw/testsdk.py | 11 +++++++++--
 1 file changed, 9 insertions(+), 2 deletions(-)

-- 
2.7.4

[meta-mingw][PATCH V2] testsdk: enhance extract sdk part

[changqing.li]

From: Changqing Li <changqing.li@windriver.com>

Current sdk type is tar.xz, but for mingw sdk, since we
have symlink under the sdk folder, 7zip which used to
extract tar.xz cannot handle it, refer 7zip upstream bug:
https://sourceforge.net/p/sevenzip/discussion/45797/thread/c71d6b96/
so oe-core have add support of change SDK type to .zip, so correct
the extract function.

Signed-off-by: Changqing Li <changqing.li@windriver.com>
---
 lib/oeqa/sdkmingw/testsdk.py | 11 +++++++++--
 1 file changed, 9 insertions(+), 2 deletions(-)

diff --git a/lib/oeqa/sdkmingw/testsdk.py b/lib/oeqa/sdkmingw/testsdk.py
index 85fe3c6..173cfd9 100644
--- a/lib/oeqa/sdkmingw/testsdk.py
+++ b/lib/oeqa/sdkmingw/testsdk.py
@@ -12,7 +12,7 @@ class TestSDKMinGW(TestSDK):
         """
         Get the name of the SDK file
         """
-        return d.expand("${SDK_DEPLOY}/${TOOLCHAIN_OUTPUTNAME}.tar.xz")
+        return d.expand("${SDK_DEPLOY}/${TOOLCHAIN_OUTPUTNAME}.${SDK_ARCHIVE_TYPE}")
 
     def extract_sdk(self, tcname, sdk_dir, d):
         """
@@ -23,7 +23,14 @@ class TestSDKMinGW(TestSDK):
         try:
             # TODO: It would be nice to try and extract the SDK in Wine to make
             # sure it is well formed
-            subprocess.check_output(['tar', '-xf', tcname, '-C', sdk_dir])
+            
+            # TODO: Extract SDK according to SDK_ARCHIVE_TYPE, need to change if
+            # oe-core support other types.
+            if d.getVar("SDK_ARCHIVE_TYPE") == "zip":
+                subprocess.check_output(['unzip', '-d', sdk_dir, tcname])
+            else:
+                subprocess.check_output(['tar', '-xf', tcname, '-C', sdk_dir])
+
         except subprocess.CalledProcessError as e:
             bb.fatal("Couldn't install the SDK:\n%s" % e.output.decode("utf-8"))
 
-- 
2.7.4

Re: [meta-mingw][PATCH V2] testsdk: enhance extract sdk part

[Joshua Watt]

Just FYI, I haven't forgot about this patch. I want to wait until the
dependent OE-core patch is at least in the master-next branch before I
apply this to the MinGW master-next branch. If you want to give me a
ping when the OE-core patch shows up in master-next, it would be very
helpful.

Thanks

On Thu, 2019-01-17 at 17:49 +0800, changqing.li@windriver.com wrote:
> From: Changqing Li <changqing.li@windriver.com>
> 
> Current sdk type is tar.xz, but for mingw sdk, since we
> have symlink under the sdk folder, 7zip which used to
> extract tar.xz cannot handle it, refer 7zip upstream bug:
> https://sourceforge.net/p/sevenzip/discussion/45797/thread/c71d6b96/
> so oe-core have add support of change SDK type to .zip, so correct
> the extract function.
> 
> Signed-off-by: Changqing Li <changqing.li@windriver.com>
> ---
>  lib/oeqa/sdkmingw/testsdk.py | 11 +++++++++--
>  1 file changed, 9 insertions(+), 2 deletions(-)
> 
> diff --git a/lib/oeqa/sdkmingw/testsdk.py
> b/lib/oeqa/sdkmingw/testsdk.py
> index 85fe3c6..173cfd9 100644
> --- a/lib/oeqa/sdkmingw/testsdk.py
> +++ b/lib/oeqa/sdkmingw/testsdk.py
> @@ -12,7 +12,7 @@ class TestSDKMinGW(TestSDK):
>          """
>          Get the name of the SDK file
>          """
> -        return
> d.expand("${SDK_DEPLOY}/${TOOLCHAIN_OUTPUTNAME}.tar.xz")
> +        return
> d.expand("${SDK_DEPLOY}/${TOOLCHAIN_OUTPUTNAME}.${SDK_ARCHIVE_TYPE}")
>  
>      def extract_sdk(self, tcname, sdk_dir, d):
>          """
> @@ -23,7 +23,14 @@ class TestSDKMinGW(TestSDK):
>          try:
>              # TODO: It would be nice to try and extract the SDK in
> Wine to make
>              # sure it is well formed
> -            subprocess.check_output(['tar', '-xf', tcname, '-C',
> sdk_dir])
> +            
> +            # TODO: Extract SDK according to SDK_ARCHIVE_TYPE, need
> to change if
> +            # oe-core support other types.
> +            if d.getVar("SDK_ARCHIVE_TYPE") == "zip":
> +                subprocess.check_output(['unzip', '-d', sdk_dir,
> tcname])
> +            else:
> +                subprocess.check_output(['tar', '-xf', tcname, '-C', 
> sdk_dir])
> +
>          except subprocess.CalledProcessError as e:
>              bb.fatal("Couldn't install the SDK:\n%s" %
> e.output.decode("utf-8"))
>  
> -- 
> 2.7.4
> 
-- 
Joshua Watt <JPEWhacker@gmail.com>

Re: [meta-mingw][PATCH V2] testsdk: enhance extract sdk part

[Changqing Li]

On 1/21/19 10:47 PM, Joshua Watt wrote:
> Just FYI, I haven't forgot about this patch. I want to wait until the
> dependent OE-core patch is at least in the master-next branch before I
> apply this to the MinGW master-next branch. If you want to give me a
> ping when the OE-core patch shows up in master-next, it would be very
> helpful.
>
> Thanks

Yes, I will do it.

>
> On Thu, 2019-01-17 at 17:49 +0800, changqing.li@windriver.com wrote:
>> From: Changqing Li <changqing.li@windriver.com>
>>
>> Current sdk type is tar.xz, but for mingw sdk, since we
>> have symlink under the sdk folder, 7zip which used to
>> extract tar.xz cannot handle it, refer 7zip upstream bug:
>> https://sourceforge.net/p/sevenzip/discussion/45797/thread/c71d6b96/
>> so oe-core have add support of change SDK type to .zip, so correct
>> the extract function.
>>
>> Signed-off-by: Changqing Li <changqing.li@windriver.com>
>> ---
>>   lib/oeqa/sdkmingw/testsdk.py | 11 +++++++++--
>>   1 file changed, 9 insertions(+), 2 deletions(-)
>>
>> diff --git a/lib/oeqa/sdkmingw/testsdk.py
>> b/lib/oeqa/sdkmingw/testsdk.py
>> index 85fe3c6..173cfd9 100644
>> --- a/lib/oeqa/sdkmingw/testsdk.py
>> +++ b/lib/oeqa/sdkmingw/testsdk.py
>> @@ -12,7 +12,7 @@ class TestSDKMinGW(TestSDK):
>>           """
>>           Get the name of the SDK file
>>           """
>> -        return
>> d.expand("${SDK_DEPLOY}/${TOOLCHAIN_OUTPUTNAME}.tar.xz")
>> +        return
>> d.expand("${SDK_DEPLOY}/${TOOLCHAIN_OUTPUTNAME}.${SDK_ARCHIVE_TYPE}")
>>   
>>       def extract_sdk(self, tcname, sdk_dir, d):
>>           """
>> @@ -23,7 +23,14 @@ class TestSDKMinGW(TestSDK):
>>           try:
>>               # TODO: It would be nice to try and extract the SDK in
>> Wine to make
>>               # sure it is well formed
>> -            subprocess.check_output(['tar', '-xf', tcname, '-C',
>> sdk_dir])
>> +
>> +            # TODO: Extract SDK according to SDK_ARCHIVE_TYPE, need
>> to change if
>> +            # oe-core support other types.
>> +            if d.getVar("SDK_ARCHIVE_TYPE") == "zip":
>> +                subprocess.check_output(['unzip', '-d', sdk_dir,
>> tcname])
>> +            else:
>> +                subprocess.check_output(['tar', '-xf', tcname, '-C',
>> sdk_dir])
>> +
>>           except subprocess.CalledProcessError as e:
>>               bb.fatal("Couldn't install the SDK:\n%s" %
>> e.output.decode("utf-8"))
>>   
>> -- 
>> 2.7.4
>>
-- 
BRs

Sandy(Li Changqing)

Re: [meta-mingw][PATCH V2] testsdk: enhance extract sdk part

[Changqing Li]

Hi, Joshua

OE-core patch have merge to master: 
https://git.openembedded.org/openembedded-core/commit/?id=57a33048a89a422cfdc986d3489c67b2d297e1e7

please help to merge this patch, thanks.

On 1/21/19 10:47 PM, Joshua Watt wrote:
> Just FYI, I haven't forgot about this patch. I want to wait until the
> dependent OE-core patch is at least in the master-next branch before I
> apply this to the MinGW master-next branch. If you want to give me a
> ping when the OE-core patch shows up in master-next, it would be very
> helpful.
>
> Thanks
>
> On Thu, 2019-01-17 at 17:49 +0800, changqing.li@windriver.com wrote:
>> From: Changqing Li <changqing.li@windriver.com>
>>
>> Current sdk type is tar.xz, but for mingw sdk, since we
>> have symlink under the sdk folder, 7zip which used to
>> extract tar.xz cannot handle it, refer 7zip upstream bug:
>> https://sourceforge.net/p/sevenzip/discussion/45797/thread/c71d6b96/
>> so oe-core have add support of change SDK type to .zip, so correct
>> the extract function.
>>
>> Signed-off-by: Changqing Li <changqing.li@windriver.com>
>> ---
>>   lib/oeqa/sdkmingw/testsdk.py | 11 +++++++++--
>>   1 file changed, 9 insertions(+), 2 deletions(-)
>>
>> diff --git a/lib/oeqa/sdkmingw/testsdk.py
>> b/lib/oeqa/sdkmingw/testsdk.py
>> index 85fe3c6..173cfd9 100644
>> --- a/lib/oeqa/sdkmingw/testsdk.py
>> +++ b/lib/oeqa/sdkmingw/testsdk.py
>> @@ -12,7 +12,7 @@ class TestSDKMinGW(TestSDK):
>>           """
>>           Get the name of the SDK file
>>           """
>> -        return
>> d.expand("${SDK_DEPLOY}/${TOOLCHAIN_OUTPUTNAME}.tar.xz")
>> +        return
>> d.expand("${SDK_DEPLOY}/${TOOLCHAIN_OUTPUTNAME}.${SDK_ARCHIVE_TYPE}")
>>   
>>       def extract_sdk(self, tcname, sdk_dir, d):
>>           """
>> @@ -23,7 +23,14 @@ class TestSDKMinGW(TestSDK):
>>           try:
>>               # TODO: It would be nice to try and extract the SDK in
>> Wine to make
>>               # sure it is well formed
>> -            subprocess.check_output(['tar', '-xf', tcname, '-C',
>> sdk_dir])
>> +
>> +            # TODO: Extract SDK according to SDK_ARCHIVE_TYPE, need
>> to change if
>> +            # oe-core support other types.
>> +            if d.getVar("SDK_ARCHIVE_TYPE") == "zip":
>> +                subprocess.check_output(['unzip', '-d', sdk_dir,
>> tcname])
>> +            else:
>> +                subprocess.check_output(['tar', '-xf', tcname, '-C',
>> sdk_dir])
>> +
>>           except subprocess.CalledProcessError as e:
>>               bb.fatal("Couldn't install the SDK:\n%s" %
>> e.output.decode("utf-8"))
>>   
>> -- 
>> 2.7.4
>>
-- 
BRs

Sandy(Li Changqing)