[Buildroot] [PATCH v2 0/3] package/ca-certificates improvements

[Buildroot] [PATCH v2 0/3] package/ca-certificates improvements

[Martin Bark]

All,

This patch set includes the following improvements to ca-certificates:

1. fix a bug meaning one of the hashes in ${TARGET_DIR}/etc/ssl/certs was wrong
2. fix rebuilds
3. make the contents of ${TARGET_DIR}/etc/ssl/certs/ca-certiicates.crt reproducible.

The result of these patches means ${TARGET_DIR}/etc/ssl/certs now exactly matches Ubuntu 18.04 and Debian sid which both use ca-certiicates 20180409 too

Thanks

Martin

Martin Bark (3):
  package/ca-certificates: don't hash certificates.crt
  package/ca-certificates: fix rebuilds
  package/ca-certificates: create ca-certificates.crt reproducibly

 package/ca-certificates/ca-certificates.mk | 12 ++++++++----
 1 file changed, 8 insertions(+), 4 deletions(-)

--
2.17.1

[Buildroot] [PATCH v2 1/3] package/ca-certificates: don't hash certificates.crt

[Martin Bark]

Currently c_rehash mistakenly hashes the certificates bundle
certificates.crt resulting in ${TAGET_DIR}/etc/ssl/certs/128805a3.0
incorrectly linking to ca-certificates.crt when it should be linked to
EE_Certification_Centre_Root_CA_2.pem

To fix this install certificates.crt to /etc/ssl/certs after we run
c_rehash to prevent it getting hashed by mistake.

Note: $(TARGET_DIR)/etc/ssl/certs/ is already removed during install so
this fix also works for rebuilds.

Signed-off-by: Martin Bark <martin@barkynet.com>

---
Changes v1 -> v2:
 - reword commit message and improve creation of ca-certificates.crt
   (Suggested By Peter Korsgaard)
---
 package/ca-certificates/ca-certificates.mk | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/package/ca-certificates/ca-certificates.mk b/package/ca-certificates/ca-certificates.mk
index c19d37788b..9685d0e6f0 100644
--- a/package/ca-certificates/ca-certificates.mk
+++ b/package/ca-certificates/ca-certificates.mk
@@ -33,11 +33,15 @@ define CA_CERTIFICATES_INSTALL_TARGET_CMDS
 	cd $(TARGET_DIR) ;\
 	for i in `find usr/share/ca-certificates -name "*.crt"` ; do \
 		ln -sf ../../../$$i etc/ssl/certs/`basename $${i} .crt`.pem ;\
-		cat $$i >>etc/ssl/certs/ca-certificates.crt ;\
-	done
+		cat $$i ;\
+	done >$(@D)/ca-certificates.crt
 
 	# Create symlinks to the certificates by their hash values
 	$(HOST_DIR)/bin/c_rehash $(TARGET_DIR)/etc/ssl/certs
+
+	# Install the certificates bundle
+	$(INSTALL) -D -m 644 $(@D)/ca-certificates.crt \
+		$(TARGET_DIR)/etc/ssl/certs/ca-certificates.crt
 endef
 
 $(eval $(generic-package))
-- 
2.17.1

[Buildroot] [PATCH v2 2/3] package/ca-certificates: fix rebuilds

[Martin Bark]

Rebuilding ca-certificates using make ca-certificates-rebuild
caused duplicate certificates to be installed in the target.  Fix this
by cleaning the build directory before building the package.

Signed-off-by: Martin Bark <martin@barkynet.com>
---
 package/ca-certificates/ca-certificates.mk | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/package/ca-certificates/ca-certificates.mk b/package/ca-certificates/ca-certificates.mk
index 9685d0e6f0..1df9543e03 100644
--- a/package/ca-certificates/ca-certificates.mk
+++ b/package/ca-certificates/ca-certificates.mk
@@ -16,7 +16,7 @@ CA_CERTIFICATES_LICENSE = GPL-2.0+ (script), MPL-2.0 (data)
 CA_CERTIFICATES_LICENSE_FILES = debian/copyright
 
 define CA_CERTIFICATES_BUILD_CMDS
-	$(TARGET_MAKE_ENV) $(MAKE) -C $(@D) all
+	$(TARGET_MAKE_ENV) $(MAKE) -C $(@D) clean all
 endef
 
 define CA_CERTIFICATES_INSTALL_TARGET_CMDS
-- 
2.17.1

[Buildroot] [PATCH v2 3/3] package/ca-certificates: create ca-certificates.crt reproducibly

[Martin Bark]

Sort the certificates into alphabetical order so the contents of
ca-certificates.crt can be built reproducibly.

Note: The certificates are sorted uppercase then lowercase filenames
so the contents of ca-certificates.crt matches the source debian package.

Signed-off-by: Martin Bark <martin@barkynet.com>
---
 package/ca-certificates/ca-certificates.mk | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/package/ca-certificates/ca-certificates.mk b/package/ca-certificates/ca-certificates.mk
index 1df9543e03..5c259aa3ab 100644
--- a/package/ca-certificates/ca-certificates.mk
+++ b/package/ca-certificates/ca-certificates.mk
@@ -31,7 +31,7 @@ define CA_CERTIFICATES_INSTALL_TARGET_CMDS
 	# Create symlinks to certificates under /etc/ssl/certs
 	# and generate the bundle
 	cd $(TARGET_DIR) ;\
-	for i in `find usr/share/ca-certificates -name "*.crt"` ; do \
+	for i in `find usr/share/ca-certificates -name "*.crt" | LC_COLLATE=C sort` ; do \
 		ln -sf ../../../$$i etc/ssl/certs/`basename $${i} .crt`.pem ;\
 		cat $$i ;\
 	done >$(@D)/ca-certificates.crt
-- 
2.17.1

[Buildroot] [PATCH v2 1/3] package/ca-certificates: don't hash certificates.crt

[Thomas Petazzoni]

Hello,

On Sat, 16 Jun 2018 23:05:59 +0100, Martin Bark wrote:
> Currently c_rehash mistakenly hashes the certificates bundle
> certificates.crt resulting in ${TAGET_DIR}/etc/ssl/certs/128805a3.0
> incorrectly linking to ca-certificates.crt when it should be linked to
> EE_Certification_Centre_Root_CA_2.pem

I can't reproduce this issue here:

output/target$ ls -l etc/ssl/certs/128805a3.0 
lrwxrwxrwx 1 thomas thomas 35 Jun 17 20:58 etc/ssl/certs/128805a3.0 -> EE_Certification_Centre_Root_CA.pem

Also, during the review, you said that you noticed this problem when
comparing the /etc/ssl/certs generated by Buildroot with the one
available in Ubuntu.

On Fedora, it looks just like this:

$ ls -l /etc/ssl/certs/
total 4
lrwxrwxrwx 1 root root   49 May 18 13:21 ca-bundle.crt -> /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem
lrwxrwxrwx 1 root root   55 May 18 13:21 ca-bundle.trust.crt -> /etc/pki/ca-trust/extracted/openssl/ca-bundle.trust.crt
-rw-r--r-- 1 root root 2516 Apr  3 13:18 Makefile

So it really doesn't look like what Buildroot produces. But I'm not
familiar at all with those certificates.

Peter, since you've reviewed the previous iteration of this patch
series, perhaps you can take care of merging it ?

Thanks!

Thomas Petazzoni
-- 
Thomas Petazzoni, CTO, Bootlin (formerly Free Electrons)
Embedded Linux and Kernel engineering
https://bootlin.com

[Buildroot] [PATCH v2 1/3] package/ca-certificates: don't hash certificates.crt

[Martin Bark]

Thomas,

On 17 June 2018 at 20:25, Thomas Petazzoni <thomas.petazzoni@bootlin.com> wrote:
> Hello,
>
> On Sat, 16 Jun 2018 23:05:59 +0100, Martin Bark wrote:
>> Currently c_rehash mistakenly hashes the certificates bundle
>> certificates.crt resulting in ${TAGET_DIR}/etc/ssl/certs/128805a3.0
>> incorrectly linking to ca-certificates.crt when it should be linked to
>> EE_Certification_Centre_Root_CA_2.pem
>
> I can't reproduce this issue here:
>
> output/target$ ls -l etc/ssl/certs/128805a3.0
> lrwxrwxrwx 1 thomas thomas 35 Jun 17 20:58 etc/ssl/certs/128805a3.0 -> EE_Certification_Centre_Root_CA.pem

Did you check other hashes under etc/ssl/certs/ ? Check for any hashes
that link to ca-certificates.crt. I suspect you have a different hash
pointing to ca-certificates.

>
> Also, during the review, you said that you noticed this problem when
> comparing the /etc/ssl/certs generated by Buildroot with the one
> available in Ubuntu.

The ca-certificates package in Buildroot is based on the Debian
package.  Buildroot currently uses version 20180409 which is the same
as currently used by Ubuntu 18.04 and Debian sid hence the contents of
etc/ssl/certs should match.

>
> On Fedora, it looks just like this:
>
> $ ls -l /etc/ssl/certs/
> total 4
> lrwxrwxrwx 1 root root   49 May 18 13:21 ca-bundle.crt -> /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem
> lrwxrwxrwx 1 root root   55 May 18 13:21 ca-bundle.trust.crt -> /etc/pki/ca-trust/extracted/openssl/ca-bundle.trust.crt
> -rw-r--r-- 1 root root 2516 Apr  3 13:18 Makefile
>
> So it really doesn't look like what Buildroot produces. But I'm not
> familiar at all with those certificates.

The Red Hat based distros and Arch Linux organise their certificates
slightly differently.  You will not be able to do an exact comparison
but in the end they achieve the same results as Debian, Ubuntu and
Buildroot.

Thanks

Martin

>
> Peter, since you've reviewed the previous iteration of this patch
> series, perhaps you can take care of merging it ?
>
> Thanks!
>
> Thomas Petazzoni
> --
> Thomas Petazzoni, CTO, Bootlin (formerly Free Electrons)
> Embedded Linux and Kernel engineering
> https://bootlin.com

[Buildroot] [PATCH v2 1/3] package/ca-certificates: don't hash certificates.crt

[Thomas Petazzoni]

Hello,

On Mon, 18 Jun 2018 10:51:34 +0100, Martin Bark wrote:

> > output/target$ ls -l etc/ssl/certs/128805a3.0
> > lrwxrwxrwx 1 thomas thomas 35 Jun 17 20:58 etc/ssl/certs/128805a3.0 -> EE_Certification_Centre_Root_CA.pem  
> 
> Did you check other hashes under etc/ssl/certs/ ? Check for any hashes
> that link to ca-certificates.crt. I suspect you have a different hash
> pointing to ca-certificates.

There are no files in /etc/ssl/certs that are symlinks to
ca-certificates.crt:

output/target$ ls -l etc/ssl/certs/| grep ca-certificates.crt
-rw-r--r-- 1 thomas thomas 207436 Jun 18 15:30 ca-certificates.crt

Best regards,

Thomas
-- 
Thomas Petazzoni, CTO, Bootlin (formerly Free Electrons)
Embedded Linux and Kernel engineering
https://bootlin.com

[Buildroot] [PATCH v2 1/3] package/ca-certificates: don't hash certificates.crt

[Martin Bark]

Thomas,

On 18 June 2018 at 15:39, Thomas Petazzoni <thomas.petazzoni@bootlin.com> wrote:
> Hello,
>
> On Mon, 18 Jun 2018 10:51:34 +0100, Martin Bark wrote:
>
>> > output/target$ ls -l etc/ssl/certs/128805a3.0
>> > lrwxrwxrwx 1 thomas thomas 35 Jun 17 20:58 etc/ssl/certs/128805a3.0 -> EE_Certification_Centre_Root_CA.pem
>>
>> Did you check other hashes under etc/ssl/certs/ ? Check for any hashes
>> that link to ca-certificates.crt. I suspect you have a different hash
>> pointing to ca-certificates.
>
> There are no files in /etc/ssl/certs that are symlinks to
> ca-certificates.crt:
>
> output/target$ ls -l etc/ssl/certs/| grep ca-certificates.crt
> -rw-r--r-- 1 thomas thomas 207436 Jun 18 15:30 ca-certificates.crt

I did some more testing and found sometimes by chance it does work,
however, i was able to reproduce the issue as follows

 docker run -it --rm buildroot/base:20180318.1724 bash
 git clone git://git.busybox.net/buildroot
 cd buildroot
 make olddefconfig
 make ca-certificates

you will see an output like this at the end of the build

 # Create symlinks to the certificates by their hash values
 /home/br-user/buildroot/output/host/bin/c_rehash
/home/br-user/buildroot/output/target/etc/ssl/certs
 Doing /home/br-user/buildroot/output/target/etc/ssl/certs
 WARNING: Skipping duplicate certificate OpenTrust_Root_CA_G1.pem

and one the the hashes will be wrong

 $ ls -l output/target/etc/ssl/certs/| grep ca-certificates.crt
 lrwxrwxrwx 1 br-user br-user     19 Jun 21 21:52 87229d21.0 ->
ca-certificates.crt

the incorrect hash is the one mentioned in the warning

 $ ls -l /etc/ssl/certs/87229d21.0
 lrwxrwxrwx 1 root root 24 Mar 18 16:30 /etc/ssl/certs/87229d21.0 ->
OpenTrust_Root_CA_G1.pem

The key issue is the

  WARNING: Skipping duplicate certificate xxxxx

where xxxx is the name of one of the files under etc/ssl/certs.
Sometimes it's ca-certificates.crt and hence you don't see any issue
(as you found).  Sometime it's one of the CA files which is the bug
i'm trying to fix.  You can run c_rehash directly

 ./output/host/bin/c_rehash ./output/target/etc/ssl/certs

and you will see the warning.  If you rm
output/target/etc/ssl/certs/ca-certificates.crt and try again then the
warning will go.

I had not released quite how random the bug.  It certainly is not
always EE_Certification_Centre_Root_CA.pem, it can be any CA that goes
wrong.  The commit message could be changed to make it clearer the bug
does not allows happen to EE_Certification_Centre_Root_CA.pem but i'm
100% sure this is a bug that needs fixing.

Thanks

Martin

>
> Best regards,
>
> Thomas
> --
> Thomas Petazzoni, CTO, Bootlin (formerly Free Electrons)
> Embedded Linux and Kernel engineering
> https://bootlin.com

[Buildroot] [PATCH v2 1/3] package/ca-certificates: don't hash certificates.crt

[Arnout Vandecappelle]

 Hi Martin,

 Sorry for the late reply,

On 21/06/2018 23:04, Martin Bark wrote:
> Thomas,
> 
> On 18 June 2018 at 15:39, Thomas Petazzoni <thomas.petazzoni@bootlin.com> wrote:
>> Hello,
>>
>> On Mon, 18 Jun 2018 10:51:34 +0100, Martin Bark wrote:
>>
>>>> output/target$ ls -l etc/ssl/certs/128805a3.0
>>>> lrwxrwxrwx 1 thomas thomas 35 Jun 17 20:58 etc/ssl/certs/128805a3.0 -> EE_Certification_Centre_Root_CA.pem
>>>
>>> Did you check other hashes under etc/ssl/certs/ ? Check for any hashes
>>> that link to ca-certificates.crt. I suspect you have a different hash
>>> pointing to ca-certificates.
>>
>> There are no files in /etc/ssl/certs that are symlinks to
>> ca-certificates.crt:
>>
>> output/target$ ls -l etc/ssl/certs/| grep ca-certificates.crt
>> -rw-r--r-- 1 thomas thomas 207436 Jun 18 15:30 ca-certificates.crt
> 
> I did some more testing and found sometimes by chance it does work,
> however, i was able to reproduce the issue as follows
> 
>  docker run -it --rm buildroot/base:20180318.1724 bash
>  git clone git://git.busybox.net/buildroot
>  cd buildroot
>  make olddefconfig
>  make ca-certificates
> 
> you will see an output like this at the end of the build
> 
>  # Create symlinks to the certificates by their hash values
>  /home/br-user/buildroot/output/host/bin/c_rehash
> /home/br-user/buildroot/output/target/etc/ssl/certs
>  Doing /home/br-user/buildroot/output/target/etc/ssl/certs
>  WARNING: Skipping duplicate certificate OpenTrust_Root_CA_G1.pem
> 
> and one the the hashes will be wrong
> 
>  $ ls -l output/target/etc/ssl/certs/| grep ca-certificates.crt
>  lrwxrwxrwx 1 br-user br-user     19 Jun 21 21:52 87229d21.0 ->
> ca-certificates.crt
> 
> the incorrect hash is the one mentioned in the warning

 So, if I understand correctly, what happens is this:

1. certificates get installed in /etc/ssl/certs.

2. All the certificates are bundled into a ca-certificates.crt file.

3. c_rehash is run. It looks at each certificate, calculates the hash, and
creates a symlink from that hash to the certificate.

 The problem is that if ca-certificates.crt exists already, c_rehash will take
some random certificate from it and create a symlink to ca-certificates.crt
instead of to the real certificate file. But depending on the order of
evaluation of the different certificate files, it may actually make the symlink
point to the real certificate.

 What you propose looks like the good solution for it. Except in case of
rebuild, so you should remove /etc/ssl/ca-certificates.crt before running
c_rehash. That would also remove the need for patch 2 I think.

 Could you check if I'm correct, and if so, resubmit the series with:

- the additional rm -f;
- a commit message that includes the explanation above;
- drops patch 2.

 If patch 2 really is needed, it needs a better explanation.


 Regards,
 Arnout

> 
>  $ ls -l /etc/ssl/certs/87229d21.0
>  lrwxrwxrwx 1 root root 24 Mar 18 16:30 /etc/ssl/certs/87229d21.0 ->
> OpenTrust_Root_CA_G1.pem
> 
> The key issue is the
> 
>   WARNING: Skipping duplicate certificate xxxxx
> 
> where xxxx is the name of one of the files under etc/ssl/certs.
> Sometimes it's ca-certificates.crt and hence you don't see any issue
> (as you found).  Sometime it's one of the CA files which is the bug
> i'm trying to fix.  You can run c_rehash directly
> 
>  ./output/host/bin/c_rehash ./output/target/etc/ssl/certs
> 
> and you will see the warning.  If you rm
> output/target/etc/ssl/certs/ca-certificates.crt and try again then the
> warning will go.
> 
> I had not released quite how random the bug.  It certainly is not
> always EE_Certification_Centre_Root_CA.pem, it can be any CA that goes
> wrong.  The commit message could be changed to make it clearer the bug
> does not allows happen to EE_Certification_Centre_Root_CA.pem but i'm
> 100% sure this is a bug that needs fixing.
> 
> Thanks
> 
> Martin
> 
>>
>> Best regards,
>>
>> Thomas
>> --
>> Thomas Petazzoni, CTO, Bootlin (formerly Free Electrons)
>> Embedded Linux and Kernel engineering
>> https://bootlin.com
> _______________________________________________
> buildroot mailing list
> buildroot at busybox.net
> http://lists.busybox.net/mailman/listinfo/buildroot
> 

[Buildroot] [PATCH v2 1/3] package/ca-certificates: don't hash certificates.crt

[Arnout Vandecappelle]

On 21/10/2018 14:15, Arnout Vandecappelle wrote:
>  Hi Martin,
>
>  Sorry for the late reply,
>
> On 21/06/2018 23:04, Martin Bark wrote:
>> Thomas,
>>
>> On 18 June 2018 at 15:39, Thomas Petazzoni <thomas.petazzoni@bootlin.com> wrote:
>>> Hello,
>>>
>>> On Mon, 18 Jun 2018 10:51:34 +0100, Martin Bark wrote:
>>>
>>>>> output/target$ ls -l etc/ssl/certs/128805a3.0
>>>>> lrwxrwxrwx 1 thomas thomas 35 Jun 17 20:58 etc/ssl/certs/128805a3.0 -> EE_Certification_Centre_Root_CA.pem
>>>> Did you check other hashes under etc/ssl/certs/ ? Check for any hashes
>>>> that link to ca-certificates.crt. I suspect you have a different hash
>>>> pointing to ca-certificates.
>>> There are no files in /etc/ssl/certs that are symlinks to
>>> ca-certificates.crt:
>>>
>>> output/target$ ls -l etc/ssl/certs/| grep ca-certificates.crt
>>> -rw-r--r-- 1 thomas thomas 207436 Jun 18 15:30 ca-certificates.crt
>> I did some more testing and found sometimes by chance it does work,
>> however, i was able to reproduce the issue as follows
>>
>>  docker run -it --rm buildroot/base:20180318.1724 bash
>>  git clone git://git.busybox.net/buildroot
>>  cd buildroot
>>  make olddefconfig
>>  make ca-certificates
>>
>> you will see an output like this at the end of the build
>>
>>  # Create symlinks to the certificates by their hash values
>>  /home/br-user/buildroot/output/host/bin/c_rehash
>> /home/br-user/buildroot/output/target/etc/ssl/certs
>>  Doing /home/br-user/buildroot/output/target/etc/ssl/certs
>>  WARNING: Skipping duplicate certificate OpenTrust_Root_CA_G1.pem
>>
>> and one the the hashes will be wrong
>>
>>  $ ls -l output/target/etc/ssl/certs/| grep ca-certificates.crt
>>  lrwxrwxrwx 1 br-user br-user     19 Jun 21 21:52 87229d21.0 ->
>> ca-certificates.crt
>>
>> the incorrect hash is the one mentioned in the warning
>  So, if I understand correctly, what happens is this:
>
> 1. certificates get installed in /etc/ssl/certs.
>
> 2. All the certificates are bundled into a ca-certificates.crt file.
>
> 3. c_rehash is run. It looks at each certificate, calculates the hash, and
> creates a symlink from that hash to the certificate.
>
>  The problem is that if ca-certificates.crt exists already, c_rehash will take
> some random certificate from it and create a symlink to ca-certificates.crt
> instead of to the real certificate file. But depending on the order of
> evaluation of the different certificate files, it may actually make the symlink
> point to the real certificate.
>
>  What you propose looks like the good solution for it. Except in case of
> rebuild, so you should remove /etc/ssl/ca-certificates.crt before running
> c_rehash. That would also remove the need for patch 2 I think.
>
>  Could you check if I'm correct, and if so, resubmit the series with:
>
> - the additional rm -f;
> - a commit message that includes the explanation above;
> - drops patch 2.
>
>  If patch 2 really is needed, it needs a better explanation.

?Investigating a bit more turns out that I was rather wrong. The rm -f is
already there, as correctly noted in your commit message. And patch 2 really is
needed, it fixes a completely different set of duplicates.


?So I've extended the commit message a little and applied to master, thanks.


?Regards,
?Arnout


>
>
>  Regards,
>  Arnout
>
>>  $ ls -l /etc/ssl/certs/87229d21.0
>>  lrwxrwxrwx 1 root root 24 Mar 18 16:30 /etc/ssl/certs/87229d21.0 ->
>> OpenTrust_Root_CA_G1.pem
>>
>> The key issue is the
>>
>>   WARNING: Skipping duplicate certificate xxxxx
>>
>> where xxxx is the name of one of the files under etc/ssl/certs.
>> Sometimes it's ca-certificates.crt and hence you don't see any issue
>> (as you found).  Sometime it's one of the CA files which is the bug
>> i'm trying to fix.  You can run c_rehash directly
>>
>>  ./output/host/bin/c_rehash ./output/target/etc/ssl/certs
>>
>> and you will see the warning.  If you rm
>> output/target/etc/ssl/certs/ca-certificates.crt and try again then the
>> warning will go.
>>
>> I had not released quite how random the bug.  It certainly is not
>> always EE_Certification_Centre_Root_CA.pem, it can be any CA that goes
>> wrong.  The commit message could be changed to make it clearer the bug
>> does not allows happen to EE_Certification_Centre_Root_CA.pem but i'm
>> 100% sure this is a bug that needs fixing.
>>
>> Thanks
>>
>> Martin
>>
>>> Best regards,
>>>
>>> Thomas
>>> --
>>> Thomas Petazzoni, CTO, Bootlin (formerly Free Electrons)
>>> Embedded Linux and Kernel engineering
>>> https://bootlin.com
>> _______________________________________________
>> buildroot mailing list
>> buildroot at busybox.net
>> http://lists.busybox.net/mailman/listinfo/buildroot
>>

[Buildroot] [PATCH v2 1/3] package/ca-certificates: don't hash certificates.crt

[Martin Bark]

Arnout,

On Sun, 21 Oct 2018 at 14:38, Arnout Vandecappelle <arnout@mind.be> wrote:
>
>
> On 21/10/2018 14:15, Arnout Vandecappelle wrote:
> >  Hi Martin,
> >
> >  Sorry for the late reply,
> >
> > On 21/06/2018 23:04, Martin Bark wrote:
> >> Thomas,
> >>
> >> On 18 June 2018 at 15:39, Thomas Petazzoni <thomas.petazzoni@bootlin.com> wrote:
> >>> Hello,
> >>>
> >>> On Mon, 18 Jun 2018 10:51:34 +0100, Martin Bark wrote:
> >>>
> >>>>> output/target$ ls -l etc/ssl/certs/128805a3.0
> >>>>> lrwxrwxrwx 1 thomas thomas 35 Jun 17 20:58 etc/ssl/certs/128805a3.0 -> EE_Certification_Centre_Root_CA.pem
> >>>> Did you check other hashes under etc/ssl/certs/ ? Check for any hashes
> >>>> that link to ca-certificates.crt. I suspect you have a different hash
> >>>> pointing to ca-certificates.
> >>> There are no files in /etc/ssl/certs that are symlinks to
> >>> ca-certificates.crt:
> >>>
> >>> output/target$ ls -l etc/ssl/certs/| grep ca-certificates.crt
> >>> -rw-r--r-- 1 thomas thomas 207436 Jun 18 15:30 ca-certificates.crt
> >> I did some more testing and found sometimes by chance it does work,
> >> however, i was able to reproduce the issue as follows
> >>
> >>  docker run -it --rm buildroot/base:20180318.1724 bash
> >>  git clone git://git.busybox.net/buildroot
> >>  cd buildroot
> >>  make olddefconfig
> >>  make ca-certificates
> >>
> >> you will see an output like this at the end of the build
> >>
> >>  # Create symlinks to the certificates by their hash values
> >>  /home/br-user/buildroot/output/host/bin/c_rehash
> >> /home/br-user/buildroot/output/target/etc/ssl/certs
> >>  Doing /home/br-user/buildroot/output/target/etc/ssl/certs
> >>  WARNING: Skipping duplicate certificate OpenTrust_Root_CA_G1.pem
> >>
> >> and one the the hashes will be wrong
> >>
> >>  $ ls -l output/target/etc/ssl/certs/| grep ca-certificates.crt
> >>  lrwxrwxrwx 1 br-user br-user     19 Jun 21 21:52 87229d21.0 ->
> >> ca-certificates.crt
> >>
> >> the incorrect hash is the one mentioned in the warning
> >  So, if I understand correctly, what happens is this:
> >
> > 1. certificates get installed in /etc/ssl/certs.
> >
> > 2. All the certificates are bundled into a ca-certificates.crt file.
> >
> > 3. c_rehash is run. It looks at each certificate, calculates the hash, and
> > creates a symlink from that hash to the certificate.
> >
> >  The problem is that if ca-certificates.crt exists already, c_rehash will take
> > some random certificate from it and create a symlink to ca-certificates.crt
> > instead of to the real certificate file. But depending on the order of
> > evaluation of the different certificate files, it may actually make the symlink
> > point to the real certificate.
> >
> >  What you propose looks like the good solution for it. Except in case of
> > rebuild, so you should remove /etc/ssl/ca-certificates.crt before running
> > c_rehash. That would also remove the need for patch 2 I think.
> >
> >  Could you check if I'm correct, and if so, resubmit the series with:
> >
> > - the additional rm -f;
> > - a commit message that includes the explanation above;
> > - drops patch 2.
> >
> >  If patch 2 really is needed, it needs a better explanation.
>
>  Investigating a bit more turns out that I was rather wrong. The rm -f is
> already there, as correctly noted in your commit message. And patch 2 really is
> needed, it fixes a completely different set of duplicates.
>
>
>  So I've extended the commit message a little and applied to master, thanks.

Thanks for committing the patches.

Martin

>
>
>  Regards,
>  Arnout
>
>
> >
> >
> >  Regards,
> >  Arnout
> >
> >>  $ ls -l /etc/ssl/certs/87229d21.0
> >>  lrwxrwxrwx 1 root root 24 Mar 18 16:30 /etc/ssl/certs/87229d21.0 ->
> >> OpenTrust_Root_CA_G1.pem
> >>
> >> The key issue is the
> >>
> >>   WARNING: Skipping duplicate certificate xxxxx
> >>
> >> where xxxx is the name of one of the files under etc/ssl/certs.
> >> Sometimes it's ca-certificates.crt and hence you don't see any issue
> >> (as you found).  Sometime it's one of the CA files which is the bug
> >> i'm trying to fix.  You can run c_rehash directly
> >>
> >>  ./output/host/bin/c_rehash ./output/target/etc/ssl/certs
> >>
> >> and you will see the warning.  If you rm
> >> output/target/etc/ssl/certs/ca-certificates.crt and try again then the
> >> warning will go.
> >>
> >> I had not released quite how random the bug.  It certainly is not
> >> always EE_Certification_Centre_Root_CA.pem, it can be any CA that goes
> >> wrong.  The commit message could be changed to make it clearer the bug
> >> does not allows happen to EE_Certification_Centre_Root_CA.pem but i'm
> >> 100% sure this is a bug that needs fixing.
> >>
> >> Thanks
> >>
> >> Martin
> >>
> >>> Best regards,
> >>>
> >>> Thomas
> >>> --
> >>> Thomas Petazzoni, CTO, Bootlin (formerly Free Electrons)
> >>> Embedded Linux and Kernel engineering
> >>> https://bootlin.com
> >> _______________________________________________
> >> buildroot mailing list
> >> buildroot at busybox.net
> >> http://lists.busybox.net/mailman/listinfo/buildroot
> >>

[Buildroot] [PATCH v2 1/3] package/ca-certificates: don't hash certificates.crt

[Peter Korsgaard]

>>>>> "Arnout" == Arnout Vandecappelle <arnout@mind.be> writes:

Hi,

>> Could you check if I'm correct, and if so, resubmit the series with:
 >> 
 >> - the additional rm -f;
 >> - a commit message that includes the explanation above;
 >> - drops patch 2.
 >> 
 >> If patch 2 really is needed, it needs a better explanation.

 > ?Investigating a bit more turns out that I was rather wrong. The rm -f is
 > already there, as correctly noted in your commit message. And patch 2 really is
 > needed, it fixes a completely different set of duplicates.


 > ?So I've extended the commit message a little and applied to master, thanks.

Committed to 2018.02.x and 2018.08.x with the extended commit message, thanks.

-- 
Bye, Peter Korsgaard