curl-native and ca-bundle

curl-native and ca-bundle

[Blaettler, Michael]

Hi all

We just had an issue in regard to curl-native.
By default curl is configured with the "--with-ca-bundle=${sysconfdir}/ssl/certs/ca-certificates.crt" flag.
In case curl-native is builded the ${sysconfdir} of the current project is compiled into the binary. Due to sstate caching the binary will be reused in other projects, but the ca-bundle is still loaded from the first project. As soon as the first project (where the initial build took place) is deleted, curl-native won't be able to fetch from HTTPS sources, because the ca-path is invalid.

As a quick solution I removed the "--with-ca-bundle" configure option in native builds and curl is now loading the default certificate chain of the build host.

Does anybody found simmilar issues in other recipes?
How do you handle them?
Is there a common approach?

Kind regards

Michael

Re: curl-native and ca-bundle

[Patrick Ohly]

On Mon, 2016-10-24 at 07:20 +0000, Blaettler, Michael wrote:
> Hi all
> 
> We just had an issue in regard to curl-native.
> By default curl is configured with the "--with-ca-bundle=${sysconfdir}/ssl/certs/ca-certificates.crt" flag.
> In case curl-native is builded the ${sysconfdir} of the current project is compiled into the binary. Due to sstate caching the binary will be reused in other projects, but the ca-bundle is still loaded from the first project. As soon as the first project (where the initial build took place) is deleted, curl-native won't be able to fetch from HTTPS sources, because the ca-path is invalid.
> 
> As a quick solution I removed the "--with-ca-bundle" configure option in native builds and curl is now loading the default certificate chain of the build host.
> 
> Does anybody found simmilar issues in other recipes?

Yes, we ran into the same issue with a CVE check tool, which also uses
libcurl.

> How do you handle them?

We had to patch the tool so that it can override the CA cert path and
then explicitly override the builtin path at runtime, see:
https://github.com/01org/meta-security-isafw/commit/d844f370d5847da08fef83b916e621ebf6b5fa37

Some colleagues recently noticed that the version of cve-check-tool in
OE-core lacks that patch. I'm not sure whether that was reported,
though. André, Ismo?

> Is there a common approach?

No, not really. Patching binaries was mentioned, but it wasn't clear how
to do that in practice.

-- 
Best Regards, Patrick Ohly

The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.

Re: curl-native and ca-bundle

[Blaettler, Michael]

Hi Patrick

What do you think of removing the --with-ca-bundle as a solution for curl-native? On my machine it works without problems.
Might this be an acceptable solution for upstream?

Kind regards

Michael

-----Ursprüngliche Nachricht-----
Von: Patrick Ohly [mailto:patrick.ohly@intel.com] 
Gesendet: Montag, 24. Oktober 2016 15:14
An: Blaettler, Michael (BT CPS R&D ZG FW ITW)
Cc: yocto@yoctoproject.org; Ismo Puustinen; André Draszik
Betreff: Re: [yocto] curl-native and ca-bundle

On Mon, 2016-10-24 at 07:20 +0000, Blaettler, Michael wrote:
> Hi all
> 
> We just had an issue in regard to curl-native.
> By default curl is configured with the "--with-ca-bundle=${sysconfdir}/ssl/certs/ca-certificates.crt" flag.
> In case curl-native is builded the ${sysconfdir} of the current project is compiled into the binary. Due to sstate caching the binary will be reused in other projects, but the ca-bundle is still loaded from the first project. As soon as the first project (where the initial build took place) is deleted, curl-native won't be able to fetch from HTTPS sources, because the ca-path is invalid.
> 
> As a quick solution I removed the "--with-ca-bundle" configure option in native builds and curl is now loading the default certificate chain of the build host.
> 
> Does anybody found simmilar issues in other recipes?

Yes, we ran into the same issue with a CVE check tool, which also uses libcurl.

> How do you handle them?

We had to patch the tool so that it can override the CA cert path and then explicitly override the builtin path at runtime, see:
https://github.com/01org/meta-security-isafw/commit/d844f370d5847da08fef83b916e621ebf6b5fa37

Some colleagues recently noticed that the version of cve-check-tool in OE-core lacks that patch. I'm not sure whether that was reported, though. André, Ismo?

> Is there a common approach?

No, not really. Patching binaries was mentioned, but it wasn't clear how to do that in practice.

--
Best Regards, Patrick Ohly

The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter.

Re: curl-native and ca-bundle

[Patrick Ohly]

On Tue, 2016-10-25 at 05:49 +0000, Blaettler, Michael wrote:
> Hi Patrick
> 
> What do you think of removing the --with-ca-bundle as a solution for
> curl-native? On my machine it works without problems.

What path does it use then? Something that configure determines based on
the current machine (sorry, I'm lazy^Wbusy right now and haven't
checked)?

I suspect that this won't work when moving the resulting lib through
sstate from one Linux distro to another if the location of the bundle
file is different on those two distros.

-- 
Best Regards, Patrick Ohly

The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.

curl-native and ca-bundle

[Blaettler, Michael]

Hi Patrick

I just checked the source code of curl.

In acinclude.m4 on line 2560, you'll find:
dnl CURL_CHECK_CA_BUNDLE
dnl -------------------------------------------------
dnl Check if a default ca-bundle should be used
dnl
dnl regarding the paths this will scan:
dnl /etc/ssl/certs/ca-certificates.crt Debian systems
dnl /etc/pki/tls/certs/ca-bundle.crt Redhat and Mandriva
dnl /usr/share/ssl/certs/ca-bundle.crt old(er) Redhat
dnl /usr/local/share/certs/ca-root-nss.crt FreeBSD
dnl /etc/ssl/cert.pem OpenBSD, FreeBSD (symlink)
dnl /etc/ssl/certs/ (ca path) SUSE

Later in the function there's a for loop, searching every path for the certificate-chain (if --with-ca-bundle is not set).
for a in /etc/ssl/certs/ca-certificates.crt \
         /etc/pki/tls/certs/ca-bundle.crt \
         /usr/share/ssl/certs/ca-bundle.crt \
         /usr/local/share/certs/ca-root-nss.crt \
         /etc/ssl/cert.pem \
         "$cac"; do
    if test -f "$a"; then
        ca="$a"
        break
    fi
done

Regarding this configuration script, removing --with-ca-bundle in curl-native should not cause any problems.

Kind regards

Michael

-----Ursprüngliche Nachricht-----
Von: Patrick Ohly [mailto:patrick.ohly@intel.com] 
Gesendet: Dienstag, 25. Oktober 2016 11:32
An: Blaettler, Michael (BT CPS R&D ZG FW ITW)
Cc: yocto@yoctoproject.org; Ismo Puustinen; André Draszik
Betreff: Re: AW: [yocto] curl-native and ca-bundle

On Tue, 2016-10-25 at 05:49 +0000, Blaettler, Michael wrote:
> Hi Patrick
> 
> What do you think of removing the --with-ca-bundle as a solution for 
> curl-native? On my machine it works without problems.

What path does it use then? Something that configure determines based on the current machine (sorry, I'm lazy^Wbusy right now and haven't checked)?

I suspect that this won't work when moving the resulting lib through sstate from one Linux distro to another if the location of the bundle file is different on those two distros.

--
Best Regards, Patrick Ohly

The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter.

Re: curl-native and ca-bundle

[Patrick Ohly]

On Wed, 2016-10-26 at 06:20 +0000, Blaettler, Michael wrote:
> Hi Patrick
> 
> I just checked the source code of curl.
> 
> In acinclude.m4 on line 2560, you'll find:
> dnl CURL_CHECK_CA_BUNDLE
> dnl -------------------------------------------------
> dnl Check if a default ca-bundle should be used
> dnl
> dnl regarding the paths this will scan:
> dnl /etc/ssl/certs/ca-certificates.crt Debian systems
> dnl /etc/pki/tls/certs/ca-bundle.crt Redhat and Mandriva
> dnl /usr/share/ssl/certs/ca-bundle.crt old(er) Redhat
> dnl /usr/local/share/certs/ca-root-nss.crt FreeBSD
> dnl /etc/ssl/cert.pem OpenBSD, FreeBSD (symlink)
> dnl /etc/ssl/certs/ (ca path) SUSE
> 
> Later in the function there's a for loop, searching every path for the certificate-chain (if --with-ca-bundle is not set).
> for a in /etc/ssl/certs/ca-certificates.crt \
>          /etc/pki/tls/certs/ca-bundle.crt \
>          /usr/share/ssl/certs/ca-bundle.crt \
>          /usr/local/share/certs/ca-root-nss.crt \
>          /etc/ssl/cert.pem \
>          "$cac"; do
>     if test -f "$a"; then
>         ca="$a"
>         break
>     fi
> done
> 
> Regarding this configuration script, removing --with-ca-bundle in curl-native should not cause any problems.

Quite the opposite, it leads exactly to the problem that I feared.

Suppose you build on distro foo where the configure script finds and
thus hardcodes in the binary ca=/etc/ssl/certs/ca-certificates.crt. Then
you build on distro bar which has /etc/pki/tls/certs/ca-bundle.crt
instead. When using uninative, it is likely that compiling curl-native
anew will be skipped and instead curl-native gets installed from the
sstate that was prepared on distro foo. The result is a curl-native that
doesn't have SSL certificates and thus https will not work.

-- 
Best Regards, Patrick Ohly

The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.