[Buildroot] test-pkg script can't handle captive portals. etc.

[Buildroot] test-pkg script can't handle captive portals. etc.

[Marcus Hoffmann]

Hey,

I just ran into an issue with the test-pkg script.
When the TOOLCHAINS_URL returns an unexpected result,
(A router login page, when the Internet got disconnected, a captive
portal login page, a MITM attack, etc.) the script does weird things and
outputs something like this:

    html>: FAILED
<!DOCTYPE: FAILED
     HTML: FAILED
     HTML: ^[ORFAILED
     EN">:
[...]

It also creates the corresponding folders inside the test-dir.

You can test this when pointing the TOOLCHAINS_URL var to any html page.

This it not a very nice way to fail and may lead to harm when parsing
untrusted input from the web.

What would be the best way to handle this case? Can the Toolchain URL be
switched to https? This would eliminate the problem.

Otherwise we should do some sanity checking that no stray html page is
returned by the curl call. But this still doesn't solve the problem of a
malicious actor.

Best wishes,
Marcus

[Buildroot] test-pkg script can't handle captive portals. etc.

[Arnout Vandecappelle]

On 28-02-17 21:30, Marcus Hoffmann wrote:
> Hey,
> 
> I just ran into an issue with the test-pkg script.
> When the TOOLCHAINS_URL returns an unexpected result,
> (A router login page, when the Internet got disconnected, a captive
> portal login page, a MITM attack, etc.) the script does weird things and
> outputs something like this:
> 
>     html>: FAILED
> <!DOCTYPE: FAILED
>      HTML: FAILED
>      HTML: ^[ORFAILED
>      EN">:
> [...]
> 
> It also creates the corresponding folders inside the test-dir.
> 
> You can test this when pointing the TOOLCHAINS_URL var to any html page.
> 
> This it not a very nice way to fail and may lead to harm when parsing
> untrusted input from the web.
> 
> What would be the best way to handle this case? Can the Toolchain URL be
> switched to https? This would eliminate the problem.

 I don't think a.b.o has https at the moment, though I guess it would be easy to
add a Let's Encrypt certificate.

 Still, a captive portal with an accepted certificate could still play tricks.
It's probably better to validate the result.

 However, I think it would be much nicer if we could just have the toolchain
defconfigs inside of Buildroot instead of using this CSV file.


> Otherwise we should do some sanity checking that no stray html page is
> returned by the curl call. But this still doesn't solve the problem of a
> malicious actor.

 I don't think a malicious actor is really something we should worry about here,
is it?

 Regards,
 Arnout


-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF

[Buildroot] test-pkg script can't handle captive portals. etc.

[Marcus Hoffmann]

Hey,

On 01.03.2017 09:46, Arnout Vandecappelle wrote:
> 
> 
> On 28-02-17 21:30, Marcus Hoffmann wrote:
>> Hey,
>>
>> I just ran into an issue with the test-pkg script.
>> When the TOOLCHAINS_URL returns an unexpected result,
>> (A router login page, when the Internet got disconnected, a captive
>> portal login page, a MITM attack, etc.) the script does weird things and
>> outputs something like this:
>>
>>     html>: FAILED
>> <!DOCTYPE: FAILED
>>      HTML: FAILED
>>      HTML: ^[ORFAILED
>>      EN">:
>> [...]
>>
>> It also creates the corresponding folders inside the test-dir.
>>
>> You can test this when pointing the TOOLCHAINS_URL var to any html page.
>>
>> This it not a very nice way to fail and may lead to harm when parsing
>> untrusted input from the web.
>>
>> What would be the best way to handle this case? Can the Toolchain URL be
>> switched to https? This would eliminate the problem.
> 
>  I don't think a.b.o has https at the moment, though I guess it would be easy to
> add a Let's Encrypt certificate.
> 
>  Still, a captive portal with an accepted certificate could still play tricks.

But it wouldn't be valid for the buildroot url, so I don't think it can(?).

> It's probably better to validate the result.
> 
>  However, I think it would be much nicer if we could just have the toolchain
> defconfigs inside of Buildroot instead of using this CSV file.

I don't know how often they change, but if this makes sense this would
be a good solution I think.

> 
> 
>> Otherwise we should do some sanity checking that no stray html page is
>> returned by the curl call. But this still doesn't solve the problem of a
>> malicious actor.
> 
>  I don't think a malicious actor is really something we should worry about here,
> is it?

Probably not terribly so. But if we can easily solve such problems (have
them locally or pull over https) we should!

> 
>  Regards,
>  Arnout
> 
> 

[Buildroot] test-pkg script can't handle captive portals. etc.

[Yann E. MORIN]

Marcus, All,

On 2017-02-28 21:30 +0100, Marcus Hoffmann spake thusly:
> I just ran into an issue with the test-pkg script.
> When the TOOLCHAINS_URL returns an unexpected result,
> (A router login page, when the Internet got disconnected, a captive
> portal login page, a MITM attack, etc.) the script does weird things and
> outputs something like this:
> 
>     html>: FAILED
> <!DOCTYPE: FAILED
>      HTML: FAILED
>      HTML: ^[ORFAILED
>      EN">:
> [...]

In this situation, I would think that curl will happily return
sucessfuly, so we would not have any hint that the download actually
failed.

For that we would have to inspect the output, and see if that looks like
an actual toolchain list.

I'll see if I can do something about it.

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] test-pkg script can't handle captive portals. etc.

[Thomas Petazzoni]

Hello,

On Wed, 1 Mar 2017 09:46:28 +0100, Arnout Vandecappelle wrote:

>  However, I think it would be much nicer if we could just have the toolchain
> defconfigs inside of Buildroot instead of using this CSV file.

This would actually also solve the problem of having the autobuilders
test master/next/lts branches, since the toolchain defconfigs to test
would be part of Buildroot, and not part of buildroot-test.

There is still the problem of all the special "exclusions" encoded in
the autobuild-run script.

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux and Kernel engineering
http://free-electrons.com