Please add new Android branches

Please add new Android branches

[Todd Kjos]

Please add the following new Android branches to kernelci testing.

repo: https://android.googlesource.com/kernel/common
branches:
- android15-6.1
- android14-6.1-lts
- android14-5.15-lts

Thanks,

-Todd

Re: Please add new Android branches

[Guillaume Tucker]

Hi Todd,

On 11/07/2023 00:41, Todd Kjos wrote:
> Please add the following new Android branches to kernelci testing.
> 
> repo: https://android.googlesource.com/kernel/common
> branches:
> - android15-6.1
> - android14-6.1-lts
> - android14-5.15-lts

I've created this PR accordingly:

  https://github.com/kernelci/kernelci-core/pull/2002

Please note that we're now operating with reduced build coverage
due to some temporary limitations with our Azure subscription.  I
know the Android kernel builds should be covered by the GCP
clusters but right now the system is designed to distribute
kernel builds randomly across all clusters so we can't easily tie
Android builds to the Android clusters.  Hopefully we'll find a
solution to go back to normal coverage within a week or two.

Thanks,
Guillaume

Re: Please add new Android branches

[Todd Kjos]

On Thu, Aug 10, 2023 at 8:56 AM Todd Kjos <tkjos@google.com> wrote:
>
>
>
> On Tue, Jul 11, 2023 at 4:57 AM Guillaume Tucker <guillaume.tucker@collabora.com> wrote:
>>
>> Hi Todd,
>>
>> On 11/07/2023 00:41, Todd Kjos wrote:
>> > Please add the following new Android branches to kernelci testing.
>> >
>> > repo: https://android.googlesource.com/kernel/common
>> > branches:
>> > - android15-6.1
>> > - android14-6.1-lts
>> > - android14-5.15-lts
>>
>> I've created this PR accordingly:
>>
>>   https://github.com/kernelci/kernelci-core/pull/2002
>>
>> Please note that we're now operating with reduced build coverage
>> due to some temporary limitations with our Azure subscription.  I
>> know the Android kernel builds should be covered by the GCP
>> clusters but right now the system is designed to distribute
>> kernel builds randomly across all clusters so we can't easily tie
>> Android builds to the Android clusters.  Hopefully we'll find a
>> solution to go back to normal coverage within a week or two.
>
>
> Hi Guillaume, Any ETA for when the full set of builds is restored? We still have greatly reduced build coverage and it's been a month.

Ping. We are getting a lot less value out of kernelci with the greatly
reduced set of builds for the Android kernels. When will the full
build coverage be resumed?

If it needs to stay reduced, can we decide precisely which builds are
done for Android kernels?

>
>>
>>
>> Thanks,
>> Guillaume
>>

Re: Please add new Android branches

[Guillaume Tucker]

Hi Todd,

On 05/09/2023 19:18, Todd Kjos wrote:
> On Thu, Aug 10, 2023 at 8:56 AM Todd Kjos <tkjos@google.com> wrote:
>>
>>
>>
>> On Tue, Jul 11, 2023 at 4:57 AM Guillaume Tucker <guillaume.tucker@collabora.com> wrote:
>>>
>>> Hi Todd,
>>>
>>> On 11/07/2023 00:41, Todd Kjos wrote:
>>>> Please add the following new Android branches to kernelci testing.
>>>>
>>>> repo: https://android.googlesource.com/kernel/common
>>>> branches:
>>>> - android15-6.1
>>>> - android14-6.1-lts
>>>> - android14-5.15-lts
>>>
>>> I've created this PR accordingly:
>>>
>>>   https://github.com/kernelci/kernelci-core/pull/2002
>>>
>>> Please note that we're now operating with reduced build coverage
>>> due to some temporary limitations with our Azure subscription.  I
>>> know the Android kernel builds should be covered by the GCP
>>> clusters but right now the system is designed to distribute
>>> kernel builds randomly across all clusters so we can't easily tie
>>> Android builds to the Android clusters.  Hopefully we'll find a
>>> solution to go back to normal coverage within a week or two.
>>
>>
>> Hi Guillaume, Any ETA for when the full set of builds is restored? We still have greatly reduced build coverage and it's been a month.
> 
> Ping. We are getting a lot less value out of kernelci with the greatly
> reduced set of builds for the Android kernels. When will the full
> build coverage be resumed?
> 
> If it needs to stay reduced, can we decide precisely which builds are
> done for Android kernels?

Sorry for the slow reply.

I think it's fine now to re-enable the Android builds and maybe
we should also take this opportunity to confirm which ones are
the most relevant to you guys.  As part of the process of
adjusting the build coverage, I also fixed the GKI builds which I
thought were the main ones.  Even if there is enough build
capacity to build "everything", keeping it streamlined to what is
really useful makes the overall system more effective.

Are there any particular combinations of arch, config, compilers
you care about more than others?

Thanks,
Guillaume

Re: Please add new Android branches

[Todd Kjos]

+Viktor Martensson +Betty Zhou

Guillaume,

I think that's a great idea. I'll get back to you with the set of
configs, arch, compilers we'd like to have coverage for.

-Todd


On Wed, Sep 6, 2023 at 11:46 AM Guillaume Tucker
<guillaume.tucker@collabora.com> wrote:
>
> Hi Todd,
>
> On 05/09/2023 19:18, Todd Kjos wrote:
> > On Thu, Aug 10, 2023 at 8:56 AM Todd Kjos <tkjos@google.com> wrote:
> >>
> >>
> >>
> >> On Tue, Jul 11, 2023 at 4:57 AM Guillaume Tucker <guillaume.tucker@collabora.com> wrote:
> >>>
> >>> Hi Todd,
> >>>
> >>> On 11/07/2023 00:41, Todd Kjos wrote:
> >>>> Please add the following new Android branches to kernelci testing.
> >>>>
> >>>> repo: https://android.googlesource.com/kernel/common
> >>>> branches:
> >>>> - android15-6.1
> >>>> - android14-6.1-lts
> >>>> - android14-5.15-lts
> >>>
> >>> I've created this PR accordingly:
> >>>
> >>>   https://github.com/kernelci/kernelci-core/pull/2002
> >>>
> >>> Please note that we're now operating with reduced build coverage
> >>> due to some temporary limitations with our Azure subscription.  I
> >>> know the Android kernel builds should be covered by the GCP
> >>> clusters but right now the system is designed to distribute
> >>> kernel builds randomly across all clusters so we can't easily tie
> >>> Android builds to the Android clusters.  Hopefully we'll find a
> >>> solution to go back to normal coverage within a week or two.
> >>
> >>
> >> Hi Guillaume, Any ETA for when the full set of builds is restored? We still have greatly reduced build coverage and it's been a month.
> >
> > Ping. We are getting a lot less value out of kernelci with the greatly
> > reduced set of builds for the Android kernels. When will the full
> > build coverage be resumed?
> >
> > If it needs to stay reduced, can we decide precisely which builds are
> > done for Android kernels?
>
> Sorry for the slow reply.
>
> I think it's fine now to re-enable the Android builds and maybe
> we should also take this opportunity to confirm which ones are
> the most relevant to you guys.  As part of the process of
> adjusting the build coverage, I also fixed the GKI builds which I
> thought were the main ones.  Even if there is enough build
> capacity to build "everything", keeping it streamlined to what is
> really useful makes the overall system more effective.
>
> Are there any particular combinations of arch, config, compilers
> you care about more than others?
>
> Thanks,
> Guillaume
>

Re: Please add new Android branches

[Todd Kjos]

Guillaume,

How about this for android trees:

1) same compilers as current config (gcc-10, clang-14)
2) archs: arm, arm64, i386, x80_64, riscv
3) same configs as current android tests
4) plus configs used by corresponding stable kernel
(https://linux.kernelci.org/job/stable) not included in #3

We get a nice benefit from sync'ing with the stable kernels since it
allows us to tell which issues originate upstream and which are
introduced by android changes. It's also useful to build each with
both toolchains so we can expose the cases where gcc-10 is OK but
clang-14 has issues (and visa-versa).

I don't think we need to build all of the combinations if there are
too many builds - we could trim out some of the configs from #4 (you
can recommend which).

We'd also like to start in an all-green state. So let's trim out the
error cases that aren't caused by android kernel code. I think there
are some 32-bit cases that fail with the clang-14 tools and some other
cases that fail in stable.

Could we start with all of the builds for #1 - #4 and then tune it up
after a few runs to get rid of long-failing cases and/or to just
reduce the load on the build service?

-Todd



On Wed, Sep 6, 2023 at 12:50 PM Todd Kjos <tkjos@google.com> wrote:
>
> +Viktor Martensson +Betty Zhou
>
> Guillaume,
>
> I think that's a great idea. I'll get back to you with the set of
> configs, arch, compilers we'd like to have coverage for.
>
> -Todd
>
>
> On Wed, Sep 6, 2023 at 11:46 AM Guillaume Tucker
> <guillaume.tucker@collabora.com> wrote:
> >
> > Hi Todd,
> >
> > On 05/09/2023 19:18, Todd Kjos wrote:
> > > On Thu, Aug 10, 2023 at 8:56 AM Todd Kjos <tkjos@google.com> wrote:
> > >>
> > >>
> > >>
> > >> On Tue, Jul 11, 2023 at 4:57 AM Guillaume Tucker <guillaume.tucker@collabora.com> wrote:
> > >>>
> > >>> Hi Todd,
> > >>>
> > >>> On 11/07/2023 00:41, Todd Kjos wrote:
> > >>>> Please add the following new Android branches to kernelci testing.
> > >>>>
> > >>>> repo: https://android.googlesource.com/kernel/common
> > >>>> branches:
> > >>>> - android15-6.1
> > >>>> - android14-6.1-lts
> > >>>> - android14-5.15-lts
> > >>>
> > >>> I've created this PR accordingly:
> > >>>
> > >>>   https://github.com/kernelci/kernelci-core/pull/2002
> > >>>
> > >>> Please note that we're now operating with reduced build coverage
> > >>> due to some temporary limitations with our Azure subscription.  I
> > >>> know the Android kernel builds should be covered by the GCP
> > >>> clusters but right now the system is designed to distribute
> > >>> kernel builds randomly across all clusters so we can't easily tie
> > >>> Android builds to the Android clusters.  Hopefully we'll find a
> > >>> solution to go back to normal coverage within a week or two.
> > >>
> > >>
> > >> Hi Guillaume, Any ETA for when the full set of builds is restored? We still have greatly reduced build coverage and it's been a month.
> > >
> > > Ping. We are getting a lot less value out of kernelci with the greatly
> > > reduced set of builds for the Android kernels. When will the full
> > > build coverage be resumed?
> > >
> > > If it needs to stay reduced, can we decide precisely which builds are
> > > done for Android kernels?
> >
> > Sorry for the slow reply.
> >
> > I think it's fine now to re-enable the Android builds and maybe
> > we should also take this opportunity to confirm which ones are
> > the most relevant to you guys.  As part of the process of
> > adjusting the build coverage, I also fixed the GKI builds which I
> > thought were the main ones.  Even if there is enough build
> > capacity to build "everything", keeping it streamlined to what is
> > really useful makes the overall system more effective.
> >
> > Are there any particular combinations of arch, config, compilers
> > you care about more than others?
> >
> > Thanks,
> > Guillaume
> >

Re: Please add new Android branches

[Nick Desaulniers]

On Tue, Sep 19, 2023 at 3:54 PM Todd Kjos <tkjos@google.com> wrote:
>
> Guillaume,
>
> How about this for android trees:
>
> 1) same compilers as current config (gcc-10, clang-14)
> 2) archs: arm, arm64, i386, x80_64, riscv
> 3) same configs as current android tests
> 4) plus configs used by corresponding stable kernel
> (https://linux.kernelci.org/job/stable) not included in #3
>
> We get a nice benefit from sync'ing with the stable kernels since it
> allows us to tell which issues originate upstream and which are
> introduced by android changes. It's also useful to build each with
> both toolchains so we can expose the cases where gcc-10 is OK but
> clang-14 has issues (and visa-versa).
>
> I don't think we need to build all of the combinations if there are
> too many builds - we could trim out some of the configs from #4 (you
> can recommend which).
>
> We'd also like to start in an all-green state. So let's trim out the
> error cases that aren't caused by android kernel code. I think there
> are some 32-bit cases that fail with the clang-14 tools and some other
> cases that fail in stable.

That would be news to me. Got any more info on that?

>
> Could we start with all of the builds for #1 - #4 and then tune it up
> after a few runs to get rid of long-failing cases and/or to just
> reduce the load on the build service?
>
> -Todd
>
>
>
> On Wed, Sep 6, 2023 at 12:50 PM Todd Kjos <tkjos@google.com> wrote:
> >
> > +Viktor Martensson +Betty Zhou
> >
> > Guillaume,
> >
> > I think that's a great idea. I'll get back to you with the set of
> > configs, arch, compilers we'd like to have coverage for.
> >
> > -Todd
> >
> >
> > On Wed, Sep 6, 2023 at 11:46 AM Guillaume Tucker
> > <guillaume.tucker@collabora.com> wrote:
> > >
> > > Hi Todd,
> > >
> > > On 05/09/2023 19:18, Todd Kjos wrote:
> > > > On Thu, Aug 10, 2023 at 8:56 AM Todd Kjos <tkjos@google.com> wrote:
> > > >>
> > > >>
> > > >>
> > > >> On Tue, Jul 11, 2023 at 4:57 AM Guillaume Tucker <guillaume.tucker@collabora.com> wrote:
> > > >>>
> > > >>> Hi Todd,
> > > >>>
> > > >>> On 11/07/2023 00:41, Todd Kjos wrote:
> > > >>>> Please add the following new Android branches to kernelci testing.
> > > >>>>
> > > >>>> repo: https://android.googlesource.com/kernel/common
> > > >>>> branches:
> > > >>>> - android15-6.1
> > > >>>> - android14-6.1-lts
> > > >>>> - android14-5.15-lts
> > > >>>
> > > >>> I've created this PR accordingly:
> > > >>>
> > > >>>   https://github.com/kernelci/kernelci-core/pull/2002
> > > >>>
> > > >>> Please note that we're now operating with reduced build coverage
> > > >>> due to some temporary limitations with our Azure subscription.  I
> > > >>> know the Android kernel builds should be covered by the GCP
> > > >>> clusters but right now the system is designed to distribute
> > > >>> kernel builds randomly across all clusters so we can't easily tie
> > > >>> Android builds to the Android clusters.  Hopefully we'll find a
> > > >>> solution to go back to normal coverage within a week or two.
> > > >>
> > > >>
> > > >> Hi Guillaume, Any ETA for when the full set of builds is restored? We still have greatly reduced build coverage and it's been a month.
> > > >
> > > > Ping. We are getting a lot less value out of kernelci with the greatly
> > > > reduced set of builds for the Android kernels. When will the full
> > > > build coverage be resumed?
> > > >
> > > > If it needs to stay reduced, can we decide precisely which builds are
> > > > done for Android kernels?
> > >
> > > Sorry for the slow reply.
> > >
> > > I think it's fine now to re-enable the Android builds and maybe
> > > we should also take this opportunity to confirm which ones are
> > > the most relevant to you guys.  As part of the process of
> > > adjusting the build coverage, I also fixed the GKI builds which I
> > > thought were the main ones.  Even if there is enough build
> > > capacity to build "everything", keeping it streamlined to what is
> > > really useful makes the overall system more effective.
> > >
> > > Are there any particular combinations of arch, config, compilers
> > > you care about more than others?
> > >
> > > Thanks,
> > > Guillaume
> > >
>


-- 
Thanks,
~Nick Desaulniers

Re: Please add new Android branches

[Todd Kjos]

On Tue, Sep 19, 2023 at 4:02 PM Nick Desaulniers
<ndesaulniers@google.com> wrote:
>
> On Tue, Sep 19, 2023 at 3:54 PM Todd Kjos <tkjos@google.com> wrote:
> >
> > Guillaume,
> >
> > How about this for android trees:
> >
> > 1) same compilers as current config (gcc-10, clang-14)
> > 2) archs: arm, arm64, i386, x80_64, riscv
> > 3) same configs as current android tests
> > 4) plus configs used by corresponding stable kernel
> > (https://linux.kernelci.org/job/stable) not included in #3
> >
> > We get a nice benefit from sync'ing with the stable kernels since it
> > allows us to tell which issues originate upstream and which are
> > introduced by android changes. It's also useful to build each with
> > both toolchains so we can expose the cases where gcc-10 is OK but
> > clang-14 has issues (and visa-versa).
> >
> > I don't think we need to build all of the combinations if there are
> > too many builds - we could trim out some of the configs from #4 (you
> > can recommend which).
> >
> > We'd also like to start in an all-green state. So let's trim out the
> > error cases that aren't caused by android kernel code. I think there
> > are some 32-bit cases that fail with the clang-14 tools and some other
> > cases that fail in stable.
>
> That would be news to me. Got any more info on that?

We've talked about it. You said at the time that they were known
issues with clan and old arm architectures, but you didn't have time
to deal with them. We're not building any of those cases now with the
reduced test set, but here is an example from June 15:

https://linux.kernelci.org/build/android/branch/android13-5.10/kernel/ASB-2023-06-05_13-5.10-9-g4d9cc0c1eadb2/

Here are the errors/warnings:

Build Logs Summary

Errors Summary

19drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c:2447:9: error: shift
count >= width of type [-Werror,-Wshift-count-overflow]
5drivers/net/ethernet/i825xx/ether1.c:237:1: error: invalid
instruction, did you mean: ldrb, ldrexb?
5drivers/net/ethernet/i825xx/ether1.c:175:1: error: invalid
instruction, did you mean: strb, strexb?
4drivers/net/ethernet/i825xx/ether1.c:238:1: error: invalid
instruction, did you mean: strb, strexb?
4drivers/net/ethernet/i825xx/ether1.c:174:1: error: invalid
instruction, did you mean: ldrb, ldrexb?
2fatal error: too many errors emitted, stopping now [-ferror-limit=]
2arm-linux-gnueabihf-gcc: error: unrecognized -march target: armv3m
2arm-linux-gnueabihf-gcc: error: missing argument to ‘-march=’
1lib/bitfield_kunit.c:93:1: error: the frame size of 4192 bytes is
larger than 2048 bytes [-Werror=frame-larger-than=]
1ld.lld: error: undefined symbol: clock_task_mult
1fs/namei.c:2131:13: error: use of bitwise '|' with boolean operands
[-Werror,-Wbitwise-instead-of-logical]
1drivers/usb/gadget/udc/pxa25x_udc.c:2328:11: error: invalid % escape
in inline assembly string
1arch/arm/mach-ep93xx/crunch-bits.S:99:2: error: invalid instruction
1arch/arm/mach-ep93xx/crunch-bits.S:98:2: error: invalid instruction
1arch/arm/mach-ep93xx/crunch-bits.S:97:2: error: invalid instruction
1arch/arm/mach-ep93xx/crunch-bits.S:96:2: error: invalid instruction
1arch/arm/mach-ep93xx/crunch-bits.S:95:2: error: invalid instruction
1arch/arm/mach-ep93xx/crunch-bits.S:94:2: error: invalid instruction
1arch/arm/mach-ep93xx/crunch-bits.S:191:2: error: invalid instruction
1arch/arm/mach-ep93xx/crunch-bits.S:190:2: error: invalid instruction
1arch/arm/mach-ep93xx/crunch-bits.S:189:2: error: invalid instruction
1arch/arm/mach-ep93xx/crunch-bits.S:188:2: error: invalid instruction
1arch/arm/mach-ep93xx/crunch-bits.S:187:2: error: invalid instruction
1arch/arm/mach-ep93xx/crunch-bits.S:186:2: error: invalid instruction
1arch/arm/mach-ep93xx/crunch-bits.S:185:2: error: invalid instruction
1arch/arm/mach-ep93xx/crunch-bits.S:184:2: error: invalid instruction
1arch/arm/mach-ep93xx/crunch-bits.S:183:2: error: invalid instruction
1arch/arm/mach-ep93xx/crunch-bits.S:182:2: error: invalid instruction
1arch/arm/mach-ep93xx/crunch-bits.S:181:2: error: invalid instruction
1arch/arm/mach-ep93xx/crunch-bits.S:180:2: error: invalid instruction
1arch/arm/mach-ep93xx/crunch-bits.S:179:2: error: invalid instruction
1arch/arm/mach-ep93xx/crunch-bits.S:178:2: error: invalid instruction
1arch/arm/mach-ep93xx/crunch-bits.S:177:2: error: invalid instruction
1arch/arm/mach-ep93xx/crunch-bits.S:176:2: error: invalid instruction
1arch/arm/mach-ep93xx/crunch-bits.S:174:2: error: invalid instruction
1arch/arm/mach-ep93xx/crunch-bits.S:173:2: error: invalid instruction
1arch/arm/mach-ep93xx/crunch-bits.S:172:2: error: invalid instruction
1arch/arm/mach-ep93xx/crunch-bits.S:171:2: error: invalid instruction
1arch/arm/mach-ep93xx/crunch-bits.S:170:2: error: invalid instruction
1arch/arm/mach-ep93xx/crunch-bits.S:169:2: error: invalid instruction
1arch/arm/mach-ep93xx/crunch-bits.S:168:2: error: invalid instruction
1arch/arm/mach-ep93xx/crunch-bits.S:167:2: error: invalid instruction
1arch/arm/mach-ep93xx/crunch-bits.S:166:2: error: invalid instruction
1arch/arm/mach-ep93xx/crunch-bits.S:165:2: error: invalid instruction
1arch/arm/mach-ep93xx/crunch-bits.S:164:2: error: invalid instruction
1arch/arm/mach-ep93xx/crunch-bits.S:163:2: error: invalid instruction
1arch/arm/mach-ep93xx/crunch-bits.S:162:2: error: invalid instruction
1arch/arm/mach-ep93xx/crunch-bits.S:161:2: error: invalid instruction
1arch/arm/mach-ep93xx/crunch-bits.S:160:2: error: invalid instruction
1arch/arm/mach-ep93xx/crunch-bits.S:159:2: error: invalid instruction
1arch/arm/mach-ep93xx/crunch-bits.S:158:2: error: invalid instruction
1arch/arm/mach-ep93xx/crunch-bits.S:157:2: error: invalid instruction
1arch/arm/mach-ep93xx/crunch-bits.S:156:2: error: invalid instruction
1arch/arm/mach-ep93xx/crunch-bits.S:155:2: error: invalid instruction
1arch/arm/mach-ep93xx/crunch-bits.S:154:2: error: invalid instruction
1arch/arm/mach-ep93xx/crunch-bits.S:153:2: error: invalid instruction
1arch/arm/mach-ep93xx/crunch-bits.S:152:2: error: invalid instruction
1arch/arm/mach-ep93xx/crunch-bits.S:151:2: error: invalid instruction
1arch/arm/mach-ep93xx/crunch-bits.S:149:2: error: invalid instruction
1arch/arm/mach-ep93xx/crunch-bits.S:148:2: error: invalid instruction
1arch/arm/mach-ep93xx/crunch-bits.S:144:2: error: invalid instruction
1arch/arm/mach-ep93xx/crunch-bits.S:141:2: error: invalid instruction
1arch/arm/mach-ep93xx/crunch-bits.S:140:2: error: invalid instruction
1arch/arm/mach-ep93xx/crunch-bits.S:138:2: error: invalid instruction
1arch/arm/mach-ep93xx/crunch-bits.S:137:2: error: invalid instruction
1arch/arm/mach-ep93xx/crunch-bits.S:136:2: error: invalid instruction
1arch/arm/mach-ep93xx/crunch-bits.S:135:2: error: invalid instruction
1arch/arm/mach-ep93xx/crunch-bits.S:134:2: error: invalid instruction
1arch/arm/mach-ep93xx/crunch-bits.S:133:2: error: invalid instruction
1arch/arm/mach-ep93xx/crunch-bits.S:132:2: error: invalid instruction
1arch/arm/mach-ep93xx/crunch-bits.S:131:2: error: invalid instruction
1arch/arm/mach-ep93xx/crunch-bits.S:130:2: error: invalid instruction
1arch/arm/mach-ep93xx/crunch-bits.S:129:2: error: invalid instruction
1arch/arm/mach-ep93xx/crunch-bits.S:128:2: error: invalid instruction
1arch/arm/mach-ep93xx/crunch-bits.S:127:2: error: invalid instruction
1arch/arm/mach-ep93xx/crunch-bits.S:126:2: error: invalid instruction
1arch/arm/mach-ep93xx/crunch-bits.S:125:2: error: invalid instruction
1arch/arm/mach-ep93xx/crunch-bits.S:124:2: error: invalid instruction
1arch/arm/mach-ep93xx/crunch-bits.S:123:2: error: invalid instruction
1arch/arm/mach-ep93xx/crunch-bits.S:122:2: error: invalid instruction
1arch/arm/mach-ep93xx/crunch-bits.S:121:2: error: invalid instruction
1arch/arm/mach-ep93xx/crunch-bits.S:120:2: error: invalid instruction
1arch/arm/mach-ep93xx/crunch-bits.S:119:2: error: invalid instruction
1arch/arm/mach-ep93xx/crunch-bits.S:118:2: error: invalid instruction
1arch/arm/mach-ep93xx/crunch-bits.S:117:2: error: invalid instruction
1arch/arm/mach-ep93xx/crunch-bits.S:116:2: error: invalid instruction
1arch/arm/mach-ep93xx/crunch-bits.S:115:2: error: invalid instruction
1arch/arm/mach-ep93xx/crunch-bits.S:109:2: error: invalid instruction
1arch/arm/mach-ep93xx/crunch-bits.S:108:2: error: invalid instruction
1arch/arm/mach-ep93xx/crunch-bits.S:107:2: error: invalid instruction
1arch/arm/mach-ep93xx/crunch-bits.S:106:2: error: invalid instruction
1arch/arm/mach-ep93xx/crunch-bits.S:105:2: error: invalid instruction
1arch/arm/mach-ep93xx/crunch-bits.S:104:2: error: invalid instruction
1arch/arm/mach-ep93xx/crunch-bits.S:103:2: error: invalid instruction
1arch/arm/mach-ep93xx/crunch-bits.S:102:2: error: invalid instruction
1arch/arm/mach-ep93xx/crunch-bits.S:101:2: error: invalid instruction
1arch/arm/mach-ep93xx/crunch-bits.S:100:2: error: invalid instruction
1<stdin>:830:2: error: syscall fstat64 not implemented [-Werror,-W#warnings]
1<stdin>:830:2: error: #warning syscall fstat64 not implemented [-Werror=cpp]
1<stdin>:1127:2: error: syscall fstatat64 not implemented [-Werror,-W#warnings]
1<stdin>:1127:2: error: #warning syscall fstatat64 not implemented [-Werror=cpp]
1:1: error: invalid instruction, did you mean: ldrb, ldrexb?
1/tmp/kci/linux/build/../kernel/sched/fair.c:4998: undefined reference
to `clock_task_mult'
1/tmp/kci/linux/build/../kernel/sched/fair.c:4983: undefined reference
to `clock_task_mult'
1/tmp/kci/linux/build/../kernel/sched/fair.c:11476: undefined
reference to `clock_task_mult'

Warnings Summary

2163ld.lld: warning: lld uses blx instruction, no object with
architecture supporting feature detected
188clang: warning: argument unused during compilation:
'-march=armv7-a' [-Wunused-command-line-argument]
92clang: warning: argument unused during compilation: '-march=armv6k'
[-Wunused-command-line-argument]
5clang: warning: argument unused during compilation: '-march=armv7-m'
[-Wunused-command-line-argument]
2cc1: all warnings being treated as errors
22 warnings generated.
2./usr/include/linux/bcache.h:355:2: warning: field '' with variable
sized type 'union jset::(anonymous at
./usr/include/linux/bcache.h:355:2)' not at the end of a struct or
class is a GNU extension [-Wgnu-variable-sized-type-not-at-end]
2./usr/include/linux/bcache.h:354:2: warning: field '' with variable
sized type 'union jset::(anonymous at
./usr/include/linux/bcache.h:354:2)' not at the end of a struct or
class is a GNU extension [-Wgnu-variable-sized-type-not-at-end]
1fs/namei.c:2131:13: note: cast one or both operands to int to silence
this warning
1WARNING: modpost: module pstore uses symbol sync_filesystem from
namespace VFS_internal_I_am_really_a_filesystem_and_am_NOT_a_driver,
but does not import it.
1WARNING: modpost: module pstore uses symbol clear_inode from
namespace VFS_internal_I_am_really_a_filesystem_and_am_NOT_a_driver,
but does not import it.
1WARNING: modpost: module dax uses symbol unlock_new_inode from
namespace VFS_internal_I_am_really_a_filesystem_and_am_NOT_a_driver,
but does not import it.
1WARNING: modpost: module dax uses symbol inode_init_once from
namespace VFS_internal_I_am_really_a_filesystem_and_am_NOT_a_driver,
but does not import it.
1WARNING: modpost: module dax uses symbol iget5_locked from namespace
VFS_internal_I_am_really_a_filesystem_and_am_NOT_a_driver, but does
not import it.
1#warning syscall fstatat64 not implemented
1#warning syscall fstat64 not implemented


>
> >
> > Could we start with all of the builds for #1 - #4 and then tune it up
> > after a few runs to get rid of long-failing cases and/or to just
> > reduce the load on the build service?
> >
> > -Todd
> >
> >
> >
> > On Wed, Sep 6, 2023 at 12:50 PM Todd Kjos <tkjos@google.com> wrote:
> > >
> > > +Viktor Martensson +Betty Zhou
> > >
> > > Guillaume,
> > >
> > > I think that's a great idea. I'll get back to you with the set of
> > > configs, arch, compilers we'd like to have coverage for.
> > >
> > > -Todd
> > >
> > >
> > > On Wed, Sep 6, 2023 at 11:46 AM Guillaume Tucker
> > > <guillaume.tucker@collabora.com> wrote:
> > > >
> > > > Hi Todd,
> > > >
> > > > On 05/09/2023 19:18, Todd Kjos wrote:
> > > > > On Thu, Aug 10, 2023 at 8:56 AM Todd Kjos <tkjos@google.com> wrote:
> > > > >>
> > > > >>
> > > > >>
> > > > >> On Tue, Jul 11, 2023 at 4:57 AM Guillaume Tucker <guillaume.tucker@collabora.com> wrote:
> > > > >>>
> > > > >>> Hi Todd,
> > > > >>>
> > > > >>> On 11/07/2023 00:41, Todd Kjos wrote:
> > > > >>>> Please add the following new Android branches to kernelci testing.
> > > > >>>>
> > > > >>>> repo: https://android.googlesource.com/kernel/common
> > > > >>>> branches:
> > > > >>>> - android15-6.1
> > > > >>>> - android14-6.1-lts
> > > > >>>> - android14-5.15-lts
> > > > >>>
> > > > >>> I've created this PR accordingly:
> > > > >>>
> > > > >>>   https://github.com/kernelci/kernelci-core/pull/2002
> > > > >>>
> > > > >>> Please note that we're now operating with reduced build coverage
> > > > >>> due to some temporary limitations with our Azure subscription.  I
> > > > >>> know the Android kernel builds should be covered by the GCP
> > > > >>> clusters but right now the system is designed to distribute
> > > > >>> kernel builds randomly across all clusters so we can't easily tie
> > > > >>> Android builds to the Android clusters.  Hopefully we'll find a
> > > > >>> solution to go back to normal coverage within a week or two.
> > > > >>
> > > > >>
> > > > >> Hi Guillaume, Any ETA for when the full set of builds is restored? We still have greatly reduced build coverage and it's been a month.
> > > > >
> > > > > Ping. We are getting a lot less value out of kernelci with the greatly
> > > > > reduced set of builds for the Android kernels. When will the full
> > > > > build coverage be resumed?
> > > > >
> > > > > If it needs to stay reduced, can we decide precisely which builds are
> > > > > done for Android kernels?
> > > >
> > > > Sorry for the slow reply.
> > > >
> > > > I think it's fine now to re-enable the Android builds and maybe
> > > > we should also take this opportunity to confirm which ones are
> > > > the most relevant to you guys.  As part of the process of
> > > > adjusting the build coverage, I also fixed the GKI builds which I
> > > > thought were the main ones.  Even if there is enough build
> > > > capacity to build "everything", keeping it streamlined to what is
> > > > really useful makes the overall system more effective.
> > > >
> > > > Are there any particular combinations of arch, config, compilers
> > > > you care about more than others?
> > > >
> > > > Thanks,
> > > > Guillaume
> > > >
> >
>
>
> --
> Thanks,
> ~Nick Desaulniers

Re: Please add new Android branches

[Nick Desaulniers]

On Tue, Sep 19, 2023 at 4:12 PM Todd Kjos <tkjos@google.com> wrote:
>
> On Tue, Sep 19, 2023 at 4:02 PM Nick Desaulniers
> <ndesaulniers@google.com> wrote:
> >
> > On Tue, Sep 19, 2023 at 3:54 PM Todd Kjos <tkjos@google.com> wrote:
> > >
> > > Guillaume,
> > >
> > > How about this for android trees:
> > >
> > > 1) same compilers as current config (gcc-10, clang-14)
> > > 2) archs: arm, arm64, i386, x80_64, riscv
> > > 3) same configs as current android tests
> > > 4) plus configs used by corresponding stable kernel
> > > (https://linux.kernelci.org/job/stable) not included in #3
> > >
> > > We get a nice benefit from sync'ing with the stable kernels since it
> > > allows us to tell which issues originate upstream and which are
> > > introduced by android changes. It's also useful to build each with
> > > both toolchains so we can expose the cases where gcc-10 is OK but
> > > clang-14 has issues (and visa-versa).

Also, why clang-14? Clang 18 is ToT, clang 17 is latest release.

> > >
> > > I don't think we need to build all of the combinations if there are
> > > too many builds - we could trim out some of the configs from #4 (you
> > > can recommend which).
> > >
> > > We'd also like to start in an all-green state. So let's trim out the
> > > error cases that aren't caused by android kernel code. I think there
> > > are some 32-bit cases that fail with the clang-14 tools and some other
> > > cases that fail in stable.
> >
> > That would be news to me. Got any more info on that?
>
> We've talked about it. You said at the time that they were known
> issues with clan and old arm architectures, but you didn't have time
> to deal with them. We're not building any of those cases now with the
> reduced test set, but here is an example from June 15:
>
> https://linux.kernelci.org/build/android/branch/android13-5.10/kernel/ASB-2023-06-05_13-5.10-9-g4d9cc0c1eadb2/

Ah, the long tail of configs.  I guess since ACK doesn't have a
gki_defconfig for ARCH=arm, a grab bag of arbitrary configs is the
next best thing.

Half of the red exclamation marks on that link are are "0 warnings - 0
errors".  What's up with that?

For the ones with errors, clicking logs gives you HTTP 404.  Guessing
those aren't retained very long? Surprised then the landing page is.

>
> Here are the errors/warnings:
>
> Build Logs Summary
>
> Errors Summary
>
> 19drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c:2447:9: error: shift
> count >= width of type [-Werror,-Wshift-count-overflow]
> 5drivers/net/ethernet/i825xx/ether1.c:237:1: error: invalid
> instruction, did you mean: ldrb, ldrexb?
> 5drivers/net/ethernet/i825xx/ether1.c:175:1: error: invalid
> instruction, did you mean: strb, strexb?
> 4drivers/net/ethernet/i825xx/ether1.c:238:1: error: invalid
> instruction, did you mean: strb, strexb?
> 4drivers/net/ethernet/i825xx/ether1.c:174:1: error: invalid
> instruction, did you mean: ldrb, ldrexb?
> 2fatal error: too many errors emitted, stopping now [-ferror-limit=]
> 2arm-linux-gnueabihf-gcc: error: unrecognized -march target: armv3m
> 2arm-linux-gnueabihf-gcc: error: missing argument to ‘-march=’
> 1lib/bitfield_kunit.c:93:1: error: the frame size of 4192 bytes is
> larger than 2048 bytes [-Werror=frame-larger-than=]
> 1ld.lld: error: undefined symbol: clock_task_mult
> 1fs/namei.c:2131:13: error: use of bitwise '|' with boolean operands
> [-Werror,-Wbitwise-instead-of-logical]
> 1drivers/usb/gadget/udc/pxa25x_udc.c:2328:11: error: invalid % escape
> in inline assembly string
> 1arch/arm/mach-ep93xx/crunch-bits.S:99:2: error: invalid instruction
> 1arch/arm/mach-ep93xx/crunch-bits.S:98:2: error: invalid instruction
> 1arch/arm/mach-ep93xx/crunch-bits.S:97:2: error: invalid instruction
> 1arch/arm/mach-ep93xx/crunch-bits.S:96:2: error: invalid instruction
> 1arch/arm/mach-ep93xx/crunch-bits.S:95:2: error: invalid instruction
> 1arch/arm/mach-ep93xx/crunch-bits.S:94:2: error: invalid instruction
> 1arch/arm/mach-ep93xx/crunch-bits.S:191:2: error: invalid instruction
> 1arch/arm/mach-ep93xx/crunch-bits.S:190:2: error: invalid instruction
> 1arch/arm/mach-ep93xx/crunch-bits.S:189:2: error: invalid instruction
> 1arch/arm/mach-ep93xx/crunch-bits.S:188:2: error: invalid instruction
> 1arch/arm/mach-ep93xx/crunch-bits.S:187:2: error: invalid instruction
> 1arch/arm/mach-ep93xx/crunch-bits.S:186:2: error: invalid instruction
> 1arch/arm/mach-ep93xx/crunch-bits.S:185:2: error: invalid instruction
> 1arch/arm/mach-ep93xx/crunch-bits.S:184:2: error: invalid instruction
> 1arch/arm/mach-ep93xx/crunch-bits.S:183:2: error: invalid instruction
> 1arch/arm/mach-ep93xx/crunch-bits.S:182:2: error: invalid instruction
> 1arch/arm/mach-ep93xx/crunch-bits.S:181:2: error: invalid instruction
> 1arch/arm/mach-ep93xx/crunch-bits.S:180:2: error: invalid instruction
> 1arch/arm/mach-ep93xx/crunch-bits.S:179:2: error: invalid instruction
> 1arch/arm/mach-ep93xx/crunch-bits.S:178:2: error: invalid instruction
> 1arch/arm/mach-ep93xx/crunch-bits.S:177:2: error: invalid instruction
> 1arch/arm/mach-ep93xx/crunch-bits.S:176:2: error: invalid instruction
> 1arch/arm/mach-ep93xx/crunch-bits.S:174:2: error: invalid instruction
> 1arch/arm/mach-ep93xx/crunch-bits.S:173:2: error: invalid instruction
> 1arch/arm/mach-ep93xx/crunch-bits.S:172:2: error: invalid instruction
> 1arch/arm/mach-ep93xx/crunch-bits.S:171:2: error: invalid instruction
> 1arch/arm/mach-ep93xx/crunch-bits.S:170:2: error: invalid instruction
> 1arch/arm/mach-ep93xx/crunch-bits.S:169:2: error: invalid instruction
> 1arch/arm/mach-ep93xx/crunch-bits.S:168:2: error: invalid instruction
> 1arch/arm/mach-ep93xx/crunch-bits.S:167:2: error: invalid instruction
> 1arch/arm/mach-ep93xx/crunch-bits.S:166:2: error: invalid instruction
> 1arch/arm/mach-ep93xx/crunch-bits.S:165:2: error: invalid instruction
> 1arch/arm/mach-ep93xx/crunch-bits.S:164:2: error: invalid instruction
> 1arch/arm/mach-ep93xx/crunch-bits.S:163:2: error: invalid instruction
> 1arch/arm/mach-ep93xx/crunch-bits.S:162:2: error: invalid instruction
> 1arch/arm/mach-ep93xx/crunch-bits.S:161:2: error: invalid instruction
> 1arch/arm/mach-ep93xx/crunch-bits.S:160:2: error: invalid instruction
> 1arch/arm/mach-ep93xx/crunch-bits.S:159:2: error: invalid instruction
> 1arch/arm/mach-ep93xx/crunch-bits.S:158:2: error: invalid instruction
> 1arch/arm/mach-ep93xx/crunch-bits.S:157:2: error: invalid instruction
> 1arch/arm/mach-ep93xx/crunch-bits.S:156:2: error: invalid instruction
> 1arch/arm/mach-ep93xx/crunch-bits.S:155:2: error: invalid instruction
> 1arch/arm/mach-ep93xx/crunch-bits.S:154:2: error: invalid instruction
> 1arch/arm/mach-ep93xx/crunch-bits.S:153:2: error: invalid instruction
> 1arch/arm/mach-ep93xx/crunch-bits.S:152:2: error: invalid instruction
> 1arch/arm/mach-ep93xx/crunch-bits.S:151:2: error: invalid instruction
> 1arch/arm/mach-ep93xx/crunch-bits.S:149:2: error: invalid instruction
> 1arch/arm/mach-ep93xx/crunch-bits.S:148:2: error: invalid instruction
> 1arch/arm/mach-ep93xx/crunch-bits.S:144:2: error: invalid instruction
> 1arch/arm/mach-ep93xx/crunch-bits.S:141:2: error: invalid instruction
> 1arch/arm/mach-ep93xx/crunch-bits.S:140:2: error: invalid instruction
> 1arch/arm/mach-ep93xx/crunch-bits.S:138:2: error: invalid instruction
> 1arch/arm/mach-ep93xx/crunch-bits.S:137:2: error: invalid instruction
> 1arch/arm/mach-ep93xx/crunch-bits.S:136:2: error: invalid instruction
> 1arch/arm/mach-ep93xx/crunch-bits.S:135:2: error: invalid instruction
> 1arch/arm/mach-ep93xx/crunch-bits.S:134:2: error: invalid instruction
> 1arch/arm/mach-ep93xx/crunch-bits.S:133:2: error: invalid instruction
> 1arch/arm/mach-ep93xx/crunch-bits.S:132:2: error: invalid instruction
> 1arch/arm/mach-ep93xx/crunch-bits.S:131:2: error: invalid instruction
> 1arch/arm/mach-ep93xx/crunch-bits.S:130:2: error: invalid instruction
> 1arch/arm/mach-ep93xx/crunch-bits.S:129:2: error: invalid instruction
> 1arch/arm/mach-ep93xx/crunch-bits.S:128:2: error: invalid instruction
> 1arch/arm/mach-ep93xx/crunch-bits.S:127:2: error: invalid instruction
> 1arch/arm/mach-ep93xx/crunch-bits.S:126:2: error: invalid instruction
> 1arch/arm/mach-ep93xx/crunch-bits.S:125:2: error: invalid instruction
> 1arch/arm/mach-ep93xx/crunch-bits.S:124:2: error: invalid instruction
> 1arch/arm/mach-ep93xx/crunch-bits.S:123:2: error: invalid instruction
> 1arch/arm/mach-ep93xx/crunch-bits.S:122:2: error: invalid instruction
> 1arch/arm/mach-ep93xx/crunch-bits.S:121:2: error: invalid instruction
> 1arch/arm/mach-ep93xx/crunch-bits.S:120:2: error: invalid instruction
> 1arch/arm/mach-ep93xx/crunch-bits.S:119:2: error: invalid instruction
> 1arch/arm/mach-ep93xx/crunch-bits.S:118:2: error: invalid instruction
> 1arch/arm/mach-ep93xx/crunch-bits.S:117:2: error: invalid instruction
> 1arch/arm/mach-ep93xx/crunch-bits.S:116:2: error: invalid instruction
> 1arch/arm/mach-ep93xx/crunch-bits.S:115:2: error: invalid instruction
> 1arch/arm/mach-ep93xx/crunch-bits.S:109:2: error: invalid instruction
> 1arch/arm/mach-ep93xx/crunch-bits.S:108:2: error: invalid instruction
> 1arch/arm/mach-ep93xx/crunch-bits.S:107:2: error: invalid instruction
> 1arch/arm/mach-ep93xx/crunch-bits.S:106:2: error: invalid instruction
> 1arch/arm/mach-ep93xx/crunch-bits.S:105:2: error: invalid instruction
> 1arch/arm/mach-ep93xx/crunch-bits.S:104:2: error: invalid instruction
> 1arch/arm/mach-ep93xx/crunch-bits.S:103:2: error: invalid instruction
> 1arch/arm/mach-ep93xx/crunch-bits.S:102:2: error: invalid instruction
> 1arch/arm/mach-ep93xx/crunch-bits.S:101:2: error: invalid instruction
> 1arch/arm/mach-ep93xx/crunch-bits.S:100:2: error: invalid instruction
> 1<stdin>:830:2: error: syscall fstat64 not implemented [-Werror,-W#warnings]
> 1<stdin>:830:2: error: #warning syscall fstat64 not implemented [-Werror=cpp]
> 1<stdin>:1127:2: error: syscall fstatat64 not implemented [-Werror,-W#warnings]
> 1<stdin>:1127:2: error: #warning syscall fstatat64 not implemented [-Werror=cpp]
> 1:1: error: invalid instruction, did you mean: ldrb, ldrexb?
> 1/tmp/kci/linux/build/../kernel/sched/fair.c:4998: undefined reference
> to `clock_task_mult'
> 1/tmp/kci/linux/build/../kernel/sched/fair.c:4983: undefined reference
> to `clock_task_mult'
> 1/tmp/kci/linux/build/../kernel/sched/fair.c:11476: undefined
> reference to `clock_task_mult'
>
> Warnings Summary
>
> 2163ld.lld: warning: lld uses blx instruction, no object with
> architecture supporting feature detected
> 188clang: warning: argument unused during compilation:
> '-march=armv7-a' [-Wunused-command-line-argument]
> 92clang: warning: argument unused during compilation: '-march=armv6k'
> [-Wunused-command-line-argument]
> 5clang: warning: argument unused during compilation: '-march=armv7-m'
> [-Wunused-command-line-argument]
> 2cc1: all warnings being treated as errors
> 22 warnings generated.
> 2./usr/include/linux/bcache.h:355:2: warning: field '' with variable
> sized type 'union jset::(anonymous at
> ./usr/include/linux/bcache.h:355:2)' not at the end of a struct or
> class is a GNU extension [-Wgnu-variable-sized-type-not-at-end]
> 2./usr/include/linux/bcache.h:354:2: warning: field '' with variable
> sized type 'union jset::(anonymous at
> ./usr/include/linux/bcache.h:354:2)' not at the end of a struct or
> class is a GNU extension [-Wgnu-variable-sized-type-not-at-end]
> 1fs/namei.c:2131:13: note: cast one or both operands to int to silence
> this warning
> 1WARNING: modpost: module pstore uses symbol sync_filesystem from
> namespace VFS_internal_I_am_really_a_filesystem_and_am_NOT_a_driver,
> but does not import it.
> 1WARNING: modpost: module pstore uses symbol clear_inode from
> namespace VFS_internal_I_am_really_a_filesystem_and_am_NOT_a_driver,
> but does not import it.
> 1WARNING: modpost: module dax uses symbol unlock_new_inode from
> namespace VFS_internal_I_am_really_a_filesystem_and_am_NOT_a_driver,
> but does not import it.
> 1WARNING: modpost: module dax uses symbol inode_init_once from
> namespace VFS_internal_I_am_really_a_filesystem_and_am_NOT_a_driver,
> but does not import it.
> 1WARNING: modpost: module dax uses symbol iget5_locked from namespace
> VFS_internal_I_am_really_a_filesystem_and_am_NOT_a_driver, but does
> not import it.
> 1#warning syscall fstatat64 not implemented
> 1#warning syscall fstat64 not implemented
>
>
> >
> > >
> > > Could we start with all of the builds for #1 - #4 and then tune it up
> > > after a few runs to get rid of long-failing cases and/or to just
> > > reduce the load on the build service?
> > >
> > > -Todd
> > >
> > >
> > >
> > > On Wed, Sep 6, 2023 at 12:50 PM Todd Kjos <tkjos@google.com> wrote:
> > > >
> > > > +Viktor Martensson +Betty Zhou
> > > >
> > > > Guillaume,
> > > >
> > > > I think that's a great idea. I'll get back to you with the set of
> > > > configs, arch, compilers we'd like to have coverage for.
> > > >
> > > > -Todd
> > > >
> > > >
> > > > On Wed, Sep 6, 2023 at 11:46 AM Guillaume Tucker
> > > > <guillaume.tucker@collabora.com> wrote:
> > > > >
> > > > > Hi Todd,
> > > > >
> > > > > On 05/09/2023 19:18, Todd Kjos wrote:
> > > > > > On Thu, Aug 10, 2023 at 8:56 AM Todd Kjos <tkjos@google.com> wrote:
> > > > > >>
> > > > > >>
> > > > > >>
> > > > > >> On Tue, Jul 11, 2023 at 4:57 AM Guillaume Tucker <guillaume.tucker@collabora.com> wrote:
> > > > > >>>
> > > > > >>> Hi Todd,
> > > > > >>>
> > > > > >>> On 11/07/2023 00:41, Todd Kjos wrote:
> > > > > >>>> Please add the following new Android branches to kernelci testing.
> > > > > >>>>
> > > > > >>>> repo: https://android.googlesource.com/kernel/common
> > > > > >>>> branches:
> > > > > >>>> - android15-6.1
> > > > > >>>> - android14-6.1-lts
> > > > > >>>> - android14-5.15-lts
> > > > > >>>
> > > > > >>> I've created this PR accordingly:
> > > > > >>>
> > > > > >>>   https://github.com/kernelci/kernelci-core/pull/2002
> > > > > >>>
> > > > > >>> Please note that we're now operating with reduced build coverage
> > > > > >>> due to some temporary limitations with our Azure subscription.  I
> > > > > >>> know the Android kernel builds should be covered by the GCP
> > > > > >>> clusters but right now the system is designed to distribute
> > > > > >>> kernel builds randomly across all clusters so we can't easily tie
> > > > > >>> Android builds to the Android clusters.  Hopefully we'll find a
> > > > > >>> solution to go back to normal coverage within a week or two.
> > > > > >>
> > > > > >>
> > > > > >> Hi Guillaume, Any ETA for when the full set of builds is restored? We still have greatly reduced build coverage and it's been a month.
> > > > > >
> > > > > > Ping. We are getting a lot less value out of kernelci with the greatly
> > > > > > reduced set of builds for the Android kernels. When will the full
> > > > > > build coverage be resumed?
> > > > > >
> > > > > > If it needs to stay reduced, can we decide precisely which builds are
> > > > > > done for Android kernels?
> > > > >
> > > > > Sorry for the slow reply.
> > > > >
> > > > > I think it's fine now to re-enable the Android builds and maybe
> > > > > we should also take this opportunity to confirm which ones are
> > > > > the most relevant to you guys.  As part of the process of
> > > > > adjusting the build coverage, I also fixed the GKI builds which I
> > > > > thought were the main ones.  Even if there is enough build
> > > > > capacity to build "everything", keeping it streamlined to what is
> > > > > really useful makes the overall system more effective.
> > > > >
> > > > > Are there any particular combinations of arch, config, compilers
> > > > > you care about more than others?
> > > > >
> > > > > Thanks,
> > > > > Guillaume
> > > > >
> > >
> >
> >
> > --
> > Thanks,
> > ~Nick Desaulniers



-- 
Thanks,
~Nick Desaulniers

Re: Please add new Android branches

[Todd Kjos]

On Tue, Sep 19, 2023 at 4:19 PM Nick Desaulniers
<ndesaulniers@google.com> wrote:
>
> On Tue, Sep 19, 2023 at 4:12 PM Todd Kjos <tkjos@google.com> wrote:
> >
> > On Tue, Sep 19, 2023 at 4:02 PM Nick Desaulniers
> > <ndesaulniers@google.com> wrote:
> > >
> > > On Tue, Sep 19, 2023 at 3:54 PM Todd Kjos <tkjos@google.com> wrote:
> > > >
> > > > Guillaume,
> > > >
> > > > How about this for android trees:
> > > >
> > > > 1) same compilers as current config (gcc-10, clang-14)
> > > > 2) archs: arm, arm64, i386, x80_64, riscv
> > > > 3) same configs as current android tests
> > > > 4) plus configs used by corresponding stable kernel
> > > > (https://linux.kernelci.org/job/stable) not included in #3
> > > >
> > > > We get a nice benefit from sync'ing with the stable kernels since it
> > > > allows us to tell which issues originate upstream and which are
> > > > introduced by android changes. It's also useful to build each with
> > > > both toolchains so we can expose the cases where gcc-10 is OK but
> > > > clang-14 has issues (and visa-versa).
>
> Also, why clang-14? Clang 18 is ToT, clang 17 is latest release.

Those are just the defaults in kernelci today. I'm obviously fine with
moving forward to clang-18 if it is available in kernelci.

>
>
> > > >
> > > > I don't think we need to build all of the combinations if there are
> > > > too many builds - we could trim out some of the configs from #4 (you
> > > > can recommend which).
> > > >
> > > > We'd also like to start in an all-green state. So let's trim out the
> > > > error cases that aren't caused by android kernel code. I think there
> > > > are some 32-bit cases that fail with the clang-14 tools and some other
> > > > cases that fail in stable.
> > >
> > > That would be news to me. Got any more info on that?
> >
> > We've talked about it. You said at the time that they were known
> > issues with clan and old arm architectures, but you didn't have time
> > to deal with them. We're not building any of those cases now with the
> > reduced test set, but here is an example from June 15:
> >
> > https://linux.kernelci.org/build/android/branch/android13-5.10/kernel/ASB-2023-06-05_13-5.10-9-g4d9cc0c1eadb2/
>
> Ah, the long tail of configs.  I guess since ACK doesn't have a
> gki_defconfig for ARCH=arm, a grab bag of arbitrary configs is the
> next best thing.

Much of the kernelci tests for us are really about kernel hygiene. We
do a lot of testing internally of gki_defconfig for the architectures
we support, but there are non-GKI downstream users of our kernels and
kernelci is the primary way we make sure we aren't breaking other
builds with different configs. It is also the only place where
gki_defconfig is built with gcc which is useful to make sure we
haven't accidentally broken gcc builds.

>
> Half of the red exclamation marks on that link are are "0 warnings - 0
> errors".  What's up with that?

This more recent build has "0 warnings - 0 errors":
https://linux.kernelci.org/build/android/branch/android15-6.1/kernel/ASB-2023-07-05_14-6.1-2620-gf2d0380464d3/

Here is the error (not sure why it bubbled up as 0/0):

  AR      drivers/base/power/built-in.a
../fs/namei.c:2172:13: error: use of bitwise '|' with boolean operands
[-Werror,-Wbitwise-instead-of-logical]
        } while (!(has_zero(a, &adata, &constants) | has_zero(b,
&bdata, &constants)));

~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
                                                   ||
../fs/namei.c:2172:13: note: cast one or both operands to int to
silence this warning
  CC      drivers/base/firmware_loader/builtin/main.o
1 error generated.
make[3]: *** [../scripts/Makefile.build:250: fs/namei.o] Error 1
make[3]: *** Waiting for unfinished jobs....
make[2]: *** [../scripts/Makefile.build:500: fs] Error 2

>
> For the ones with errors, clicking logs gives you HTTP 404.  Guessing
> those aren't retained very long? Surprised then the landing page is.

Agreed. If you want to take action on them, we'd need to retry those
builds (probably can do that locally since you have the branch,
toolchain, and config info).

I'd like to just abandon those builds for old architectures so our
builds are clean unless we break something.

>
> >
> > Here are the errors/warnings:
> >
> > Build Logs Summary
> >
> > Errors Summary
> >
> > 19drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c:2447:9: error: shift
> > count >= width of type [-Werror,-Wshift-count-overflow]
> > 5drivers/net/ethernet/i825xx/ether1.c:237:1: error: invalid
> > instruction, did you mean: ldrb, ldrexb?
> > 5drivers/net/ethernet/i825xx/ether1.c:175:1: error: invalid
> > instruction, did you mean: strb, strexb?
> > 4drivers/net/ethernet/i825xx/ether1.c:238:1: error: invalid
> > instruction, did you mean: strb, strexb?
> > 4drivers/net/ethernet/i825xx/ether1.c:174:1: error: invalid
> > instruction, did you mean: ldrb, ldrexb?
> > 2fatal error: too many errors emitted, stopping now [-ferror-limit=]
> > 2arm-linux-gnueabihf-gcc: error: unrecognized -march target: armv3m
> > 2arm-linux-gnueabihf-gcc: error: missing argument to ‘-march=’
> > 1lib/bitfield_kunit.c:93:1: error: the frame size of 4192 bytes is
> > larger than 2048 bytes [-Werror=frame-larger-than=]
> > 1ld.lld: error: undefined symbol: clock_task_mult
> > 1fs/namei.c:2131:13: error: use of bitwise '|' with boolean operands
> > [-Werror,-Wbitwise-instead-of-logical]
> > 1drivers/usb/gadget/udc/pxa25x_udc.c:2328:11: error: invalid % escape
> > in inline assembly string
> > 1arch/arm/mach-ep93xx/crunch-bits.S:99:2: error: invalid instruction
> > 1arch/arm/mach-ep93xx/crunch-bits.S:98:2: error: invalid instruction
> > 1arch/arm/mach-ep93xx/crunch-bits.S:97:2: error: invalid instruction
> > 1arch/arm/mach-ep93xx/crunch-bits.S:96:2: error: invalid instruction
> > 1arch/arm/mach-ep93xx/crunch-bits.S:95:2: error: invalid instruction
> > 1arch/arm/mach-ep93xx/crunch-bits.S:94:2: error: invalid instruction
> > 1arch/arm/mach-ep93xx/crunch-bits.S:191:2: error: invalid instruction
> > 1arch/arm/mach-ep93xx/crunch-bits.S:190:2: error: invalid instruction
> > 1arch/arm/mach-ep93xx/crunch-bits.S:189:2: error: invalid instruction
> > 1arch/arm/mach-ep93xx/crunch-bits.S:188:2: error: invalid instruction
> > 1arch/arm/mach-ep93xx/crunch-bits.S:187:2: error: invalid instruction
> > 1arch/arm/mach-ep93xx/crunch-bits.S:186:2: error: invalid instruction
> > 1arch/arm/mach-ep93xx/crunch-bits.S:185:2: error: invalid instruction
> > 1arch/arm/mach-ep93xx/crunch-bits.S:184:2: error: invalid instruction
> > 1arch/arm/mach-ep93xx/crunch-bits.S:183:2: error: invalid instruction
> > 1arch/arm/mach-ep93xx/crunch-bits.S:182:2: error: invalid instruction
> > 1arch/arm/mach-ep93xx/crunch-bits.S:181:2: error: invalid instruction
> > 1arch/arm/mach-ep93xx/crunch-bits.S:180:2: error: invalid instruction
> > 1arch/arm/mach-ep93xx/crunch-bits.S:179:2: error: invalid instruction
> > 1arch/arm/mach-ep93xx/crunch-bits.S:178:2: error: invalid instruction
> > 1arch/arm/mach-ep93xx/crunch-bits.S:177:2: error: invalid instruction
> > 1arch/arm/mach-ep93xx/crunch-bits.S:176:2: error: invalid instruction
> > 1arch/arm/mach-ep93xx/crunch-bits.S:174:2: error: invalid instruction
> > 1arch/arm/mach-ep93xx/crunch-bits.S:173:2: error: invalid instruction
> > 1arch/arm/mach-ep93xx/crunch-bits.S:172:2: error: invalid instruction
> > 1arch/arm/mach-ep93xx/crunch-bits.S:171:2: error: invalid instruction
> > 1arch/arm/mach-ep93xx/crunch-bits.S:170:2: error: invalid instruction
> > 1arch/arm/mach-ep93xx/crunch-bits.S:169:2: error: invalid instruction
> > 1arch/arm/mach-ep93xx/crunch-bits.S:168:2: error: invalid instruction
> > 1arch/arm/mach-ep93xx/crunch-bits.S:167:2: error: invalid instruction
> > 1arch/arm/mach-ep93xx/crunch-bits.S:166:2: error: invalid instruction
> > 1arch/arm/mach-ep93xx/crunch-bits.S:165:2: error: invalid instruction
> > 1arch/arm/mach-ep93xx/crunch-bits.S:164:2: error: invalid instruction
> > 1arch/arm/mach-ep93xx/crunch-bits.S:163:2: error: invalid instruction
> > 1arch/arm/mach-ep93xx/crunch-bits.S:162:2: error: invalid instruction
> > 1arch/arm/mach-ep93xx/crunch-bits.S:161:2: error: invalid instruction
> > 1arch/arm/mach-ep93xx/crunch-bits.S:160:2: error: invalid instruction
> > 1arch/arm/mach-ep93xx/crunch-bits.S:159:2: error: invalid instruction
> > 1arch/arm/mach-ep93xx/crunch-bits.S:158:2: error: invalid instruction
> > 1arch/arm/mach-ep93xx/crunch-bits.S:157:2: error: invalid instruction
> > 1arch/arm/mach-ep93xx/crunch-bits.S:156:2: error: invalid instruction
> > 1arch/arm/mach-ep93xx/crunch-bits.S:155:2: error: invalid instruction
> > 1arch/arm/mach-ep93xx/crunch-bits.S:154:2: error: invalid instruction
> > 1arch/arm/mach-ep93xx/crunch-bits.S:153:2: error: invalid instruction
> > 1arch/arm/mach-ep93xx/crunch-bits.S:152:2: error: invalid instruction
> > 1arch/arm/mach-ep93xx/crunch-bits.S:151:2: error: invalid instruction
> > 1arch/arm/mach-ep93xx/crunch-bits.S:149:2: error: invalid instruction
> > 1arch/arm/mach-ep93xx/crunch-bits.S:148:2: error: invalid instruction
> > 1arch/arm/mach-ep93xx/crunch-bits.S:144:2: error: invalid instruction
> > 1arch/arm/mach-ep93xx/crunch-bits.S:141:2: error: invalid instruction
> > 1arch/arm/mach-ep93xx/crunch-bits.S:140:2: error: invalid instruction
> > 1arch/arm/mach-ep93xx/crunch-bits.S:138:2: error: invalid instruction
> > 1arch/arm/mach-ep93xx/crunch-bits.S:137:2: error: invalid instruction
> > 1arch/arm/mach-ep93xx/crunch-bits.S:136:2: error: invalid instruction
> > 1arch/arm/mach-ep93xx/crunch-bits.S:135:2: error: invalid instruction
> > 1arch/arm/mach-ep93xx/crunch-bits.S:134:2: error: invalid instruction
> > 1arch/arm/mach-ep93xx/crunch-bits.S:133:2: error: invalid instruction
> > 1arch/arm/mach-ep93xx/crunch-bits.S:132:2: error: invalid instruction
> > 1arch/arm/mach-ep93xx/crunch-bits.S:131:2: error: invalid instruction
> > 1arch/arm/mach-ep93xx/crunch-bits.S:130:2: error: invalid instruction
> > 1arch/arm/mach-ep93xx/crunch-bits.S:129:2: error: invalid instruction
> > 1arch/arm/mach-ep93xx/crunch-bits.S:128:2: error: invalid instruction
> > 1arch/arm/mach-ep93xx/crunch-bits.S:127:2: error: invalid instruction
> > 1arch/arm/mach-ep93xx/crunch-bits.S:126:2: error: invalid instruction
> > 1arch/arm/mach-ep93xx/crunch-bits.S:125:2: error: invalid instruction
> > 1arch/arm/mach-ep93xx/crunch-bits.S:124:2: error: invalid instruction
> > 1arch/arm/mach-ep93xx/crunch-bits.S:123:2: error: invalid instruction
> > 1arch/arm/mach-ep93xx/crunch-bits.S:122:2: error: invalid instruction
> > 1arch/arm/mach-ep93xx/crunch-bits.S:121:2: error: invalid instruction
> > 1arch/arm/mach-ep93xx/crunch-bits.S:120:2: error: invalid instruction
> > 1arch/arm/mach-ep93xx/crunch-bits.S:119:2: error: invalid instruction
> > 1arch/arm/mach-ep93xx/crunch-bits.S:118:2: error: invalid instruction
> > 1arch/arm/mach-ep93xx/crunch-bits.S:117:2: error: invalid instruction
> > 1arch/arm/mach-ep93xx/crunch-bits.S:116:2: error: invalid instruction
> > 1arch/arm/mach-ep93xx/crunch-bits.S:115:2: error: invalid instruction
> > 1arch/arm/mach-ep93xx/crunch-bits.S:109:2: error: invalid instruction
> > 1arch/arm/mach-ep93xx/crunch-bits.S:108:2: error: invalid instruction
> > 1arch/arm/mach-ep93xx/crunch-bits.S:107:2: error: invalid instruction
> > 1arch/arm/mach-ep93xx/crunch-bits.S:106:2: error: invalid instruction
> > 1arch/arm/mach-ep93xx/crunch-bits.S:105:2: error: invalid instruction
> > 1arch/arm/mach-ep93xx/crunch-bits.S:104:2: error: invalid instruction
> > 1arch/arm/mach-ep93xx/crunch-bits.S:103:2: error: invalid instruction
> > 1arch/arm/mach-ep93xx/crunch-bits.S:102:2: error: invalid instruction
> > 1arch/arm/mach-ep93xx/crunch-bits.S:101:2: error: invalid instruction
> > 1arch/arm/mach-ep93xx/crunch-bits.S:100:2: error: invalid instruction
> > 1<stdin>:830:2: error: syscall fstat64 not implemented [-Werror,-W#warnings]
> > 1<stdin>:830:2: error: #warning syscall fstat64 not implemented [-Werror=cpp]
> > 1<stdin>:1127:2: error: syscall fstatat64 not implemented [-Werror,-W#warnings]
> > 1<stdin>:1127:2: error: #warning syscall fstatat64 not implemented [-Werror=cpp]
> > 1:1: error: invalid instruction, did you mean: ldrb, ldrexb?
> > 1/tmp/kci/linux/build/../kernel/sched/fair.c:4998: undefined reference
> > to `clock_task_mult'
> > 1/tmp/kci/linux/build/../kernel/sched/fair.c:4983: undefined reference
> > to `clock_task_mult'
> > 1/tmp/kci/linux/build/../kernel/sched/fair.c:11476: undefined
> > reference to `clock_task_mult'
> >
> > Warnings Summary
> >
> > 2163ld.lld: warning: lld uses blx instruction, no object with
> > architecture supporting feature detected
> > 188clang: warning: argument unused during compilation:
> > '-march=armv7-a' [-Wunused-command-line-argument]
> > 92clang: warning: argument unused during compilation: '-march=armv6k'
> > [-Wunused-command-line-argument]
> > 5clang: warning: argument unused during compilation: '-march=armv7-m'
> > [-Wunused-command-line-argument]
> > 2cc1: all warnings being treated as errors
> > 22 warnings generated.
> > 2./usr/include/linux/bcache.h:355:2: warning: field '' with variable
> > sized type 'union jset::(anonymous at
> > ./usr/include/linux/bcache.h:355:2)' not at the end of a struct or
> > class is a GNU extension [-Wgnu-variable-sized-type-not-at-end]
> > 2./usr/include/linux/bcache.h:354:2: warning: field '' with variable
> > sized type 'union jset::(anonymous at
> > ./usr/include/linux/bcache.h:354:2)' not at the end of a struct or
> > class is a GNU extension [-Wgnu-variable-sized-type-not-at-end]
> > 1fs/namei.c:2131:13: note: cast one or both operands to int to silence
> > this warning
> > 1WARNING: modpost: module pstore uses symbol sync_filesystem from
> > namespace VFS_internal_I_am_really_a_filesystem_and_am_NOT_a_driver,
> > but does not import it.
> > 1WARNING: modpost: module pstore uses symbol clear_inode from
> > namespace VFS_internal_I_am_really_a_filesystem_and_am_NOT_a_driver,
> > but does not import it.
> > 1WARNING: modpost: module dax uses symbol unlock_new_inode from
> > namespace VFS_internal_I_am_really_a_filesystem_and_am_NOT_a_driver,
> > but does not import it.
> > 1WARNING: modpost: module dax uses symbol inode_init_once from
> > namespace VFS_internal_I_am_really_a_filesystem_and_am_NOT_a_driver,
> > but does not import it.
> > 1WARNING: modpost: module dax uses symbol iget5_locked from namespace
> > VFS_internal_I_am_really_a_filesystem_and_am_NOT_a_driver, but does
> > not import it.
> > 1#warning syscall fstatat64 not implemented
> > 1#warning syscall fstat64 not implemented
> >
> >
> > >
> > > >
> > > > Could we start with all of the builds for #1 - #4 and then tune it up
> > > > after a few runs to get rid of long-failing cases and/or to just
> > > > reduce the load on the build service?
> > > >
> > > > -Todd
> > > >
> > > >
> > > >
> > > > On Wed, Sep 6, 2023 at 12:50 PM Todd Kjos <tkjos@google.com> wrote:
> > > > >
> > > > > +Viktor Martensson +Betty Zhou
> > > > >
> > > > > Guillaume,
> > > > >
> > > > > I think that's a great idea. I'll get back to you with the set of
> > > > > configs, arch, compilers we'd like to have coverage for.
> > > > >
> > > > > -Todd
> > > > >
> > > > >
> > > > > On Wed, Sep 6, 2023 at 11:46 AM Guillaume Tucker
> > > > > <guillaume.tucker@collabora.com> wrote:
> > > > > >
> > > > > > Hi Todd,
> > > > > >
> > > > > > On 05/09/2023 19:18, Todd Kjos wrote:
> > > > > > > On Thu, Aug 10, 2023 at 8:56 AM Todd Kjos <tkjos@google.com> wrote:
> > > > > > >>
> > > > > > >>
> > > > > > >>
> > > > > > >> On Tue, Jul 11, 2023 at 4:57 AM Guillaume Tucker <guillaume.tucker@collabora.com> wrote:
> > > > > > >>>
> > > > > > >>> Hi Todd,
> > > > > > >>>
> > > > > > >>> On 11/07/2023 00:41, Todd Kjos wrote:
> > > > > > >>>> Please add the following new Android branches to kernelci testing.
> > > > > > >>>>
> > > > > > >>>> repo: https://android.googlesource.com/kernel/common
> > > > > > >>>> branches:
> > > > > > >>>> - android15-6.1
> > > > > > >>>> - android14-6.1-lts
> > > > > > >>>> - android14-5.15-lts
> > > > > > >>>
> > > > > > >>> I've created this PR accordingly:
> > > > > > >>>
> > > > > > >>>   https://github.com/kernelci/kernelci-core/pull/2002
> > > > > > >>>
> > > > > > >>> Please note that we're now operating with reduced build coverage
> > > > > > >>> due to some temporary limitations with our Azure subscription.  I
> > > > > > >>> know the Android kernel builds should be covered by the GCP
> > > > > > >>> clusters but right now the system is designed to distribute
> > > > > > >>> kernel builds randomly across all clusters so we can't easily tie
> > > > > > >>> Android builds to the Android clusters.  Hopefully we'll find a
> > > > > > >>> solution to go back to normal coverage within a week or two.
> > > > > > >>
> > > > > > >>
> > > > > > >> Hi Guillaume, Any ETA for when the full set of builds is restored? We still have greatly reduced build coverage and it's been a month.
> > > > > > >
> > > > > > > Ping. We are getting a lot less value out of kernelci with the greatly
> > > > > > > reduced set of builds for the Android kernels. When will the full
> > > > > > > build coverage be resumed?
> > > > > > >
> > > > > > > If it needs to stay reduced, can we decide precisely which builds are
> > > > > > > done for Android kernels?
> > > > > >
> > > > > > Sorry for the slow reply.
> > > > > >
> > > > > > I think it's fine now to re-enable the Android builds and maybe
> > > > > > we should also take this opportunity to confirm which ones are
> > > > > > the most relevant to you guys.  As part of the process of
> > > > > > adjusting the build coverage, I also fixed the GKI builds which I
> > > > > > thought were the main ones.  Even if there is enough build
> > > > > > capacity to build "everything", keeping it streamlined to what is
> > > > > > really useful makes the overall system more effective.
> > > > > >
> > > > > > Are there any particular combinations of arch, config, compilers
> > > > > > you care about more than others?
> > > > > >
> > > > > > Thanks,
> > > > > > Guillaume
> > > > > >
> > > >
> > >
> > >
> > > --
> > > Thanks,
> > > ~Nick Desaulniers
>
>
>
> --
> Thanks,
> ~Nick Desaulniers

Re: Please add new Android branches

[Nick Desaulniers]

On Tue, Sep 19, 2023 at 5:12 PM Todd Kjos <tkjos@google.com> wrote:
>
> On Tue, Sep 19, 2023 at 4:19 PM Nick Desaulniers
> <ndesaulniers@google.com> wrote:
> >
> > Half of the red exclamation marks on that link are are "0 warnings - 0
> > errors".  What's up with that?
>
> This more recent build has "0 warnings - 0 errors":
> https://linux.kernelci.org/build/android/branch/android15-6.1/kernel/ASB-2023-07-05_14-6.1-2620-gf2d0380464d3/
>
> Here is the error (not sure why it bubbled up as 0/0):
>
>   AR      drivers/base/power/built-in.a
> ../fs/namei.c:2172:13: error: use of bitwise '|' with boolean operands
> [-Werror,-Wbitwise-instead-of-logical]
>         } while (!(has_zero(a, &adata, &constants) | has_zero(b,
> &bdata, &constants)));
>
> ~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>                                                    ||
> ../fs/namei.c:2172:13: note: cast one or both operands to int to
> silence this warning
>   CC      drivers/base/firmware_loader/builtin/main.o
> 1 error generated.
> make[3]: *** [../scripts/Makefile.build:250: fs/namei.o] Error 1
> make[3]: *** Waiting for unfinished jobs....
> make[2]: *** [../scripts/Makefile.build:500: fs] Error 2

Guillaume,
Any idea what's up with that? Seems like the existence of diagnostics
is detected correctly; the number of warnings and errors is not.


-- 
Thanks,
~Nick Desaulniers

Re: Please add new Android branches

[Guillaume Tucker]

On 20/09/2023 18:57, Nick Desaulniers wrote:
> On Tue, Sep 19, 2023 at 5:12 PM Todd Kjos <tkjos@google.com> wrote:
>>
>> On Tue, Sep 19, 2023 at 4:19 PM Nick Desaulniers
>> <ndesaulniers@google.com> wrote:
>>>
>>> Half of the red exclamation marks on that link are are "0 warnings - 0
>>> errors".  What's up with that?
>>
>> This more recent build has "0 warnings - 0 errors":
>> https://linux.kernelci.org/build/android/branch/android15-6.1/kernel/ASB-2023-07-05_14-6.1-2620-gf2d0380464d3/
>>
>> Here is the error (not sure why it bubbled up as 0/0):
>>
>>   AR      drivers/base/power/built-in.a
>> ../fs/namei.c:2172:13: error: use of bitwise '|' with boolean operands
>> [-Werror,-Wbitwise-instead-of-logical]
>>         } while (!(has_zero(a, &adata, &constants) | has_zero(b,
>> &bdata, &constants)));
>>
>> ~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>>                                                    ||
>> ../fs/namei.c:2172:13: note: cast one or both operands to int to
>> silence this warning
>>   CC      drivers/base/firmware_loader/builtin/main.o
>> 1 error generated.
>> make[3]: *** [../scripts/Makefile.build:250: fs/namei.o] Error 1
>> make[3]: *** Waiting for unfinished jobs....
>> make[2]: *** [../scripts/Makefile.build:500: fs] Error 2
> 
> Guillaume,
> Any idea what's up with that? Seems like the existence of diagnostics
> is detected correctly; the number of warnings and errors is not.

Yes I think there's some heuristics in the legacy log parsing
tool to count the number of compiler warnings and errors.  This
probably is outdated with Clang now, so the build exit status is
used to detect a failure but then the stats are off.

It's better to avoid fixing this code as it's going to be retired
in a few months' time to be replaced with a fresh implementation
based on the new API.  I believe some tweaks could be done to fix
critical issues, but if the detail of warnings and errors can be
found even though the stats are off then maybe we can save the
effort of fixing it?

Cheers,
Guillaume

Re: Please add new Android branches

[Guillaume Tucker]

On 20/09/2023 00:54, Todd Kjos wrote:
> Guillaume,
> 
> How about this for android trees:
> 
> 1) same compilers as current config (gcc-10, clang-14)
> 2) archs: arm, arm64, i386, x80_64, riscv
> 3) same configs as current android tests
> 4) plus configs used by corresponding stable kernel
> (https://linux.kernelci.org/job/stable) not included in #3

OK thanks, I think we can set this up easily.  We'll get back to
you if there's any ambiguity.

> We get a nice benefit from sync'ing with the stable kernels since it
> allows us to tell which issues originate upstream and which are
> introduced by android changes. It's also useful to build each with
> both toolchains so we can expose the cases where gcc-10 is OK but
> clang-14 has issues (and visa-versa).
> 
> I don't think we need to build all of the combinations if there are
> too many builds - we could trim out some of the configs from #4 (you
> can recommend which).

OK, we can look at past results or optimise incrementally over
the weeks to streamline the overall build coverage.

> We'd also like to start in an all-green state. So let's trim out the
> error cases that aren't caused by android kernel code. I think there
> are some 32-bit cases that fail with the clang-14 tools and some other
> cases that fail in stable.
> 
> Could we start with all of the builds for #1 - #4 and then tune it up
> after a few runs to get rid of long-failing cases and/or to just
> reduce the load on the build service?

I believe this part of the discussion is still unresolved with
Nick, and maybe it would make more sense to address this when the
builds get moved to the new API where we'll have more flexibility
and more accurate log parsing etc.

Another aspect that wasn't too clear in the discussions, should
it be clang-14 or a more recent version?  Right now, mainline
builds already include clang-11 (oldest supported one) and
clang-16 and linux-next builds use clang-17.  So there is
upstream build coverage for them, and if clang-14 is the standard
version for Android kernels then I guess it would make sense to
stick to it - but that's up to you.  Doing both clang-14 and
clang-17 for Android builds would seem overkill to me though.

Cheers,
Guillaume

> On Wed, Sep 6, 2023 at 12:50 PM Todd Kjos <tkjos@google.com> wrote:
>>
>> +Viktor Martensson +Betty Zhou
>>
>> Guillaume,
>>
>> I think that's a great idea. I'll get back to you with the set of
>> configs, arch, compilers we'd like to have coverage for.
>>
>> -Todd
>>
>>
>> On Wed, Sep 6, 2023 at 11:46 AM Guillaume Tucker
>> <guillaume.tucker@collabora.com> wrote:
>>>
>>> Hi Todd,
>>>
>>> On 05/09/2023 19:18, Todd Kjos wrote:
>>>> On Thu, Aug 10, 2023 at 8:56 AM Todd Kjos <tkjos@google.com> wrote:
>>>>>
>>>>>
>>>>>
>>>>> On Tue, Jul 11, 2023 at 4:57 AM Guillaume Tucker <guillaume.tucker@collabora.com> wrote:
>>>>>>
>>>>>> Hi Todd,
>>>>>>
>>>>>> On 11/07/2023 00:41, Todd Kjos wrote:
>>>>>>> Please add the following new Android branches to kernelci testing.
>>>>>>>
>>>>>>> repo: https://android.googlesource.com/kernel/common
>>>>>>> branches:
>>>>>>> - android15-6.1
>>>>>>> - android14-6.1-lts
>>>>>>> - android14-5.15-lts
>>>>>>
>>>>>> I've created this PR accordingly:
>>>>>>
>>>>>>   https://github.com/kernelci/kernelci-core/pull/2002
>>>>>>
>>>>>> Please note that we're now operating with reduced build coverage
>>>>>> due to some temporary limitations with our Azure subscription.  I
>>>>>> know the Android kernel builds should be covered by the GCP
>>>>>> clusters but right now the system is designed to distribute
>>>>>> kernel builds randomly across all clusters so we can't easily tie
>>>>>> Android builds to the Android clusters.  Hopefully we'll find a
>>>>>> solution to go back to normal coverage within a week or two.
>>>>>
>>>>>
>>>>> Hi Guillaume, Any ETA for when the full set of builds is restored? We still have greatly reduced build coverage and it's been a month.
>>>>
>>>> Ping. We are getting a lot less value out of kernelci with the greatly
>>>> reduced set of builds for the Android kernels. When will the full
>>>> build coverage be resumed?
>>>>
>>>> If it needs to stay reduced, can we decide precisely which builds are
>>>> done for Android kernels?
>>>
>>> Sorry for the slow reply.
>>>
>>> I think it's fine now to re-enable the Android builds and maybe
>>> we should also take this opportunity to confirm which ones are
>>> the most relevant to you guys.  As part of the process of
>>> adjusting the build coverage, I also fixed the GKI builds which I
>>> thought were the main ones.  Even if there is enough build
>>> capacity to build "everything", keeping it streamlined to what is
>>> really useful makes the overall system more effective.
>>>
>>> Are there any particular combinations of arch, config, compilers
>>> you care about more than others?
>>>
>>> Thanks,
>>> Guillaume
>>>

Re: Please add new Android branches

[Todd Kjos]

On Mon, Sep 25, 2023 at 1:22 AM Guillaume Tucker
<guillaume.tucker@collabora.com> wrote:
>
> On 20/09/2023 00:54, Todd Kjos wrote:
> > Guillaume,
> >
> > How about this for android trees:
> >
> > 1) same compilers as current config (gcc-10, clang-14)
> > 2) archs: arm, arm64, i386, x80_64, riscv
> > 3) same configs as current android tests
> > 4) plus configs used by corresponding stable kernel
> > (https://linux.kernelci.org/job/stable) not included in #3
>
> OK thanks, I think we can set this up easily.  We'll get back to
> you if there's any ambiguity.
>
> > We get a nice benefit from sync'ing with the stable kernels since it
> > allows us to tell which issues originate upstream and which are
> > introduced by android changes. It's also useful to build each with
> > both toolchains so we can expose the cases where gcc-10 is OK but
> > clang-14 has issues (and visa-versa).
> >
> > I don't think we need to build all of the combinations if there are
> > too many builds - we could trim out some of the configs from #4 (you
> > can recommend which).
>
> OK, we can look at past results or optimise incrementally over
> the weeks to streamline the overall build coverage.
>
> > We'd also like to start in an all-green state. So let's trim out the
> > error cases that aren't caused by android kernel code. I think there
> > are some 32-bit cases that fail with the clang-14 tools and some other
> > cases that fail in stable.
> >
> > Could we start with all of the builds for #1 - #4 and then tune it up
> > after a few runs to get rid of long-failing cases and/or to just
> > reduce the load on the build service?
>
> I believe this part of the discussion is still unresolved with
> Nick, and maybe it would make more sense to address this when the
> builds get moved to the new API where we'll have more flexibility
> and more accurate log parsing etc.
>
> Another aspect that wasn't too clear in the discussions, should
> it be clang-14 or a more recent version?  Right now, mainline
> builds already include clang-11 (oldest supported one) and
> clang-16 and linux-next builds use clang-17.  So there is
> upstream build coverage for them, and if clang-14 is the standard
> version for Android kernels then I guess it would make sense to
> stick to it - but that's up to you.  Doing both clang-14 and
> clang-17 for Android builds would seem overkill to me though.

I'd be fine with tracking the latest stable clang version (clang-17
now) and dropping clang-14 so we can help the clang team find
regressions.

>
> Cheers,
> Guillaume
>
> > On Wed, Sep 6, 2023 at 12:50 PM Todd Kjos <tkjos@google.com> wrote:
> >>
> >> +Viktor Martensson +Betty Zhou
> >>
> >> Guillaume,
> >>
> >> I think that's a great idea. I'll get back to you with the set of
> >> configs, arch, compilers we'd like to have coverage for.
> >>
> >> -Todd
> >>
> >>
> >> On Wed, Sep 6, 2023 at 11:46 AM Guillaume Tucker
> >> <guillaume.tucker@collabora.com> wrote:
> >>>
> >>> Hi Todd,
> >>>
> >>> On 05/09/2023 19:18, Todd Kjos wrote:
> >>>> On Thu, Aug 10, 2023 at 8:56 AM Todd Kjos <tkjos@google.com> wrote:
> >>>>>
> >>>>>
> >>>>>
> >>>>> On Tue, Jul 11, 2023 at 4:57 AM Guillaume Tucker <guillaume.tucker@collabora.com> wrote:
> >>>>>>
> >>>>>> Hi Todd,
> >>>>>>
> >>>>>> On 11/07/2023 00:41, Todd Kjos wrote:
> >>>>>>> Please add the following new Android branches to kernelci testing.
> >>>>>>>
> >>>>>>> repo: https://android.googlesource.com/kernel/common
> >>>>>>> branches:
> >>>>>>> - android15-6.1
> >>>>>>> - android14-6.1-lts
> >>>>>>> - android14-5.15-lts
> >>>>>>
> >>>>>> I've created this PR accordingly:
> >>>>>>
> >>>>>>   https://github.com/kernelci/kernelci-core/pull/2002
> >>>>>>
> >>>>>> Please note that we're now operating with reduced build coverage
> >>>>>> due to some temporary limitations with our Azure subscription.  I
> >>>>>> know the Android kernel builds should be covered by the GCP
> >>>>>> clusters but right now the system is designed to distribute
> >>>>>> kernel builds randomly across all clusters so we can't easily tie
> >>>>>> Android builds to the Android clusters.  Hopefully we'll find a
> >>>>>> solution to go back to normal coverage within a week or two.
> >>>>>
> >>>>>
> >>>>> Hi Guillaume, Any ETA for when the full set of builds is restored? We still have greatly reduced build coverage and it's been a month.
> >>>>
> >>>> Ping. We are getting a lot less value out of kernelci with the greatly
> >>>> reduced set of builds for the Android kernels. When will the full
> >>>> build coverage be resumed?
> >>>>
> >>>> If it needs to stay reduced, can we decide precisely which builds are
> >>>> done for Android kernels?
> >>>
> >>> Sorry for the slow reply.
> >>>
> >>> I think it's fine now to re-enable the Android builds and maybe
> >>> we should also take this opportunity to confirm which ones are
> >>> the most relevant to you guys.  As part of the process of
> >>> adjusting the build coverage, I also fixed the GKI builds which I
> >>> thought were the main ones.  Even if there is enough build
> >>> capacity to build "everything", keeping it streamlined to what is
> >>> really useful makes the overall system more effective.
> >>>
> >>> Are there any particular combinations of arch, config, compilers
> >>> you care about more than others?
> >>>
> >>> Thanks,
> >>> Guillaume
> >>>
>

Re: Please add new Android branches

[Nick Desaulniers]

On Mon, Sep 25, 2023 at 1:16 AM Guillaume Tucker
<guillaume.tucker@collabora.com> wrote:
>
> On 20/09/2023 18:57, Nick Desaulniers wrote:
> > On Tue, Sep 19, 2023 at 5:12 PM Todd Kjos <tkjos@google.com> wrote:
> >>
> >> On Tue, Sep 19, 2023 at 4:19 PM Nick Desaulniers
> >> <ndesaulniers@google.com> wrote:
> >>>
> >>> Half of the red exclamation marks on that link are are "0 warnings - 0
> >>> errors".  What's up with that?
> >>
> >> This more recent build has "0 warnings - 0 errors":
> >> https://linux.kernelci.org/build/android/branch/android15-6.1/kernel/ASB-2023-07-05_14-6.1-2620-gf2d0380464d3/
> >>
> >> Here is the error (not sure why it bubbled up as 0/0):
> >>
> >>   AR      drivers/base/power/built-in.a
> >> ../fs/namei.c:2172:13: error: use of bitwise '|' with boolean operands
> >> [-Werror,-Wbitwise-instead-of-logical]
> >>         } while (!(has_zero(a, &adata, &constants) | has_zero(b,
> >> &bdata, &constants)));
> >>
> >> ~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> >>                                                    ||
> >> ../fs/namei.c:2172:13: note: cast one or both operands to int to
> >> silence this warning
> >>   CC      drivers/base/firmware_loader/builtin/main.o
> >> 1 error generated.
> >> make[3]: *** [../scripts/Makefile.build:250: fs/namei.o] Error 1
> >> make[3]: *** Waiting for unfinished jobs....
> >> make[2]: *** [../scripts/Makefile.build:500: fs] Error 2
> >
> > Guillaume,
> > Any idea what's up with that? Seems like the existence of diagnostics
> > is detected correctly; the number of warnings and errors is not.
>
> Yes I think there's some heuristics in the legacy log parsing
> tool to count the number of compiler warnings and errors.  This
> probably is outdated with Clang now, so the build exit status is
> used to detect a failure but then the stats are off.

Right, I think clang-17 changed the structure of its output to use
pipes `|` similar to GCC.

I've been keeping my eye on something called SARIF, which is some kind
of structured interchange format for diagnostics produced by compilers
and consumed by IDEs. (Pretty sure it's just JSON).  That might be
interesting to pursue in KernelCI so that the diagnostics don't need
hand rolled parsers (that can break).

>
> It's better to avoid fixing this code as it's going to be retired
> in a few months' time to be replaced with a fresh implementation
> based on the new API.  I believe some tweaks could be done to fix

Oh? Is there a ticket I can sub to to track that?

> critical issues, but if the detail of warnings and errors can be
> found even though the stats are off then maybe we can save the
> effort of fixing it?
>
> Cheers,
> Guillaume
>


-- 
Thanks,
~Nick Desaulniers

Re: Please add new Android branches

[Todd Kjos]

Guillaume,

I guess we didn't get to a complete plan... From above, can we enable these:

1) compilers: gcc-10, clang-17
2) archs: arm, arm64, i386, x80_64, riscv
3) same configs as current android tests
4) plus configs used by corresponding stable kernel
(https://linux.kernelci.org/job/stable) not included in #3


On Mon, Sep 25, 2023 at 8:17 AM Todd Kjos <tkjos@google.com> wrote:
>
> On Mon, Sep 25, 2023 at 1:22 AM Guillaume Tucker
> <guillaume.tucker@collabora.com> wrote:
> >
> > On 20/09/2023 00:54, Todd Kjos wrote:
> > > Guillaume,
> > >
> > > How about this for android trees:
> > >
> > > 1) same compilers as current config (gcc-10, clang-14)
> > > 2) archs: arm, arm64, i386, x80_64, riscv
> > > 3) same configs as current android tests
> > > 4) plus configs used by corresponding stable kernel
> > > (https://linux.kernelci.org/job/stable) not included in #3
> >
> > OK thanks, I think we can set this up easily.  We'll get back to
> > you if there's any ambiguity.
> >
> > > We get a nice benefit from sync'ing with the stable kernels since it
> > > allows us to tell which issues originate upstream and which are
> > > introduced by android changes. It's also useful to build each with
> > > both toolchains so we can expose the cases where gcc-10 is OK but
> > > clang-14 has issues (and visa-versa).
> > >
> > > I don't think we need to build all of the combinations if there are
> > > too many builds - we could trim out some of the configs from #4 (you
> > > can recommend which).
> >
> > OK, we can look at past results or optimise incrementally over
> > the weeks to streamline the overall build coverage.
> >
> > > We'd also like to start in an all-green state. So let's trim out the
> > > error cases that aren't caused by android kernel code. I think there
> > > are some 32-bit cases that fail with the clang-14 tools and some other
> > > cases that fail in stable.
> > >
> > > Could we start with all of the builds for #1 - #4 and then tune it up
> > > after a few runs to get rid of long-failing cases and/or to just
> > > reduce the load on the build service?
> >
> > I believe this part of the discussion is still unresolved with
> > Nick, and maybe it would make more sense to address this when the
> > builds get moved to the new API where we'll have more flexibility
> > and more accurate log parsing etc.
> >
> > Another aspect that wasn't too clear in the discussions, should
> > it be clang-14 or a more recent version?  Right now, mainline
> > builds already include clang-11 (oldest supported one) and
> > clang-16 and linux-next builds use clang-17.  So there is
> > upstream build coverage for them, and if clang-14 is the standard
> > version for Android kernels then I guess it would make sense to
> > stick to it - but that's up to you.  Doing both clang-14 and
> > clang-17 for Android builds would seem overkill to me though.
>
> I'd be fine with tracking the latest stable clang version (clang-17
> now) and dropping clang-14 so we can help the clang team find
> regressions.
>
> >
> > Cheers,
> > Guillaume
> >
> > > On Wed, Sep 6, 2023 at 12:50 PM Todd Kjos <tkjos@google.com> wrote:
> > >>
> > >> +Viktor Martensson +Betty Zhou
> > >>
> > >> Guillaume,
> > >>
> > >> I think that's a great idea. I'll get back to you with the set of
> > >> configs, arch, compilers we'd like to have coverage for.
> > >>
> > >> -Todd
> > >>
> > >>
> > >> On Wed, Sep 6, 2023 at 11:46 AM Guillaume Tucker
> > >> <guillaume.tucker@collabora.com> wrote:
> > >>>
> > >>> Hi Todd,
> > >>>
> > >>> On 05/09/2023 19:18, Todd Kjos wrote:
> > >>>> On Thu, Aug 10, 2023 at 8:56 AM Todd Kjos <tkjos@google.com> wrote:
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>> On Tue, Jul 11, 2023 at 4:57 AM Guillaume Tucker <guillaume.tucker@collabora.com> wrote:
> > >>>>>>
> > >>>>>> Hi Todd,
> > >>>>>>
> > >>>>>> On 11/07/2023 00:41, Todd Kjos wrote:
> > >>>>>>> Please add the following new Android branches to kernelci testing.
> > >>>>>>>
> > >>>>>>> repo: https://android.googlesource.com/kernel/common
> > >>>>>>> branches:
> > >>>>>>> - android15-6.1
> > >>>>>>> - android14-6.1-lts
> > >>>>>>> - android14-5.15-lts
> > >>>>>>
> > >>>>>> I've created this PR accordingly:
> > >>>>>>
> > >>>>>>   https://github.com/kernelci/kernelci-core/pull/2002
> > >>>>>>
> > >>>>>> Please note that we're now operating with reduced build coverage
> > >>>>>> due to some temporary limitations with our Azure subscription.  I
> > >>>>>> know the Android kernel builds should be covered by the GCP
> > >>>>>> clusters but right now the system is designed to distribute
> > >>>>>> kernel builds randomly across all clusters so we can't easily tie
> > >>>>>> Android builds to the Android clusters.  Hopefully we'll find a
> > >>>>>> solution to go back to normal coverage within a week or two.
> > >>>>>
> > >>>>>
> > >>>>> Hi Guillaume, Any ETA for when the full set of builds is restored? We still have greatly reduced build coverage and it's been a month.
> > >>>>
> > >>>> Ping. We are getting a lot less value out of kernelci with the greatly
> > >>>> reduced set of builds for the Android kernels. When will the full
> > >>>> build coverage be resumed?
> > >>>>
> > >>>> If it needs to stay reduced, can we decide precisely which builds are
> > >>>> done for Android kernels?
> > >>>
> > >>> Sorry for the slow reply.
> > >>>
> > >>> I think it's fine now to re-enable the Android builds and maybe
> > >>> we should also take this opportunity to confirm which ones are
> > >>> the most relevant to you guys.  As part of the process of
> > >>> adjusting the build coverage, I also fixed the GKI builds which I
> > >>> thought were the main ones.  Even if there is enough build
> > >>> capacity to build "everything", keeping it streamlined to what is
> > >>> really useful makes the overall system more effective.
> > >>>
> > >>> Are there any particular combinations of arch, config, compilers
> > >>> you care about more than others?
> > >>>
> > >>> Thanks,
> > >>> Guillaume
> > >>>
> >

Re: Please add new Android branches

[Todd Kjos]

Guillaume, any ETA on when we can ramp up the Android tests as
discussed on this thread?


On Tue, Oct 17, 2023 at 1:42 PM Todd Kjos <tkjos@google.com> wrote:
>
> Guillaume,
>
> I guess we didn't get to a complete plan... From above, can we enable these:
>
> 1) compilers: gcc-10, clang-17
> 2) archs: arm, arm64, i386, x80_64, riscv
> 3) same configs as current android tests
> 4) plus configs used by corresponding stable kernel
> (https://linux.kernelci.org/job/stable) not included in #3
>
>
> On Mon, Sep 25, 2023 at 8:17 AM Todd Kjos <tkjos@google.com> wrote:
> >
> > On Mon, Sep 25, 2023 at 1:22 AM Guillaume Tucker
> > <guillaume.tucker@collabora.com> wrote:
> > >
> > > On 20/09/2023 00:54, Todd Kjos wrote:
> > > > Guillaume,
> > > >
> > > > How about this for android trees:
> > > >
> > > > 1) same compilers as current config (gcc-10, clang-14)
> > > > 2) archs: arm, arm64, i386, x80_64, riscv
> > > > 3) same configs as current android tests
> > > > 4) plus configs used by corresponding stable kernel
> > > > (https://linux.kernelci.org/job/stable) not included in #3
> > >
> > > OK thanks, I think we can set this up easily.  We'll get back to
> > > you if there's any ambiguity.
> > >
> > > > We get a nice benefit from sync'ing with the stable kernels since it
> > > > allows us to tell which issues originate upstream and which are
> > > > introduced by android changes. It's also useful to build each with
> > > > both toolchains so we can expose the cases where gcc-10 is OK but
> > > > clang-14 has issues (and visa-versa).
> > > >
> > > > I don't think we need to build all of the combinations if there are
> > > > too many builds - we could trim out some of the configs from #4 (you
> > > > can recommend which).
> > >
> > > OK, we can look at past results or optimise incrementally over
> > > the weeks to streamline the overall build coverage.
> > >
> > > > We'd also like to start in an all-green state. So let's trim out the
> > > > error cases that aren't caused by android kernel code. I think there
> > > > are some 32-bit cases that fail with the clang-14 tools and some other
> > > > cases that fail in stable.
> > > >
> > > > Could we start with all of the builds for #1 - #4 and then tune it up
> > > > after a few runs to get rid of long-failing cases and/or to just
> > > > reduce the load on the build service?
> > >
> > > I believe this part of the discussion is still unresolved with
> > > Nick, and maybe it would make more sense to address this when the
> > > builds get moved to the new API where we'll have more flexibility
> > > and more accurate log parsing etc.
> > >
> > > Another aspect that wasn't too clear in the discussions, should
> > > it be clang-14 or a more recent version?  Right now, mainline
> > > builds already include clang-11 (oldest supported one) and
> > > clang-16 and linux-next builds use clang-17.  So there is
> > > upstream build coverage for them, and if clang-14 is the standard
> > > version for Android kernels then I guess it would make sense to
> > > stick to it - but that's up to you.  Doing both clang-14 and
> > > clang-17 for Android builds would seem overkill to me though.
> >
> > I'd be fine with tracking the latest stable clang version (clang-17
> > now) and dropping clang-14 so we can help the clang team find
> > regressions.
> >
> > >
> > > Cheers,
> > > Guillaume
> > >
> > > > On Wed, Sep 6, 2023 at 12:50 PM Todd Kjos <tkjos@google.com> wrote:
> > > >>
> > > >> +Viktor Martensson +Betty Zhou
> > > >>
> > > >> Guillaume,
> > > >>
> > > >> I think that's a great idea. I'll get back to you with the set of
> > > >> configs, arch, compilers we'd like to have coverage for.
> > > >>
> > > >> -Todd
> > > >>
> > > >>
> > > >> On Wed, Sep 6, 2023 at 11:46 AM Guillaume Tucker
> > > >> <guillaume.tucker@collabora.com> wrote:
> > > >>>
> > > >>> Hi Todd,
> > > >>>
> > > >>> On 05/09/2023 19:18, Todd Kjos wrote:
> > > >>>> On Thu, Aug 10, 2023 at 8:56 AM Todd Kjos <tkjos@google.com> wrote:
> > > >>>>>
> > > >>>>>
> > > >>>>>
> > > >>>>> On Tue, Jul 11, 2023 at 4:57 AM Guillaume Tucker <guillaume.tucker@collabora.com> wrote:
> > > >>>>>>
> > > >>>>>> Hi Todd,
> > > >>>>>>
> > > >>>>>> On 11/07/2023 00:41, Todd Kjos wrote:
> > > >>>>>>> Please add the following new Android branches to kernelci testing.
> > > >>>>>>>
> > > >>>>>>> repo: https://android.googlesource.com/kernel/common
> > > >>>>>>> branches:
> > > >>>>>>> - android15-6.1
> > > >>>>>>> - android14-6.1-lts
> > > >>>>>>> - android14-5.15-lts
> > > >>>>>>
> > > >>>>>> I've created this PR accordingly:
> > > >>>>>>
> > > >>>>>>   https://github.com/kernelci/kernelci-core/pull/2002
> > > >>>>>>
> > > >>>>>> Please note that we're now operating with reduced build coverage
> > > >>>>>> due to some temporary limitations with our Azure subscription.  I
> > > >>>>>> know the Android kernel builds should be covered by the GCP
> > > >>>>>> clusters but right now the system is designed to distribute
> > > >>>>>> kernel builds randomly across all clusters so we can't easily tie
> > > >>>>>> Android builds to the Android clusters.  Hopefully we'll find a
> > > >>>>>> solution to go back to normal coverage within a week or two.
> > > >>>>>
> > > >>>>>
> > > >>>>> Hi Guillaume, Any ETA for when the full set of builds is restored? We still have greatly reduced build coverage and it's been a month.
> > > >>>>
> > > >>>> Ping. We are getting a lot less value out of kernelci with the greatly
> > > >>>> reduced set of builds for the Android kernels. When will the full
> > > >>>> build coverage be resumed?
> > > >>>>
> > > >>>> If it needs to stay reduced, can we decide precisely which builds are
> > > >>>> done for Android kernels?
> > > >>>
> > > >>> Sorry for the slow reply.
> > > >>>
> > > >>> I think it's fine now to re-enable the Android builds and maybe
> > > >>> we should also take this opportunity to confirm which ones are
> > > >>> the most relevant to you guys.  As part of the process of
> > > >>> adjusting the build coverage, I also fixed the GKI builds which I
> > > >>> thought were the main ones.  Even if there is enough build
> > > >>> capacity to build "everything", keeping it streamlined to what is
> > > >>> really useful makes the overall system more effective.
> > > >>>
> > > >>> Are there any particular combinations of arch, config, compilers
> > > >>> you care about more than others?
> > > >>>
> > > >>> Thanks,
> > > >>> Guillaume
> > > >>>
> > >

Re: Please add new Android branches

[Denys Fedoryshchenko]

Hello Todd,

I've seen your request for updates on the Android tests. Guillaume is
probably swamped at the moment, so I'm jumping in to help out.

Let me know if there's anything I've missed or if you have any pointers
for me as I get up to speed:

I've updated android tree configs to use clang-17, instead of 14.
Regarding the stable kernel configurations, the differences are as
follows:

    Added 'imx_v6_v7_defconfig'
    Added 'omap2plus_defconfig'
    Added 'vexpress_defconfig'

Additionally, in our stable trees, we include 'arm64-chromebook' and
'x86-board' config fragments, which are tailored for running on actual
hardware. Would you require these as well?

We are also building kselftests, which are useful only for real
hardware tests. Should we build them for the Android trees?

Relevant PR: https://github.com/kernelci/kernelci-core/pull/2165

If it is ok, i will merge it at Sunday and enable in monday production
update.


On Thu, 2023-11-02 at 15:11 -0700, Todd Kjos wrote:
> Guillaume, any ETA on when we can ramp up the Android tests as
> discussed on this thread?
> 
> 
> On Tue, Oct 17, 2023 at 1:42 PM Todd Kjos <tkjos@google.com> wrote:
> > 
> > Guillaume,
> > 
> > I guess we didn't get to a complete plan... From above, can we
> > enable these:
> > 
> > 1) compilers: gcc-10, clang-17
> > 2) archs: arm, arm64, i386, x80_64, riscv
> > 3) same configs as current android tests
> > 4) plus configs used by corresponding stable kernel
> > (https://linux.kernelci.org/job/stable) not included in #3
> > 
> > 
> > On Mon, Sep 25, 2023 at 8:17 AM Todd Kjos <tkjos@google.com> wrote:
> > > 
> > > On Mon, Sep 25, 2023 at 1:22 AM Guillaume Tucker
> > > <guillaume.tucker@collabora.com> wrote:
> > > > 
> > > > On 20/09/2023 00:54, Todd Kjos wrote:
> > > > > Guillaume,
> > > > > 
> > > > > How about this for android trees:
> > > > > 
> > > > > 1) same compilers as current config (gcc-10, clang-14)
> > > > > 2) archs: arm, arm64, i386, x80_64, riscv
> > > > > 3) same configs as current android tests
> > > > > 4) plus configs used by corresponding stable kernel
> > > > > (https://linux.kernelci.org/job/stable) not included in #3
> > > > 
> > > > OK thanks, I think we can set this up easily.  We'll get back
> > > > to
> > > > you if there's any ambiguity.
> > > > 
> > > > > We get a nice benefit from sync'ing with the stable kernels
> > > > > since it
> > > > > allows us to tell which issues originate upstream and which
> > > > > are
> > > > > introduced by android changes. It's also useful to build each
> > > > > with
> > > > > both toolchains so we can expose the cases where gcc-10 is OK
> > > > > but
> > > > > clang-14 has issues (and visa-versa).
> > > > > 
> > > > > I don't think we need to build all of the combinations if
> > > > > there are
> > > > > too many builds - we could trim out some of the configs from
> > > > > #4 (you
> > > > > can recommend which).
> > > > 
> > > > OK, we can look at past results or optimise incrementally over
> > > > the weeks to streamline the overall build coverage.
> > > > 
> > > > > We'd also like to start in an all-green state. So let's trim
> > > > > out the
> > > > > error cases that aren't caused by android kernel code. I
> > > > > think there
> > > > > are some 32-bit cases that fail with the clang-14 tools and
> > > > > some other
> > > > > cases that fail in stable.
> > > > > 
> > > > > Could we start with all of the builds for #1 - #4 and then
> > > > > tune it up
> > > > > after a few runs to get rid of long-failing cases and/or to
> > > > > just
> > > > > reduce the load on the build service?
> > > > 
> > > > I believe this part of the discussion is still unresolved with
> > > > Nick, and maybe it would make more sense to address this when
> > > > the
> > > > builds get moved to the new API where we'll have more
> > > > flexibility
> > > > and more accurate log parsing etc.
> > > > 
> > > > Another aspect that wasn't too clear in the discussions, should
> > > > it be clang-14 or a more recent version?  Right now, mainline
> > > > builds already include clang-11 (oldest supported one) and
> > > > clang-16 and linux-next builds use clang-17.  So there is
> > > > upstream build coverage for them, and if clang-14 is the
> > > > standard
> > > > version for Android kernels then I guess it would make sense to
> > > > stick to it - but that's up to you.  Doing both clang-14 and
> > > > clang-17 for Android builds would seem overkill to me though.
> > > 
> > > I'd be fine with tracking the latest stable clang version (clang-
> > > 17
> > > now) and dropping clang-14 so we can help the clang team find
> > > regressions.
> > > 
> > > > 
> > > > Cheers,
> > > > Guillaume
> > > > 
> > > > > On Wed, Sep 6, 2023 at 12:50 PM Todd Kjos <tkjos@google.com>
> > > > > wrote:
> > > > > > 
> > > > > > +Viktor Martensson +Betty Zhou
> > > > > > 
> > > > > > Guillaume,
> > > > > > 
> > > > > > I think that's a great idea. I'll get back to you with the
> > > > > > set of
> > > > > > configs, arch, compilers we'd like to have coverage for.
> > > > > > 
> > > > > > -Todd
> > > > > > 
> > > > > > 
> > > > > > On Wed, Sep 6, 2023 at 11:46 AM Guillaume Tucker
> > > > > > <guillaume.tucker@collabora.com> wrote:
> > > > > > > 
> > > > > > > Hi Todd,
> > > > > > > 
> > > > > > > On 05/09/2023 19:18, Todd Kjos wrote:
> > > > > > > > On Thu, Aug 10, 2023 at 8:56 AM Todd Kjos
> > > > > > > > <tkjos@google.com> wrote:
> > > > > > > > > 
> > > > > > > > > 
> > > > > > > > > 
> > > > > > > > > On Tue, Jul 11, 2023 at 4:57 AM Guillaume Tucker
> > > > > > > > > <guillaume.tucker@collabora.com> wrote:
> > > > > > > > > > 
> > > > > > > > > > Hi Todd,
> > > > > > > > > > 
> > > > > > > > > > On 11/07/2023 00:41, Todd Kjos wrote:
> > > > > > > > > > > Please add the following new Android branches to
> > > > > > > > > > > kernelci testing.
> > > > > > > > > > > 
> > > > > > > > > > > repo:
> > > > > > > > > > > https://android.googlesource.com/kernel/common
> > > > > > > > > > > branches:
> > > > > > > > > > > - android15-6.1
> > > > > > > > > > > - android14-6.1-lts
> > > > > > > > > > > - android14-5.15-lts
> > > > > > > > > > 
> > > > > > > > > > I've created this PR accordingly:
> > > > > > > > > > 
> > > > > > > > > >  
> > > > > > > > > > https://github.com/kernelci/kernelci-core/pull/2002
> > > > > > > > > > 
> > > > > > > > > > Please note that we're now operating with reduced
> > > > > > > > > > build coverage
> > > > > > > > > > due to some temporary limitations with our Azure
> > > > > > > > > > subscription.  I
> > > > > > > > > > know the Android kernel builds should be covered by
> > > > > > > > > > the GCP
> > > > > > > > > > clusters but right now the system is designed to
> > > > > > > > > > distribute
> > > > > > > > > > kernel builds randomly across all clusters so we
> > > > > > > > > > can't easily tie
> > > > > > > > > > Android builds to the Android clusters.  Hopefully
> > > > > > > > > > we'll find a
> > > > > > > > > > solution to go back to normal coverage within a
> > > > > > > > > > week or two.
> > > > > > > > > 
> > > > > > > > > 
> > > > > > > > > Hi Guillaume, Any ETA for when the full set of builds
> > > > > > > > > is restored? We still have greatly reduced build
> > > > > > > > > coverage and it's been a month.
> > > > > > > > 
> > > > > > > > Ping. We are getting a lot less value out of kernelci
> > > > > > > > with the greatly
> > > > > > > > reduced set of builds for the Android kernels. When
> > > > > > > > will the full
> > > > > > > > build coverage be resumed?
> > > > > > > > 
> > > > > > > > If it needs to stay reduced, can we decide precisely
> > > > > > > > which builds are
> > > > > > > > done for Android kernels?
> > > > > > > 
> > > > > > > Sorry for the slow reply.
> > > > > > > 
> > > > > > > I think it's fine now to re-enable the Android builds and
> > > > > > > maybe
> > > > > > > we should also take this opportunity to confirm which
> > > > > > > ones are
> > > > > > > the most relevant to you guys.  As part of the process of
> > > > > > > adjusting the build coverage, I also fixed the GKI builds
> > > > > > > which I
> > > > > > > thought were the main ones.  Even if there is enough
> > > > > > > build
> > > > > > > capacity to build "everything", keeping it streamlined to
> > > > > > > what is
> > > > > > > really useful makes the overall system more effective.
> > > > > > > 
> > > > > > > Are there any particular combinations of arch, config,
> > > > > > > compilers
> > > > > > > you care about more than others?
> > > > > > > 
> > > > > > > Thanks,
> > > > > > > Guillaume
> > > > > > > 
> > > > 
> 

Re: Please add new Android branches

[Todd Kjos]

On Fri, Nov 3, 2023 at 6:57 PM Denys Fedoryshchenko
<denys.f@collabora.com> wrote:
>
> Hello Todd,
>
> I've seen your request for updates on the Android tests. Guillaume is
> probably swamped at the moment, so I'm jumping in to help out.
>
> Let me know if there's anything I've missed or if you have any pointers
> for me as I get up to speed:
>
> I've updated android tree configs to use clang-17, instead of 14.
> Regarding the stable kernel configurations, the differences are as
> follows:
>
>     Added 'imx_v6_v7_defconfig'
>     Added 'omap2plus_defconfig'
>     Added 'vexpress_defconfig'
>
> Additionally, in our stable trees, we include 'arm64-chromebook' and
> 'x86-board' config fragments, which are tailored for running on actual
> hardware. Would you require these as well?

It would be great to add this to ensure we haven't broken anything
that is supported in stable.

>
> We are also building kselftests, which are useful only for real
> hardware tests. Should we build them for the Android trees?

Yes, please do.

>
> Relevant PR: https://github.com/kernelci/kernelci-core/pull/2165
>
> If it is ok, i will merge it at Sunday and enable in monday production
> update.

Thanks Denys. I expect we will want to watch these builds for a few
weeks and probably trim out a few that we don't think are as useful
(this is part of the discussion on this thread). I'll reach out when
we are ready for that.

I look forward to seeing the increased coverage!

>
>
> On Thu, 2023-11-02 at 15:11 -0700, Todd Kjos wrote:
> > Guillaume, any ETA on when we can ramp up the Android tests as
> > discussed on this thread?
> >
> >
> > On Tue, Oct 17, 2023 at 1:42 PM Todd Kjos <tkjos@google.com> wrote:
> > >
> > > Guillaume,
> > >
> > > I guess we didn't get to a complete plan... From above, can we
> > > enable these:
> > >
> > > 1) compilers: gcc-10, clang-17
> > > 2) archs: arm, arm64, i386, x80_64, riscv
> > > 3) same configs as current android tests
> > > 4) plus configs used by corresponding stable kernel
> > > (https://linux.kernelci.org/job/stable) not included in #3
> > >
> > >
> > > On Mon, Sep 25, 2023 at 8:17 AM Todd Kjos <tkjos@google.com> wrote:
> > > >
> > > > On Mon, Sep 25, 2023 at 1:22 AM Guillaume Tucker
> > > > <guillaume.tucker@collabora.com> wrote:
> > > > >
> > > > > On 20/09/2023 00:54, Todd Kjos wrote:
> > > > > > Guillaume,
> > > > > >
> > > > > > How about this for android trees:
> > > > > >
> > > > > > 1) same compilers as current config (gcc-10, clang-14)
> > > > > > 2) archs: arm, arm64, i386, x80_64, riscv
> > > > > > 3) same configs as current android tests
> > > > > > 4) plus configs used by corresponding stable kernel
> > > > > > (https://linux.kernelci.org/job/stable) not included in #3
> > > > >
> > > > > OK thanks, I think we can set this up easily.  We'll get back
> > > > > to
> > > > > you if there's any ambiguity.
> > > > >
> > > > > > We get a nice benefit from sync'ing with the stable kernels
> > > > > > since it
> > > > > > allows us to tell which issues originate upstream and which
> > > > > > are
> > > > > > introduced by android changes. It's also useful to build each
> > > > > > with
> > > > > > both toolchains so we can expose the cases where gcc-10 is OK
> > > > > > but
> > > > > > clang-14 has issues (and visa-versa).
> > > > > >
> > > > > > I don't think we need to build all of the combinations if
> > > > > > there are
> > > > > > too many builds - we could trim out some of the configs from
> > > > > > #4 (you
> > > > > > can recommend which).
> > > > >
> > > > > OK, we can look at past results or optimise incrementally over
> > > > > the weeks to streamline the overall build coverage.
> > > > >
> > > > > > We'd also like to start in an all-green state. So let's trim
> > > > > > out the
> > > > > > error cases that aren't caused by android kernel code. I
> > > > > > think there
> > > > > > are some 32-bit cases that fail with the clang-14 tools and
> > > > > > some other
> > > > > > cases that fail in stable.
> > > > > >
> > > > > > Could we start with all of the builds for #1 - #4 and then
> > > > > > tune it up
> > > > > > after a few runs to get rid of long-failing cases and/or to
> > > > > > just
> > > > > > reduce the load on the build service?
> > > > >
> > > > > I believe this part of the discussion is still unresolved with
> > > > > Nick, and maybe it would make more sense to address this when
> > > > > the
> > > > > builds get moved to the new API where we'll have more
> > > > > flexibility
> > > > > and more accurate log parsing etc.
> > > > >
> > > > > Another aspect that wasn't too clear in the discussions, should
> > > > > it be clang-14 or a more recent version?  Right now, mainline
> > > > > builds already include clang-11 (oldest supported one) and
> > > > > clang-16 and linux-next builds use clang-17.  So there is
> > > > > upstream build coverage for them, and if clang-14 is the
> > > > > standard
> > > > > version for Android kernels then I guess it would make sense to
> > > > > stick to it - but that's up to you.  Doing both clang-14 and
> > > > > clang-17 for Android builds would seem overkill to me though.
> > > >
> > > > I'd be fine with tracking the latest stable clang version (clang-
> > > > 17
> > > > now) and dropping clang-14 so we can help the clang team find
> > > > regressions.
> > > >
> > > > >
> > > > > Cheers,
> > > > > Guillaume
> > > > >
> > > > > > On Wed, Sep 6, 2023 at 12:50 PM Todd Kjos <tkjos@google.com>
> > > > > > wrote:
> > > > > > >
> > > > > > > +Viktor Martensson +Betty Zhou
> > > > > > >
> > > > > > > Guillaume,
> > > > > > >
> > > > > > > I think that's a great idea. I'll get back to you with the
> > > > > > > set of
> > > > > > > configs, arch, compilers we'd like to have coverage for.
> > > > > > >
> > > > > > > -Todd
> > > > > > >
> > > > > > >
> > > > > > > On Wed, Sep 6, 2023 at 11:46 AM Guillaume Tucker
> > > > > > > <guillaume.tucker@collabora.com> wrote:
> > > > > > > >
> > > > > > > > Hi Todd,
> > > > > > > >
> > > > > > > > On 05/09/2023 19:18, Todd Kjos wrote:
> > > > > > > > > On Thu, Aug 10, 2023 at 8:56 AM Todd Kjos
> > > > > > > > > <tkjos@google.com> wrote:
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > On Tue, Jul 11, 2023 at 4:57 AM Guillaume Tucker
> > > > > > > > > > <guillaume.tucker@collabora.com> wrote:
> > > > > > > > > > >
> > > > > > > > > > > Hi Todd,
> > > > > > > > > > >
> > > > > > > > > > > On 11/07/2023 00:41, Todd Kjos wrote:
> > > > > > > > > > > > Please add the following new Android branches to
> > > > > > > > > > > > kernelci testing.
> > > > > > > > > > > >
> > > > > > > > > > > > repo:
> > > > > > > > > > > > https://android.googlesource.com/kernel/common
> > > > > > > > > > > > branches:
> > > > > > > > > > > > - android15-6.1
> > > > > > > > > > > > - android14-6.1-lts
> > > > > > > > > > > > - android14-5.15-lts
> > > > > > > > > > >
> > > > > > > > > > > I've created this PR accordingly:
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > https://github.com/kernelci/kernelci-core/pull/2002
> > > > > > > > > > >
> > > > > > > > > > > Please note that we're now operating with reduced
> > > > > > > > > > > build coverage
> > > > > > > > > > > due to some temporary limitations with our Azure
> > > > > > > > > > > subscription.  I
> > > > > > > > > > > know the Android kernel builds should be covered by
> > > > > > > > > > > the GCP
> > > > > > > > > > > clusters but right now the system is designed to
> > > > > > > > > > > distribute
> > > > > > > > > > > kernel builds randomly across all clusters so we
> > > > > > > > > > > can't easily tie
> > > > > > > > > > > Android builds to the Android clusters.  Hopefully
> > > > > > > > > > > we'll find a
> > > > > > > > > > > solution to go back to normal coverage within a
> > > > > > > > > > > week or two.
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > Hi Guillaume, Any ETA for when the full set of builds
> > > > > > > > > > is restored? We still have greatly reduced build
> > > > > > > > > > coverage and it's been a month.
> > > > > > > > >
> > > > > > > > > Ping. We are getting a lot less value out of kernelci
> > > > > > > > > with the greatly
> > > > > > > > > reduced set of builds for the Android kernels. When
> > > > > > > > > will the full
> > > > > > > > > build coverage be resumed?
> > > > > > > > >
> > > > > > > > > If it needs to stay reduced, can we decide precisely
> > > > > > > > > which builds are
> > > > > > > > > done for Android kernels?
> > > > > > > >
> > > > > > > > Sorry for the slow reply.
> > > > > > > >
> > > > > > > > I think it's fine now to re-enable the Android builds and
> > > > > > > > maybe
> > > > > > > > we should also take this opportunity to confirm which
> > > > > > > > ones are
> > > > > > > > the most relevant to you guys.  As part of the process of
> > > > > > > > adjusting the build coverage, I also fixed the GKI builds
> > > > > > > > which I
> > > > > > > > thought were the main ones.  Even if there is enough
> > > > > > > > build
> > > > > > > > capacity to build "everything", keeping it streamlined to
> > > > > > > > what is
> > > > > > > > really useful makes the overall system more effective.
> > > > > > > >
> > > > > > > > Are there any particular combinations of arch, config,
> > > > > > > > compilers
> > > > > > > > you care about more than others?
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > > Guillaume
> > > > > > > >
> > > > >
> >
>
>

Re: Please add new Android branches

[Guillaume Tucker]

On 06/11/2023 19:03, Todd Kjos wrote:
> On Fri, Nov 3, 2023 at 6:57 PM Denys Fedoryshchenko
> <denys.f@collabora.com> wrote:
>>
>> Hello Todd,
>>
>> I've seen your request for updates on the Android tests. Guillaume is
>> probably swamped at the moment, so I'm jumping in to help out.
>>
>> Let me know if there's anything I've missed or if you have any pointers
>> for me as I get up to speed:
>>
>> I've updated android tree configs to use clang-17, instead of 14.
>> Regarding the stable kernel configurations, the differences are as
>> follows:
>>
>>     Added 'imx_v6_v7_defconfig'
>>     Added 'omap2plus_defconfig'
>>     Added 'vexpress_defconfig'
>>
>> Additionally, in our stable trees, we include 'arm64-chromebook' and
>> 'x86-board' config fragments, which are tailored for running on actual
>> hardware. Would you require these as well?
> 
> It would be great to add this to ensure we haven't broken anything
> that is supported in stable.

I don't think it's worth adding these extra builds, the Android
kernel trees aren't really tested in LAVA labs and these
fragments are only there to enable more platforms to boot with
mainline.

>> We are also building kselftests, which are useful only for real
>> hardware tests. Should we build them for the Android trees?
> 
> Yes, please do.

It's already done, we don't need to add anything else.  The
kselftest build step is run with all builds now.  Just some
builds have more config fragments enabled to build more kselftest
binaries, but that's not useful as kselftest isn't run with
Android trees anyway.

>> Relevant PR: https://github.com/kernelci/kernelci-core/pull/2165
>>
>> If it is ok, i will merge it at Sunday and enable in monday production
>> update.
> 
> Thanks Denys. I expect we will want to watch these builds for a few
> weeks and probably trim out a few that we don't think are as useful
> (this is part of the discussion on this thread). I'll reach out when
> we are ready for that.
> 
> I look forward to seeing the increased coverage!

Probably we should remove the fragments for x86-chromebook,
x86-board and kselftest for the reasons explained above.  We
really need to make sure the build and test configs are
streamlined, what might look like more coverage can actually be
just added noise that makes the system less efficient and
increase the costs of operation.

Then as mentioned before, I'm sure you'll be glad to know that
the GKI builds got fixed in July as part of the efforts to
improve the overall signal/noise ratio.

Thanks,
Guillaume

>> On Thu, 2023-11-02 at 15:11 -0700, Todd Kjos wrote:
>>> Guillaume, any ETA on when we can ramp up the Android tests as
>>> discussed on this thread?
>>>
>>>
>>> On Tue, Oct 17, 2023 at 1:42 PM Todd Kjos <tkjos@google.com> wrote:
>>>>
>>>> Guillaume,
>>>>
>>>> I guess we didn't get to a complete plan... From above, can we
>>>> enable these:
>>>>
>>>> 1) compilers: gcc-10, clang-17
>>>> 2) archs: arm, arm64, i386, x80_64, riscv
>>>> 3) same configs as current android tests
>>>> 4) plus configs used by corresponding stable kernel
>>>> (https://linux.kernelci.org/job/stable) not included in #3
>>>>
>>>>
>>>> On Mon, Sep 25, 2023 at 8:17 AM Todd Kjos <tkjos@google.com> wrote:
>>>>>
>>>>> On Mon, Sep 25, 2023 at 1:22 AM Guillaume Tucker
>>>>> <guillaume.tucker@collabora.com> wrote:
>>>>>>
>>>>>> On 20/09/2023 00:54, Todd Kjos wrote:
>>>>>>> Guillaume,
>>>>>>>
>>>>>>> How about this for android trees:
>>>>>>>
>>>>>>> 1) same compilers as current config (gcc-10, clang-14)
>>>>>>> 2) archs: arm, arm64, i386, x80_64, riscv
>>>>>>> 3) same configs as current android tests
>>>>>>> 4) plus configs used by corresponding stable kernel
>>>>>>> (https://linux.kernelci.org/job/stable) not included in #3
>>>>>>
>>>>>> OK thanks, I think we can set this up easily.  We'll get back
>>>>>> to
>>>>>> you if there's any ambiguity.
>>>>>>
>>>>>>> We get a nice benefit from sync'ing with the stable kernels
>>>>>>> since it
>>>>>>> allows us to tell which issues originate upstream and which
>>>>>>> are
>>>>>>> introduced by android changes. It's also useful to build each
>>>>>>> with
>>>>>>> both toolchains so we can expose the cases where gcc-10 is OK
>>>>>>> but
>>>>>>> clang-14 has issues (and visa-versa).
>>>>>>>
>>>>>>> I don't think we need to build all of the combinations if
>>>>>>> there are
>>>>>>> too many builds - we could trim out some of the configs from
>>>>>>> #4 (you
>>>>>>> can recommend which).
>>>>>>
>>>>>> OK, we can look at past results or optimise incrementally over
>>>>>> the weeks to streamline the overall build coverage.
>>>>>>
>>>>>>> We'd also like to start in an all-green state. So let's trim
>>>>>>> out the
>>>>>>> error cases that aren't caused by android kernel code. I
>>>>>>> think there
>>>>>>> are some 32-bit cases that fail with the clang-14 tools and
>>>>>>> some other
>>>>>>> cases that fail in stable.
>>>>>>>
>>>>>>> Could we start with all of the builds for #1 - #4 and then
>>>>>>> tune it up
>>>>>>> after a few runs to get rid of long-failing cases and/or to
>>>>>>> just
>>>>>>> reduce the load on the build service?
>>>>>>
>>>>>> I believe this part of the discussion is still unresolved with
>>>>>> Nick, and maybe it would make more sense to address this when
>>>>>> the
>>>>>> builds get moved to the new API where we'll have more
>>>>>> flexibility
>>>>>> and more accurate log parsing etc.
>>>>>>
>>>>>> Another aspect that wasn't too clear in the discussions, should
>>>>>> it be clang-14 or a more recent version?  Right now, mainline
>>>>>> builds already include clang-11 (oldest supported one) and
>>>>>> clang-16 and linux-next builds use clang-17.  So there is
>>>>>> upstream build coverage for them, and if clang-14 is the
>>>>>> standard
>>>>>> version for Android kernels then I guess it would make sense to
>>>>>> stick to it - but that's up to you.  Doing both clang-14 and
>>>>>> clang-17 for Android builds would seem overkill to me though.
>>>>>
>>>>> I'd be fine with tracking the latest stable clang version (clang-
>>>>> 17
>>>>> now) and dropping clang-14 so we can help the clang team find
>>>>> regressions.
>>>>>
>>>>>>
>>>>>> Cheers,
>>>>>> Guillaume
>>>>>>
>>>>>>> On Wed, Sep 6, 2023 at 12:50 PM Todd Kjos <tkjos@google.com>
>>>>>>> wrote:
>>>>>>>>
>>>>>>>> +Viktor Martensson +Betty Zhou
>>>>>>>>
>>>>>>>> Guillaume,
>>>>>>>>
>>>>>>>> I think that's a great idea. I'll get back to you with the
>>>>>>>> set of
>>>>>>>> configs, arch, compilers we'd like to have coverage for.
>>>>>>>>
>>>>>>>> -Todd
>>>>>>>>
>>>>>>>>
>>>>>>>> On Wed, Sep 6, 2023 at 11:46 AM Guillaume Tucker
>>>>>>>> <guillaume.tucker@collabora.com> wrote:
>>>>>>>>>
>>>>>>>>> Hi Todd,
>>>>>>>>>
>>>>>>>>> On 05/09/2023 19:18, Todd Kjos wrote:
>>>>>>>>>> On Thu, Aug 10, 2023 at 8:56 AM Todd Kjos
>>>>>>>>>> <tkjos@google.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Tue, Jul 11, 2023 at 4:57 AM Guillaume Tucker
>>>>>>>>>>> <guillaume.tucker@collabora.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> Hi Todd,
>>>>>>>>>>>>
>>>>>>>>>>>> On 11/07/2023 00:41, Todd Kjos wrote:
>>>>>>>>>>>>> Please add the following new Android branches to
>>>>>>>>>>>>> kernelci testing.
>>>>>>>>>>>>>
>>>>>>>>>>>>> repo:
>>>>>>>>>>>>> https://android.googlesource.com/kernel/common
>>>>>>>>>>>>> branches:
>>>>>>>>>>>>> - android15-6.1
>>>>>>>>>>>>> - android14-6.1-lts
>>>>>>>>>>>>> - android14-5.15-lts
>>>>>>>>>>>>
>>>>>>>>>>>> I've created this PR accordingly:
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> https://github.com/kernelci/kernelci-core/pull/2002
>>>>>>>>>>>>
>>>>>>>>>>>> Please note that we're now operating with reduced
>>>>>>>>>>>> build coverage
>>>>>>>>>>>> due to some temporary limitations with our Azure
>>>>>>>>>>>> subscription.  I
>>>>>>>>>>>> know the Android kernel builds should be covered by
>>>>>>>>>>>> the GCP
>>>>>>>>>>>> clusters but right now the system is designed to
>>>>>>>>>>>> distribute
>>>>>>>>>>>> kernel builds randomly across all clusters so we
>>>>>>>>>>>> can't easily tie
>>>>>>>>>>>> Android builds to the Android clusters.  Hopefully
>>>>>>>>>>>> we'll find a
>>>>>>>>>>>> solution to go back to normal coverage within a
>>>>>>>>>>>> week or two.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Hi Guillaume, Any ETA for when the full set of builds
>>>>>>>>>>> is restored? We still have greatly reduced build
>>>>>>>>>>> coverage and it's been a month.
>>>>>>>>>>
>>>>>>>>>> Ping. We are getting a lot less value out of kernelci
>>>>>>>>>> with the greatly
>>>>>>>>>> reduced set of builds for the Android kernels. When
>>>>>>>>>> will the full
>>>>>>>>>> build coverage be resumed?
>>>>>>>>>>
>>>>>>>>>> If it needs to stay reduced, can we decide precisely
>>>>>>>>>> which builds are
>>>>>>>>>> done for Android kernels?
>>>>>>>>>
>>>>>>>>> Sorry for the slow reply.
>>>>>>>>>
>>>>>>>>> I think it's fine now to re-enable the Android builds and
>>>>>>>>> maybe
>>>>>>>>> we should also take this opportunity to confirm which
>>>>>>>>> ones are
>>>>>>>>> the most relevant to you guys.  As part of the process of
>>>>>>>>> adjusting the build coverage, I also fixed the GKI builds
>>>>>>>>> which I
>>>>>>>>> thought were the main ones.  Even if there is enough
>>>>>>>>> build
>>>>>>>>> capacity to build "everything", keeping it streamlined to
>>>>>>>>> what is
>>>>>>>>> really useful makes the overall system more effective.
>>>>>>>>>
>>>>>>>>> Are there any particular combinations of arch, config,
>>>>>>>>> compilers
>>>>>>>>> you care about more than others?
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Guillaume
>>>>>>>>>
>>>>>>
>>>
>>
>>

Re: Please add new Android branches

[Todd Kjos]

can we add allyesconfig and allmodconfig to the android profiles?
These have been the most fruitful at finding build issues in the past.


On Tue, Nov 7, 2023 at 12:26 AM Guillaume Tucker
<guillaume.tucker@collabora.com> wrote:
>
> On 06/11/2023 19:03, Todd Kjos wrote:
> > On Fri, Nov 3, 2023 at 6:57 PM Denys Fedoryshchenko
> > <denys.f@collabora.com> wrote:
> >>
> >> Hello Todd,
> >>
> >> I've seen your request for updates on the Android tests. Guillaume is
> >> probably swamped at the moment, so I'm jumping in to help out.
> >>
> >> Let me know if there's anything I've missed or if you have any pointers
> >> for me as I get up to speed:
> >>
> >> I've updated android tree configs to use clang-17, instead of 14.
> >> Regarding the stable kernel configurations, the differences are as
> >> follows:
> >>
> >>     Added 'imx_v6_v7_defconfig'
> >>     Added 'omap2plus_defconfig'
> >>     Added 'vexpress_defconfig'
> >>
> >> Additionally, in our stable trees, we include 'arm64-chromebook' and
> >> 'x86-board' config fragments, which are tailored for running on actual
> >> hardware. Would you require these as well?
> >
> > It would be great to add this to ensure we haven't broken anything
> > that is supported in stable.
>
> I don't think it's worth adding these extra builds, the Android
> kernel trees aren't really tested in LAVA labs and these
> fragments are only there to enable more platforms to boot with
> mainline.
>
> >> We are also building kselftests, which are useful only for real
> >> hardware tests. Should we build them for the Android trees?
> >
> > Yes, please do.
>
> It's already done, we don't need to add anything else.  The
> kselftest build step is run with all builds now.  Just some
> builds have more config fragments enabled to build more kselftest
> binaries, but that's not useful as kselftest isn't run with
> Android trees anyway.
>
> >> Relevant PR: https://github.com/kernelci/kernelci-core/pull/2165
> >>
> >> If it is ok, i will merge it at Sunday and enable in monday production
> >> update.
> >
> > Thanks Denys. I expect we will want to watch these builds for a few
> > weeks and probably trim out a few that we don't think are as useful
> > (this is part of the discussion on this thread). I'll reach out when
> > we are ready for that.
> >
> > I look forward to seeing the increased coverage!
>
> Probably we should remove the fragments for x86-chromebook,
> x86-board and kselftest for the reasons explained above.  We
> really need to make sure the build and test configs are
> streamlined, what might look like more coverage can actually be
> just added noise that makes the system less efficient and
> increase the costs of operation.
>
> Then as mentioned before, I'm sure you'll be glad to know that
> the GKI builds got fixed in July as part of the efforts to
> improve the overall signal/noise ratio.
>
> Thanks,
> Guillaume
>
> >> On Thu, 2023-11-02 at 15:11 -0700, Todd Kjos wrote:
> >>> Guillaume, any ETA on when we can ramp up the Android tests as
> >>> discussed on this thread?
> >>>
> >>>
> >>> On Tue, Oct 17, 2023 at 1:42 PM Todd Kjos <tkjos@google.com> wrote:
> >>>>
> >>>> Guillaume,
> >>>>
> >>>> I guess we didn't get to a complete plan... From above, can we
> >>>> enable these:
> >>>>
> >>>> 1) compilers: gcc-10, clang-17
> >>>> 2) archs: arm, arm64, i386, x80_64, riscv
> >>>> 3) same configs as current android tests
> >>>> 4) plus configs used by corresponding stable kernel
> >>>> (https://linux.kernelci.org/job/stable) not included in #3
> >>>>
> >>>>
> >>>> On Mon, Sep 25, 2023 at 8:17 AM Todd Kjos <tkjos@google.com> wrote:
> >>>>>
> >>>>> On Mon, Sep 25, 2023 at 1:22 AM Guillaume Tucker
> >>>>> <guillaume.tucker@collabora.com> wrote:
> >>>>>>
> >>>>>> On 20/09/2023 00:54, Todd Kjos wrote:
> >>>>>>> Guillaume,
> >>>>>>>
> >>>>>>> How about this for android trees:
> >>>>>>>
> >>>>>>> 1) same compilers as current config (gcc-10, clang-14)
> >>>>>>> 2) archs: arm, arm64, i386, x80_64, riscv
> >>>>>>> 3) same configs as current android tests
> >>>>>>> 4) plus configs used by corresponding stable kernel
> >>>>>>> (https://linux.kernelci.org/job/stable) not included in #3
> >>>>>>
> >>>>>> OK thanks, I think we can set this up easily.  We'll get back
> >>>>>> to
> >>>>>> you if there's any ambiguity.
> >>>>>>
> >>>>>>> We get a nice benefit from sync'ing with the stable kernels
> >>>>>>> since it
> >>>>>>> allows us to tell which issues originate upstream and which
> >>>>>>> are
> >>>>>>> introduced by android changes. It's also useful to build each
> >>>>>>> with
> >>>>>>> both toolchains so we can expose the cases where gcc-10 is OK
> >>>>>>> but
> >>>>>>> clang-14 has issues (and visa-versa).
> >>>>>>>
> >>>>>>> I don't think we need to build all of the combinations if
> >>>>>>> there are
> >>>>>>> too many builds - we could trim out some of the configs from
> >>>>>>> #4 (you
> >>>>>>> can recommend which).
> >>>>>>
> >>>>>> OK, we can look at past results or optimise incrementally over
> >>>>>> the weeks to streamline the overall build coverage.
> >>>>>>
> >>>>>>> We'd also like to start in an all-green state. So let's trim
> >>>>>>> out the
> >>>>>>> error cases that aren't caused by android kernel code. I
> >>>>>>> think there
> >>>>>>> are some 32-bit cases that fail with the clang-14 tools and
> >>>>>>> some other
> >>>>>>> cases that fail in stable.
> >>>>>>>
> >>>>>>> Could we start with all of the builds for #1 - #4 and then
> >>>>>>> tune it up
> >>>>>>> after a few runs to get rid of long-failing cases and/or to
> >>>>>>> just
> >>>>>>> reduce the load on the build service?
> >>>>>>
> >>>>>> I believe this part of the discussion is still unresolved with
> >>>>>> Nick, and maybe it would make more sense to address this when
> >>>>>> the
> >>>>>> builds get moved to the new API where we'll have more
> >>>>>> flexibility
> >>>>>> and more accurate log parsing etc.
> >>>>>>
> >>>>>> Another aspect that wasn't too clear in the discussions, should
> >>>>>> it be clang-14 or a more recent version?  Right now, mainline
> >>>>>> builds already include clang-11 (oldest supported one) and
> >>>>>> clang-16 and linux-next builds use clang-17.  So there is
> >>>>>> upstream build coverage for them, and if clang-14 is the
> >>>>>> standard
> >>>>>> version for Android kernels then I guess it would make sense to
> >>>>>> stick to it - but that's up to you.  Doing both clang-14 and
> >>>>>> clang-17 for Android builds would seem overkill to me though.
> >>>>>
> >>>>> I'd be fine with tracking the latest stable clang version (clang-
> >>>>> 17
> >>>>> now) and dropping clang-14 so we can help the clang team find
> >>>>> regressions.
> >>>>>
> >>>>>>
> >>>>>> Cheers,
> >>>>>> Guillaume
> >>>>>>
> >>>>>>> On Wed, Sep 6, 2023 at 12:50 PM Todd Kjos <tkjos@google.com>
> >>>>>>> wrote:
> >>>>>>>>
> >>>>>>>> +Viktor Martensson +Betty Zhou
> >>>>>>>>
> >>>>>>>> Guillaume,
> >>>>>>>>
> >>>>>>>> I think that's a great idea. I'll get back to you with the
> >>>>>>>> set of
> >>>>>>>> configs, arch, compilers we'd like to have coverage for.
> >>>>>>>>
> >>>>>>>> -Todd
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On Wed, Sep 6, 2023 at 11:46 AM Guillaume Tucker
> >>>>>>>> <guillaume.tucker@collabora.com> wrote:
> >>>>>>>>>
> >>>>>>>>> Hi Todd,
> >>>>>>>>>
> >>>>>>>>> On 05/09/2023 19:18, Todd Kjos wrote:
> >>>>>>>>>> On Thu, Aug 10, 2023 at 8:56 AM Todd Kjos
> >>>>>>>>>> <tkjos@google.com> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> On Tue, Jul 11, 2023 at 4:57 AM Guillaume Tucker
> >>>>>>>>>>> <guillaume.tucker@collabora.com> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>> Hi Todd,
> >>>>>>>>>>>>
> >>>>>>>>>>>> On 11/07/2023 00:41, Todd Kjos wrote:
> >>>>>>>>>>>>> Please add the following new Android branches to
> >>>>>>>>>>>>> kernelci testing.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> repo:
> >>>>>>>>>>>>> https://android.googlesource.com/kernel/common
> >>>>>>>>>>>>> branches:
> >>>>>>>>>>>>> - android15-6.1
> >>>>>>>>>>>>> - android14-6.1-lts
> >>>>>>>>>>>>> - android14-5.15-lts
> >>>>>>>>>>>>
> >>>>>>>>>>>> I've created this PR accordingly:
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> https://github.com/kernelci/kernelci-core/pull/2002
> >>>>>>>>>>>>
> >>>>>>>>>>>> Please note that we're now operating with reduced
> >>>>>>>>>>>> build coverage
> >>>>>>>>>>>> due to some temporary limitations with our Azure
> >>>>>>>>>>>> subscription.  I
> >>>>>>>>>>>> know the Android kernel builds should be covered by
> >>>>>>>>>>>> the GCP
> >>>>>>>>>>>> clusters but right now the system is designed to
> >>>>>>>>>>>> distribute
> >>>>>>>>>>>> kernel builds randomly across all clusters so we
> >>>>>>>>>>>> can't easily tie
> >>>>>>>>>>>> Android builds to the Android clusters.  Hopefully
> >>>>>>>>>>>> we'll find a
> >>>>>>>>>>>> solution to go back to normal coverage within a
> >>>>>>>>>>>> week or two.
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> Hi Guillaume, Any ETA for when the full set of builds
> >>>>>>>>>>> is restored? We still have greatly reduced build
> >>>>>>>>>>> coverage and it's been a month.
> >>>>>>>>>>
> >>>>>>>>>> Ping. We are getting a lot less value out of kernelci
> >>>>>>>>>> with the greatly
> >>>>>>>>>> reduced set of builds for the Android kernels. When
> >>>>>>>>>> will the full
> >>>>>>>>>> build coverage be resumed?
> >>>>>>>>>>
> >>>>>>>>>> If it needs to stay reduced, can we decide precisely
> >>>>>>>>>> which builds are
> >>>>>>>>>> done for Android kernels?
> >>>>>>>>>
> >>>>>>>>> Sorry for the slow reply.
> >>>>>>>>>
> >>>>>>>>> I think it's fine now to re-enable the Android builds and
> >>>>>>>>> maybe
> >>>>>>>>> we should also take this opportunity to confirm which
> >>>>>>>>> ones are
> >>>>>>>>> the most relevant to you guys.  As part of the process of
> >>>>>>>>> adjusting the build coverage, I also fixed the GKI builds
> >>>>>>>>> which I
> >>>>>>>>> thought were the main ones.  Even if there is enough
> >>>>>>>>> build
> >>>>>>>>> capacity to build "everything", keeping it streamlined to
> >>>>>>>>> what is
> >>>>>>>>> really useful makes the overall system more effective.
> >>>>>>>>>
> >>>>>>>>> Are there any particular combinations of arch, config,
> >>>>>>>>> compilers
> >>>>>>>>> you care about more than others?
> >>>>>>>>>
> >>>>>>>>> Thanks,
> >>>>>>>>> Guillaume
> >>>>>>>>>
> >>>>>>
> >>>
> >>
> >>
>

Re: Please add new Android branches

[Todd Kjos]

Please add the following new Android branch to kernelci testing.

repo: https://android.googlesource.com/kernel/common
branch:
- android15-6.6

Thanks,

-Todd

On Mon, Jul 10, 2023 at 3:41 PM Todd Kjos <tkjos@google.com> wrote:
>
> Please add the following new Android branches to kernelci testing.
>
> repo: https://android.googlesource.com/kernel/common
> branches:
> - android15-6.1
> - android14-6.1-lts
> - android14-5.15-lts
>
> Thanks,
>
> -Todd

Re: Please add new Android branches

[Denys Fedoryshchenko]

Hello Todd,

PR created https://github.com/kernelci/kernelci-core/pull/2345
It will be merged and included in today's production release.

Thanks!

On Sat, 2024-02-03 at 10:43 -0800, Todd Kjos wrote:
> Please add the following new Android branch to kernelci testing.
> 
> repo: https://android.googlesource.com/kernel/common
> branch:
> - android15-6.6
> 
> Thanks,
> 
> -Todd
> 
> On Mon, Jul 10, 2023 at 3:41 PM Todd Kjos <tkjos@google.com> wrote:
> > 
> > Please add the following new Android branches to kernelci testing.
> > 
> > repo: https://android.googlesource.com/kernel/common
> > branches:
> > - android15-6.1
> > - android14-6.1-lts
> > - android14-5.15-lts
> > 
> > Thanks,
> > 
> > -Todd
> 

Re: Please add new Android branches

[Todd Kjos]

I see android15-6.6 on our dashboards now. Thanks!

On Sun, Feb 4, 2024 at 10:03 PM Denys Fedoryshchenko
<denys.f@collabora.com> wrote:
>
> Hello Todd,
>
> PR created https://github.com/kernelci/kernelci-core/pull/2345
> It will be merged and included in today's production release.
>
> Thanks!
>
> On Sat, 2024-02-03 at 10:43 -0800, Todd Kjos wrote:
> > Please add the following new Android branch to kernelci testing.
> >
> > repo: https://android.googlesource.com/kernel/common
> > branch:
> > - android15-6.6
> >
> > Thanks,
> >
> > -Todd
> >
> > On Mon, Jul 10, 2023 at 3:41 PM Todd Kjos <tkjos@google.com> wrote:
> > >
> > > Please add the following new Android branches to kernelci testing.
> > >
> > > repo: https://android.googlesource.com/kernel/common
> > > branches:
> > > - android15-6.1
> > > - android14-6.1-lts
> > > - android14-5.15-lts
> > >
> > > Thanks,
> > >
> > > -Todd
> >
>
>