[U-Boot] [PATCH 00/93] dm: Move towards completing CONFIG_BLK migration

[Simon Glass <sjg@chromium.org>]

Hi Tom.

On Sun, 25 Nov 2018 at 18:12, Tom Rini <trini@konsulko.com> wrote:
>
> On Sat, Nov 24, 2018 at 12:41:53PM -0700, Simon Glass wrote:
> > Hi Tom,
> >
> > On Fri, 23 Nov 2018 at 12:38, Tom Rini <trini@konsulko.com> wrote:
> > >
> > > On Fri, Nov 23, 2018 at 05:04:46AM -0700, Simon Glass wrote:
> > > > Hi Tom,
> > > >
> > > > On Thu, 22 Nov 2018 at 16:31, Tom Rini <trini@konsulko.com> wrote:
> > > > >
> > > > > On Thu, Nov 22, 2018 at 01:50:34PM -0700, Simon Glass wrote:
> > > > > > Hi Tom,
> > > > > >
> > > > > > On Wed, 21 Nov 2018 at 08:10, Tom Rini <trini@konsulko.com> wrote:
> > > > > > >
> > > > > > > On Tue, Nov 20, 2018 at 08:53:12AM -0500, Tom Rini wrote:
> > > > > > > > On Tue, Nov 20, 2018 at 02:45:24PM +0100, Marek Vasut wrote:
> > > > > > > > > On 11/20/2018 02:42 PM, Tom Rini wrote:
> > > > > > > > > > On Tue, Nov 20, 2018 at 02:40:43PM +0100, Marek Vasut wrote:
> > > > > > > > > >> On 11/20/2018 02:37 PM, Tom Rini wrote:
> > > > > > > > > >>> On Tue, Nov 20, 2018 at 01:42:15PM +0100, Soeren Moch wrote:
> > > > > > > > > >>>>
> > > > > > > > > >>>>
> > > > > > > > > >>>> On 19.11.18 16:52, Simon Glass wrote:
> > > > > > > > > >>>>> All boards should now be migrated to use CONFIG_BLK. This series removes
> > > > > > > > > >>>>> those with build problems using this option.
> > > > > > > > > >>>>>
> > > > > > > > > >>>>> If maintainers want to keep these boards in they should send a patch in
> > > > > > > > > >>>>> the next week or two. Otherwise the board will be removed in the next
> > > > > > > > > >>>>> release, and will need to be added and re-reviewed later.
> > > > > > > > > >>>> Fabio, Stefano,
> > > > > > > > > >>>>
> > > > > > > > > >>>> it seems (almost?) all i.mx6 boards should be removed within two weeks.
> > > > > > > > > >>>> But would it not make more sense to convert the reference boards first
> > > > > > > > > >>>> (mx6sabresd
> > > > > > > > > >>>> in my case for tbs2910), and let hobbyist maintainers like me take this
> > > > > > > > > >>>> as example for
> > > > > > > > > >>>> their own modifications?
> > > > > > > > > >>>
> > > > > > > > > >>> So, I replied to the main thread earlier but no, we're not going to drop
> > > > > > > > > >>> everything in 2 weeks, especially since there's a lot of false positives
> > > > > > > > > >>> in this series.
> > > > > > > > > >>>
> > > > > > > > > >>>> Simon, Tom,
> > > > > > > > > >>>>
> > > > > > > > > >>>> is this really the usual u-boot working style to remove about hundred
> > > > > > > > > >>>> boards within
> > > > > > > > > >>>> two weeks without prior warning? As hobbyist board maintainer I try to
> > > > > > > > > >>>> follow
> > > > > > > > > >>>> new developments, and more than once I fixed up regressions introduced
> > > > > > > > > >>>> by others
> > > > > > > > > >>>> in general code.
> > > > > > > > > >>>> But I cannot follow all development details without any heads-up. And
> > > > > > > > > >>>> even the
> > > > > > > > > >>>> NXP folks seem to be surprised about this.
> > > > > > > > > >>>>
> > > > > > > > > >>>> All problems with this transition seem to be located around usbstorage
> > > > > > > > > >>>> and sata.
> > > > > > > > > >>>> This is for sure not really very board specific. Is there any migration
> > > > > > > > > >>>> guide, or
> > > > > > > > > >>>> examples how other SoC architectures did this conversion?
> > > > > > > > > >>>
> > > > > > > > > >>> I'll admit this hasn't been our best notification.  But, the deadline
> > > > > > > > > >>> was discussed about a year ago (and then no, I didn't get a build-time
> > > > > > > > > >>> warning in).  Then around v2018.05 I said it wasn't going to be a
> > > > > > > > > >>> removal type problem yet as we had a lot of boards to fixup still, and
> > > > > > > > > >>> repeated that at v2018.07.  That did lead to a lot of things getting
> > > > > > > > > >>> addressed.  But yes, we still have some large areas that after a few
> > > > > > > > > >>> years still have not been converted, and that puts me in a hard spot
> > > > > > > > > >>> too.
> > > > > > > > > >>
> > > > > > > > > >> Build time warning for a year would be good ?
> > > > > > > > > >
> > > > > > > > > > A year for this?  No.  New deadlines?  That's not too far off from what
> > > > > > > > > > we've done historically, so yes.
> > > > > > > > >
> > > > > > > > > Give people some sort of breathing space to get the conversion done.
> > > > > > > > > Stressing people out by arbitrary deadlines will lead nowhere.
> > > > > > > >
> > > > > > > > Sure, agreed.  I didn't say we're going to drop all these boards, nor
> > > > > > > > are we going to drop SATA and USB Storage (if those are still all that's
> > > > > > > > left to convert) for this release.  But given that we proposed a
> > > > > > > > deadline in August 2017, made email-but-not-build noise about it between
> > > > > > > > May and July/August of this year, no, I also don't think setting a new
> > > > > > > > deadline of November 2019 is the right call either.
> > > > > > > >
> > > > > > > > So, really, lets see what the fails to build boards are with BLK being
> > > > > > > > on when we have block devices.  Then assess what a good deadline is.
> > > > > > > >
> > > > > > > > > >> Maybe we need some generic Makefile macro to set those up.
> > > > > > > > > >
> > > > > > > > > > It would be nice, yes.  I think the problem here is (or, was) the
> > > > > > > > > > complex set of options that didn't work.
> > > > > > > > >
> > > > > > > > > The problem was many people didn't know about the conversion deadline or
> > > > > > > > > simply forgot. And reminding them with a 100-patch series removing half
> > > > > > > > > of the boards is like splashing icy water bucket in their sleeping faces.
> > > > > > > >
> > > > > > > > Alright.  But we've already tried less shocking approaches to
> > > > > > > > conversion, but in public (over the summer when Simon listed most of
> > > > > > > > these boards in a series but I _think_ his script failed to CC the
> > > > > > > > universe and didn't follow up with a repost that did email everyone) and
> > > > > > > > perhaps private too (I honestly don't recall if I did, or just intended
> > > > > > > > to, ask you about the USB side of this on IRC).
> > > > > > >
> > > > > > > And, for the record, the USB side I had in mind here was converted, and
> > > > > > > I just forgot, my fault.  In fact, as I go through some hack-and-slash
> > > > > > > to see what needs a real conversion (either board updated, or drivers
> > > > > > > updated) at least some part of this is needing to adjust dependencies to
> > > > > > > force things on with BLK.  For example, if we have MMC we must have
> > > > > > > DM_MMC and BLK, and if we have USB we must have DM_USB and BLK.
> > > > > >
> > > > > > Well, once we are through the migration we can remove BLK.
> > > > >
> > > > > I don't think BLK the symbol goes away, really.  We don't want more
> > > > > objects built unconditionally and we will continue to have block
> > > > > device-less builds.
> > > >
> > > > Yes that's right - but it becomes an optional feature rather than an
> > > > indication of completed migration.
> > > >
> > > > >
> > > > > > Yes all the block devices are related, and we should use DM for all of
> > > > > > them to make this work.
> > > > > >
> > > > > > I am not sure if there is a better way to do this with Kconfig.
> > > > > >
> > > > > > Thanks for helping with all of this. I have found it quite tricky to
> > > > > > plot a path forward which is why I am been putting it off for several
> > > > > > months.
> > > > >
> > > > > Thanks for starting it off.  I think part of the big problem we'll have
> > > > > here is that unfortunately we can't key off of the BLK migration itself
> > > > > as it's short-hand for having started using OF_CONTROL.  What I'm
> > > > > currently working through is being able to have all block drivers using
> > > > > BLK and everything is still building / linking as unconverted drivers
> > > > > now depend on BROKEN.  This is showing we have a few widely used but
> > > > > unconverted drivers over in Freescale/NXP land.
> > > >
> > > > That's a good idea. Does it work OK with DM_MMC and DM_USB? Is this a
> > > > way we can keep unconverted boards around for a while without holding
> > > > up migration? They won't work properly but will build?
> > >
> > > I have some worthwhile changes in my WIP branch I haven't yet posted,
> > > but, I think the problem is that we can't make BLK conversion the next
> > > hard deadline.  In order to make BLK (and DM) hard requirements, all of
> > > MMC needs to be switched over (along with all of USB block related and
> > > SATA, but MMC is the big one).  That in turn shows a _lot_ of different
> > > problems.  We have:
> > > - Drivers used by platforms which are using DM for other things but
> > >   aren't converted
> > > - Platforms that could be switched to using DM but haven't yet, and
> > >   sometimes their storage controller is converted and sometimes it
> > >   isn't.
> > > - What feels like almost all of PowerPC/MPC85xx at least.
> > >
> > > So I think we should maybe try is:
> > > - Pressing on the various folks that use MPC85xx/iMX to get some of
> > >   their drivers converted.  This I think will allow us to fairly soon
> > >   mark SCSI/SATA as fully converted.
> > > - I want to try re-working some of the Kconfig logic today so that we
> > >   have for example, DM_MMC pulling in BLK.
> > > - Check in further with our iMX maintainers to see what they feel is
> > >   misssing before being able to fully convert.
> > > - Check in with our Marvell maintainers to see what's missing before
> > >   they can fully convert.
> > > - Check in with our USB maintainers to see what's missing before we can
> > >   fully convert them, aside from the PowerPC issue.
> >
> > So you are suggesting a more 'bottom-up' approach where driver owners
> > do work before board owners?
>
> In essence, yes.  We _do_ have some board issues (for example,
> omap3_beagle doesn't Just Work when switched to DM_MMC, and my first
> reaction is that it probably needs to be updated to have all the various
> DTB files for the various "beagleboard" variants and updated to load the
> right one in SPL) but we have a lot more widely used drivers that need
> converting.  I tried to cover this in my RFC series today but as I also
> listed above, unless we're going to remove huge swaths of boards, we
> can't pull the trigger yet.  But I do hope this will have set off enough
> alarm bells to get this technical debt paid down a bit.

Yes I think the alarms are ringing :-) The body count is definitely
too high right now.

>
> > Anyway this seems reasonable to me. For the case where boards could be
> > converted (drivers exist) but are not, perhaps a removal patch makes
> > sense.
>
> In time, yes, after we've had the build-time warnings showing up for a
> bit.  And some boards can be easily converted, I'm pretty sure (ie all
> of sunxi).

Yes that seems quite close.

I'll take a look at some other subsystems that might be close, or need
conversion. Perhaps we should add warnings for most/all of them.

Regards,
Simon