[Ksummit-discuss] [MAINTAINERS SUMMIT] Challenges in Upstream vs. Embargoed Development in Intel Graphics.

[Ksummit-discuss] [MAINTAINERS SUMMIT] Challenges in Upstream vs. Embargoed Development in Intel Graphics.

[Rodrigo Vivi]

[-- Attachment #1: Type: text/plain, Size: 810 bytes --]

Hi there,

I've submitted a proposal for plumber's referred track. There, I want to
talk
about tools and challenges we have on embargo development vs upstream one
and
how to get focus on upstream first and upstream always mentality.

The name of plumbers proposal talk is:
"Unveiling Intel Graphics Internal Development"

I'm not sure if the talk will get accepted, but anyway I'd like to have the
chance to talk to other maintainers to exchange views on different ways of
maintaining this kind of embargo development including challenges, tools,
processes, and rules.

So I'm interested in hallway tracks of Maintainer / Kernel Summit, or maybe
a
bof session if there's interest.

Please let me know if there's interested or if further information and/or
clarification is needed.

Thanks in advance,
Rodrigo.

[-- Attachment #2: Type: text/html, Size: 1142 bytes --]

Re: [Ksummit-discuss] [MAINTAINERS SUMMIT] Challenges in Upstream vs. Embargoed Development in Intel Graphics.

[Leon Romanovsky]

[-- Attachment #1: Type: text/plain, Size: 1309 bytes --]

On Tue, Sep 04, 2018 at 12:54:16PM -0700, Rodrigo Vivi wrote:
> Hi there,
>
> I've submitted a proposal for plumber's referred track. There, I want to
> talk
> about tools and challenges we have on embargo development vs upstream one
> and
> how to get focus on upstream first and upstream always mentality.
>
> The name of plumbers proposal talk is:
> "Unveiling Intel Graphics Internal Development"
>
> I'm not sure if the talk will get accepted, but anyway I'd like to have the
> chance to talk to other maintainers to exchange views on different ways of
> maintaining this kind of embargo development including challenges, tools,
> processes, and rules.
>
> So I'm interested in hallway tracks of Maintainer / Kernel Summit, or maybe
> a
> bof session if there's interest.
>
> Please let me know if there's interested or if further information and/or
> clarification is needed.

What is "embargo development"? Are you referring to US government
restrictions or to anything else?

Also can you please explain why should we know about internal Intel
development flow?

Thanks

>
> Thanks in advance,
> Rodrigo.

> _______________________________________________
> Ksummit-discuss mailing list
> Ksummit-discuss@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/ksummit-discuss


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 801 bytes --]

Re: [Ksummit-discuss] [MAINTAINERS SUMMIT] Challenges in Upstream vs. Embargoed Development in Intel Graphics.

[Rodrigo Vivi]

On Tue, Sep 4, 2018 at 9:22 PM Leon Romanovsky <leon@kernel.org> wrote:
>
> On Tue, Sep 04, 2018 at 12:54:16PM -0700, Rodrigo Vivi wrote:
> > Hi there,
> >
> > I've submitted a proposal for plumber's referred track. There, I want to
> > talk
> > about tools and challenges we have on embargo development vs upstream one
> > and
> > how to get focus on upstream first and upstream always mentality.
> >
> > The name of plumbers proposal talk is:
> > "Unveiling Intel Graphics Internal Development"
> >
> > I'm not sure if the talk will get accepted, but anyway I'd like to have the
> > chance to talk to other maintainers to exchange views on different ways of
> > maintaining this kind of embargo development including challenges, tools,
> > processes, and rules.
> >
> > So I'm interested in hallway tracks of Maintainer / Kernel Summit, or maybe
> > a
> > bof session if there's interest.
> >
> > Please let me know if there's interested or if further information and/or
> > clarification is needed.
>
> What is "embargo development"? Are you referring to US government
> restrictions or to anything else?

No. nothing to do with government.
Embargoed by company's temporary restrictions.

Sorry for not being clear here.

>
> Also can you please explain why should we know about internal Intel
> development flow?

First of all I don't believe that we are the only one that need to
keep this kind
of flow and i915 has a very active development and we are strongly
committed with
upstream development. Our golden rules for internal development is
upstream first, upstream always.
So, maybe sharing some knowledge and lessons we learned on the past years
might be useful to someone else that might still struggle with closed source
style of development.

Also we have some challenges on keeping everything updated and
ready for upstreaming at any moment. Maybe someone else has better tools
and better procedures than us, but we are just not aware because we
never talked.

Ultimately there are some essential questions that I'd like to open to community
and ask for feedback and ideas. Some issues and gaps that cannot be solved, or
even come to a consensus on, internally.

But my hope is that we can keep that on the constructive space.

Thanks,
Rodrigo.

>
> Thanks
>
> >
> > Thanks in advance,
> > Rodrigo.
>
> > _______________________________________________
> > Ksummit-discuss mailing list
> > Ksummit-discuss@lists.linuxfoundation.org
> > https://lists.linuxfoundation.org/mailman/listinfo/ksummit-discuss
>


-- 
Rodrigo Vivi
Blog: http://blog.vivi.eng.br

Re: [Ksummit-discuss] [MAINTAINERS SUMMIT] Challenges in Upstream vs. Embargoed Development in Intel Graphics.

[Leon Romanovsky]

[-- Attachment #1: Type: text/plain, Size: 3441 bytes --]

On Tue, Sep 04, 2018 at 09:49:13PM -0700, Rodrigo Vivi wrote:
> On Tue, Sep 4, 2018 at 9:22 PM Leon Romanovsky <leon@kernel.org> wrote:
> >
> > On Tue, Sep 04, 2018 at 12:54:16PM -0700, Rodrigo Vivi wrote:
> > > Hi there,
> > >
> > > I've submitted a proposal for plumber's referred track. There, I want to
> > > talk
> > > about tools and challenges we have on embargo development vs upstream one
> > > and
> > > how to get focus on upstream first and upstream always mentality.
> > >
> > > The name of plumbers proposal talk is:
> > > "Unveiling Intel Graphics Internal Development"
> > >
> > > I'm not sure if the talk will get accepted, but anyway I'd like to have the
> > > chance to talk to other maintainers to exchange views on different ways of
> > > maintaining this kind of embargo development including challenges, tools,
> > > processes, and rules.
> > >
> > > So I'm interested in hallway tracks of Maintainer / Kernel Summit, or maybe
> > > a
> > > bof session if there's interest.
> > >
> > > Please let me know if there's interested or if further information and/or
> > > clarification is needed.
> >
> > What is "embargo development"? Are you referring to US government
> > restrictions or to anything else?
>
> No. nothing to do with government.
> Embargoed by company's temporary restrictions.
>
> Sorry for not being clear here.
>
> >
> > Also can you please explain why should we know about internal Intel
> > development flow?
>
> First of all I don't believe that we are the only one that need to
> keep this kind
> of flow and i915 has a very active development and we are strongly
> committed with
> upstream development. Our golden rules for internal development is
> upstream first, upstream always.

Good, indeed you are not alone in this attitude.

> So, maybe sharing some knowledge and lessons we learned on the past years
> might be useful to someone else that might still struggle with closed source
> style of development.

Honestly, I hardly can see how this topic fits "kernel summit" format,
but maybe I'm wrong here.

>
> Also we have some challenges on keeping everything updated and
> ready for upstreaming at any moment. Maybe someone else has better tools
> and better procedures than us, but we are just not aware because we
> never talked.

As the one who was involved in transforming Mellanox from closed-source
company to be upstream-driven company, I can assure you that all changes
are contained inside the company. They include change of responsibilities,
organization restructure, legal, HR and many more. Those changes take
a loooooot of time with enormous internal pushback.

>
> Ultimately there are some essential questions that I'd like to open to community
> and ask for feedback and ideas. Some issues and gaps that cannot be solved, or
> even come to a consensus on, internally.
>
> But my hope is that we can keep that on the constructive space.
>

First you need to raise your questions in this ML, but from the little I
saw in this thread, you are struggling with business justification to
reduce "embargo".

Thanks

> Thanks,
> Rodrigo.
>
> >
> > Thanks
> >
> > >
> > > Thanks in advance,
> > > Rodrigo.
> >
> > > _______________________________________________
> > > Ksummit-discuss mailing list
> > > Ksummit-discuss@lists.linuxfoundation.org
> > > https://lists.linuxfoundation.org/mailman/listinfo/ksummit-discuss
> >
>
>
> --
> Rodrigo Vivi
> Blog: http://blog.vivi.eng.br

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 801 bytes --]

Re: [Ksummit-discuss] [MAINTAINERS SUMMIT] Challenges in Upstream vs. Embargoed Development in Intel Graphics.

[Greg KH]

On Tue, Sep 04, 2018 at 09:49:13PM -0700, Rodrigo Vivi wrote:
> On Tue, Sep 4, 2018 at 9:22 PM Leon Romanovsky <leon@kernel.org> wrote:
> >
> > On Tue, Sep 04, 2018 at 12:54:16PM -0700, Rodrigo Vivi wrote:
> > > Hi there,
> > >
> > > I've submitted a proposal for plumber's referred track. There, I want to
> > > talk
> > > about tools and challenges we have on embargo development vs upstream one
> > > and
> > > how to get focus on upstream first and upstream always mentality.
> > >
> > > The name of plumbers proposal talk is:
> > > "Unveiling Intel Graphics Internal Development"
> > >
> > > I'm not sure if the talk will get accepted, but anyway I'd like to have the
> > > chance to talk to other maintainers to exchange views on different ways of
> > > maintaining this kind of embargo development including challenges, tools,
> > > processes, and rules.
> > >
> > > So I'm interested in hallway tracks of Maintainer / Kernel Summit, or maybe
> > > a
> > > bof session if there's interest.
> > >
> > > Please let me know if there's interested or if further information and/or
> > > clarification is needed.
> >
> > What is "embargo development"? Are you referring to US government
> > restrictions or to anything else?
> 
> No. nothing to do with government.
> Embargoed by company's temporary restrictions.
> 
> Sorry for not being clear here.

Why do we care about something like this?  It sounds like it is an Intel
issue in that they want to delay pushing stuff upstream.  Why does
upstream care about this?

> > Also can you please explain why should we know about internal Intel
> > development flow?
> 
> First of all I don't believe that we are the only one that need to
> keep this kind of flow and i915 has a very active development and we
> are strongly committed with upstream development. Our golden rules for
> internal development is upstream first, upstream always.

Great!  So if this rule runs into opposition to people within your
company, doesn't that sound like a meeting with those company members is
the best solution?

> So, maybe sharing some knowledge and lessons we learned on the past
> years might be useful to someone else that might still struggle with
> closed source style of development.

Ah, so you want to talk about how to change your process to work better
with companies that don't like doing upstream-first work?  That sounds
like a nice talk, but you need to make that a bit more clear here :)

> Also we have some challenges on keeping everything updated and ready
> for upstreaming at any moment.

"any moment"?  Don't you know this ahead of time?  If not, that sounds
like a company problem...

thanks,

greg k-h

Re: [Ksummit-discuss] [MAINTAINERS SUMMIT] Challenges in Upstream vs. Embargoed Development in Intel Graphics.

[Daniel Vetter]

On Wed, Sep 5, 2018 at 9:48 AM, Greg KH <gregkh@linuxfoundation.org> wrote:
> On Tue, Sep 04, 2018 at 09:49:13PM -0700, Rodrigo Vivi wrote:
>> On Tue, Sep 4, 2018 at 9:22 PM Leon Romanovsky <leon@kernel.org> wrote:
>> >
>> > On Tue, Sep 04, 2018 at 12:54:16PM -0700, Rodrigo Vivi wrote:
>> > > Hi there,
>> > >
>> > > I've submitted a proposal for plumber's referred track. There, I want to
>> > > talk
>> > > about tools and challenges we have on embargo development vs upstream one
>> > > and
>> > > how to get focus on upstream first and upstream always mentality.
>> > >
>> > > The name of plumbers proposal talk is:
>> > > "Unveiling Intel Graphics Internal Development"
>> > >
>> > > I'm not sure if the talk will get accepted, but anyway I'd like to have the
>> > > chance to talk to other maintainers to exchange views on different ways of
>> > > maintaining this kind of embargo development including challenges, tools,
>> > > processes, and rules.
>> > >
>> > > So I'm interested in hallway tracks of Maintainer / Kernel Summit, or maybe
>> > > a
>> > > bof session if there's interest.
>> > >
>> > > Please let me know if there's interested or if further information and/or
>> > > clarification is needed.
>> >
>> > What is "embargo development"? Are you referring to US government
>> > restrictions or to anything else?
>>
>> No. nothing to do with government.
>> Embargoed by company's temporary restrictions.
>>
>> Sorry for not being clear here.
>
> Why do we care about something like this?  It sounds like it is an Intel
> issue in that they want to delay pushing stuff upstream.  Why does
> upstream care about this?
>
>> > Also can you please explain why should we know about internal Intel
>> > development flow?
>>
>> First of all I don't believe that we are the only one that need to
>> keep this kind of flow and i915 has a very active development and we
>> are strongly committed with upstream development. Our golden rules for
>> internal development is upstream first, upstream always.
>
> Great!  So if this rule runs into opposition to people within your
> company, doesn't that sound like a meeting with those company members is
> the best solution?
>
>> So, maybe sharing some knowledge and lessons we learned on the past
>> years might be useful to someone else that might still struggle with
>> closed source style of development.
>
> Ah, so you want to talk about how to change your process to work better
> with companies that don't like doing upstream-first work?  That sounds
> like a nice talk, but you need to make that a bit more clear here :)
>
>> Also we have some challenges on keeping everything updated and ready
>> for upstreaming at any moment.
>
> "any moment"?  Don't you know this ahead of time?  If not, that sounds
> like a company problem...

The only companies which do not have this problems (at least in the
graphics space) are those which don't care one bit about upstream at
all. So yes, all the problems we have (and everyone else I've talked
to in gfx) are dealing with are direct consequences of doing
upstream-first drivers for a proprietary piece of hardware developed
behind closed doors. And until we have exclusively open source IP
hardware this problem won't go away.

This was an invitation to exchange experience in how to best deal with
the fallout - I do actually know that this is not an intel-only issue
because I chatted with non-Intel people about how they deal with this.
And I thought that figuring out how to do better upstream-first is
very much within the scope of ks/lpc, hence why I suggested Rodrigo
brings this up as a talk proposal.

Cheers, Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch

Re: [Ksummit-discuss] [MAINTAINERS SUMMIT] Challenges in Upstream vs. Embargoed Development in Intel Graphics.

[Greg KH]

On Wed, Sep 05, 2018 at 10:17:44AM +0200, Daniel Vetter wrote:
> On Wed, Sep 5, 2018 at 9:48 AM, Greg KH <gregkh@linuxfoundation.org> wrote:
> > On Tue, Sep 04, 2018 at 09:49:13PM -0700, Rodrigo Vivi wrote:
> >> On Tue, Sep 4, 2018 at 9:22 PM Leon Romanovsky <leon@kernel.org> wrote:
> >> >
> >> > On Tue, Sep 04, 2018 at 12:54:16PM -0700, Rodrigo Vivi wrote:
> >> > > Hi there,
> >> > >
> >> > > I've submitted a proposal for plumber's referred track. There, I want to
> >> > > talk
> >> > > about tools and challenges we have on embargo development vs upstream one
> >> > > and
> >> > > how to get focus on upstream first and upstream always mentality.
> >> > >
> >> > > The name of plumbers proposal talk is:
> >> > > "Unveiling Intel Graphics Internal Development"
> >> > >
> >> > > I'm not sure if the talk will get accepted, but anyway I'd like to have the
> >> > > chance to talk to other maintainers to exchange views on different ways of
> >> > > maintaining this kind of embargo development including challenges, tools,
> >> > > processes, and rules.
> >> > >
> >> > > So I'm interested in hallway tracks of Maintainer / Kernel Summit, or maybe
> >> > > a
> >> > > bof session if there's interest.
> >> > >
> >> > > Please let me know if there's interested or if further information and/or
> >> > > clarification is needed.
> >> >
> >> > What is "embargo development"? Are you referring to US government
> >> > restrictions or to anything else?
> >>
> >> No. nothing to do with government.
> >> Embargoed by company's temporary restrictions.
> >>
> >> Sorry for not being clear here.
> >
> > Why do we care about something like this?  It sounds like it is an Intel
> > issue in that they want to delay pushing stuff upstream.  Why does
> > upstream care about this?
> >
> >> > Also can you please explain why should we know about internal Intel
> >> > development flow?
> >>
> >> First of all I don't believe that we are the only one that need to
> >> keep this kind of flow and i915 has a very active development and we
> >> are strongly committed with upstream development. Our golden rules for
> >> internal development is upstream first, upstream always.
> >
> > Great!  So if this rule runs into opposition to people within your
> > company, doesn't that sound like a meeting with those company members is
> > the best solution?
> >
> >> So, maybe sharing some knowledge and lessons we learned on the past
> >> years might be useful to someone else that might still struggle with
> >> closed source style of development.
> >
> > Ah, so you want to talk about how to change your process to work better
> > with companies that don't like doing upstream-first work?  That sounds
> > like a nice talk, but you need to make that a bit more clear here :)
> >
> >> Also we have some challenges on keeping everything updated and ready
> >> for upstreaming at any moment.
> >
> > "any moment"?  Don't you know this ahead of time?  If not, that sounds
> > like a company problem...
> 
> The only companies which do not have this problems (at least in the
> graphics space) are those which don't care one bit about upstream at
> all. So yes, all the problems we have (and everyone else I've talked
> to in gfx) are dealing with are direct consequences of doing
> upstream-first drivers for a proprietary piece of hardware developed
> behind closed doors. And until we have exclusively open source IP
> hardware this problem won't go away.

Why are graphics companies "special" here somehow?  What about network
driver companies (like Intel?)  CPU manufactures?  IB controller
companies?  Any company really...

> This was an invitation to exchange experience in how to best deal with
> the fallout - I do actually know that this is not an intel-only issue
> because I chatted with non-Intel people about how they deal with this.
> And I thought that figuring out how to do better upstream-first is
> very much within the scope of ks/lpc, hence why I suggested Rodrigo
> brings this up as a talk proposal.

Ok, a "Here is how we work with pre-release hardware and upstream" type
of talk, giving suggestions for how to deal with this at a company level
would be a nice proposal.  But that was a bit hard to tease out of the
original submission here :)

thanks,

greg k-h

Re: [Ksummit-discuss] [MAINTAINERS SUMMIT] Challenges in Upstream vs. Embargoed Development in Intel Graphics.

[Daniel Vetter]

On Wed, Sep 5, 2018 at 10:31 AM, Greg KH <gregkh@linuxfoundation.org> wrote:
> On Wed, Sep 05, 2018 at 10:17:44AM +0200, Daniel Vetter wrote:
>> On Wed, Sep 5, 2018 at 9:48 AM, Greg KH <gregkh@linuxfoundation.org> wrote:
>> > On Tue, Sep 04, 2018 at 09:49:13PM -0700, Rodrigo Vivi wrote:
>> >> On Tue, Sep 4, 2018 at 9:22 PM Leon Romanovsky <leon@kernel.org> wrote:
>> >> >
>> >> > On Tue, Sep 04, 2018 at 12:54:16PM -0700, Rodrigo Vivi wrote:
>> >> > > Hi there,
>> >> > >
>> >> > > I've submitted a proposal for plumber's referred track. There, I want to
>> >> > > talk
>> >> > > about tools and challenges we have on embargo development vs upstream one
>> >> > > and
>> >> > > how to get focus on upstream first and upstream always mentality.
>> >> > >
>> >> > > The name of plumbers proposal talk is:
>> >> > > "Unveiling Intel Graphics Internal Development"
>> >> > >
>> >> > > I'm not sure if the talk will get accepted, but anyway I'd like to have the
>> >> > > chance to talk to other maintainers to exchange views on different ways of
>> >> > > maintaining this kind of embargo development including challenges, tools,
>> >> > > processes, and rules.
>> >> > >
>> >> > > So I'm interested in hallway tracks of Maintainer / Kernel Summit, or maybe
>> >> > > a
>> >> > > bof session if there's interest.
>> >> > >
>> >> > > Please let me know if there's interested or if further information and/or
>> >> > > clarification is needed.
>> >> >
>> >> > What is "embargo development"? Are you referring to US government
>> >> > restrictions or to anything else?
>> >>
>> >> No. nothing to do with government.
>> >> Embargoed by company's temporary restrictions.
>> >>
>> >> Sorry for not being clear here.
>> >
>> > Why do we care about something like this?  It sounds like it is an Intel
>> > issue in that they want to delay pushing stuff upstream.  Why does
>> > upstream care about this?
>> >
>> >> > Also can you please explain why should we know about internal Intel
>> >> > development flow?
>> >>
>> >> First of all I don't believe that we are the only one that need to
>> >> keep this kind of flow and i915 has a very active development and we
>> >> are strongly committed with upstream development. Our golden rules for
>> >> internal development is upstream first, upstream always.
>> >
>> > Great!  So if this rule runs into opposition to people within your
>> > company, doesn't that sound like a meeting with those company members is
>> > the best solution?
>> >
>> >> So, maybe sharing some knowledge and lessons we learned on the past
>> >> years might be useful to someone else that might still struggle with
>> >> closed source style of development.
>> >
>> > Ah, so you want to talk about how to change your process to work better
>> > with companies that don't like doing upstream-first work?  That sounds
>> > like a nice talk, but you need to make that a bit more clear here :)
>> >
>> >> Also we have some challenges on keeping everything updated and ready
>> >> for upstreaming at any moment.
>> >
>> > "any moment"?  Don't you know this ahead of time?  If not, that sounds
>> > like a company problem...
>>
>> The only companies which do not have this problems (at least in the
>> graphics space) are those which don't care one bit about upstream at
>> all. So yes, all the problems we have (and everyone else I've talked
>> to in gfx) are dealing with are direct consequences of doing
>> upstream-first drivers for a proprietary piece of hardware developed
>> behind closed doors. And until we have exclusively open source IP
>> hardware this problem won't go away.
>
> Why are graphics companies "special" here somehow?  What about network
> driver companies (like Intel?)  CPU manufactures?  IB controller
> companies?  Any company really...

I have no idea how this works outside of intel or graphics, I tend to
not chat with those people as much as I do with graphics people. Would
be interesting to know how it is outside of the graphics bubble
indeed.

Looking just at what intel does, this is a problem everywhere.
Graphics is a bit worse, simply because our driver stack moves faster
and is bigger (our driver is bigger than most kernel subsystems in
their entirety). So we have slightly more fancy tooling and process to
handle things. But the problem is the exact same thing for all hw
teams here at Intel. Extrapolating from what I know I'd be surprised
if this isn't a problem everywhere.

Note that the simplest solution here is to forget about upstream and
ship on LTS kernels only. Avoids all the pain we have. And that's what
pretty much everyone in the SoC/gfx space is actually doing, outside
of very few exceptions like AMD&Intel. Most of the SoC drivers you see
are reverse-engineered, so naturally they don't have to deal with
pre-production hw fun simply because they can't even get at the hw
this early in the cycle.

>> This was an invitation to exchange experience in how to best deal with
>> the fallout - I do actually know that this is not an intel-only issue
>> because I chatted with non-Intel people about how they deal with this.
>> And I thought that figuring out how to do better upstream-first is
>> very much within the scope of ks/lpc, hence why I suggested Rodrigo
>> brings this up as a talk proposal.
>
> Ok, a "Here is how we work with pre-release hardware and upstream" type
> of talk, giving suggestions for how to deal with this at a company level
> would be a nice proposal.  But that was a bit hard to tease out of the
> original submission here :)

Yeah Rodrigo's text maybe had a bit much intel-lingo in it, without
more context to explain. Still, I found it a bit suprising that you
assume a submission from the maintainers of one of the most aggressive
upstream-first driver teams would have no idea how to do upstream
development and therefore it must be their self-inflicted pain for not
doing the right thing ...

Thanks, Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch

Re: [Ksummit-discuss] [MAINTAINERS SUMMIT] Challenges in Upstream vs. Embargoed Development in Intel Graphics.

[Leon Romanovsky]

[-- Attachment #1: Type: text/plain, Size: 6772 bytes --]

On Wed, Sep 05, 2018 at 11:00:12AM +0200, Daniel Vetter wrote:
> On Wed, Sep 5, 2018 at 10:31 AM, Greg KH <gregkh@linuxfoundation.org> wrote:
> > On Wed, Sep 05, 2018 at 10:17:44AM +0200, Daniel Vetter wrote:
> >> On Wed, Sep 5, 2018 at 9:48 AM, Greg KH <gregkh@linuxfoundation.org> wrote:
> >> > On Tue, Sep 04, 2018 at 09:49:13PM -0700, Rodrigo Vivi wrote:
> >> >> On Tue, Sep 4, 2018 at 9:22 PM Leon Romanovsky <leon@kernel.org> wrote:
> >> >> >
> >> >> > On Tue, Sep 04, 2018 at 12:54:16PM -0700, Rodrigo Vivi wrote:
> >> >> > > Hi there,
> >> >> > >
> >> >> > > I've submitted a proposal for plumber's referred track. There, I want to
> >> >> > > talk
> >> >> > > about tools and challenges we have on embargo development vs upstream one
> >> >> > > and
> >> >> > > how to get focus on upstream first and upstream always mentality.
> >> >> > >
> >> >> > > The name of plumbers proposal talk is:
> >> >> > > "Unveiling Intel Graphics Internal Development"
> >> >> > >
> >> >> > > I'm not sure if the talk will get accepted, but anyway I'd like to have the
> >> >> > > chance to talk to other maintainers to exchange views on different ways of
> >> >> > > maintaining this kind of embargo development including challenges, tools,
> >> >> > > processes, and rules.
> >> >> > >
> >> >> > > So I'm interested in hallway tracks of Maintainer / Kernel Summit, or maybe
> >> >> > > a
> >> >> > > bof session if there's interest.
> >> >> > >
> >> >> > > Please let me know if there's interested or if further information and/or
> >> >> > > clarification is needed.
> >> >> >
> >> >> > What is "embargo development"? Are you referring to US government
> >> >> > restrictions or to anything else?
> >> >>
> >> >> No. nothing to do with government.
> >> >> Embargoed by company's temporary restrictions.
> >> >>
> >> >> Sorry for not being clear here.
> >> >
> >> > Why do we care about something like this?  It sounds like it is an Intel
> >> > issue in that they want to delay pushing stuff upstream.  Why does
> >> > upstream care about this?
> >> >
> >> >> > Also can you please explain why should we know about internal Intel
> >> >> > development flow?
> >> >>
> >> >> First of all I don't believe that we are the only one that need to
> >> >> keep this kind of flow and i915 has a very active development and we
> >> >> are strongly committed with upstream development. Our golden rules for
> >> >> internal development is upstream first, upstream always.
> >> >
> >> > Great!  So if this rule runs into opposition to people within your
> >> > company, doesn't that sound like a meeting with those company members is
> >> > the best solution?
> >> >
> >> >> So, maybe sharing some knowledge and lessons we learned on the past
> >> >> years might be useful to someone else that might still struggle with
> >> >> closed source style of development.
> >> >
> >> > Ah, so you want to talk about how to change your process to work better
> >> > with companies that don't like doing upstream-first work?  That sounds
> >> > like a nice talk, but you need to make that a bit more clear here :)
> >> >
> >> >> Also we have some challenges on keeping everything updated and ready
> >> >> for upstreaming at any moment.
> >> >
> >> > "any moment"?  Don't you know this ahead of time?  If not, that sounds
> >> > like a company problem...
> >>
> >> The only companies which do not have this problems (at least in the
> >> graphics space) are those which don't care one bit about upstream at
> >> all. So yes, all the problems we have (and everyone else I've talked
> >> to in gfx) are dealing with are direct consequences of doing
> >> upstream-first drivers for a proprietary piece of hardware developed
> >> behind closed doors. And until we have exclusively open source IP
> >> hardware this problem won't go away.
> >
> > Why are graphics companies "special" here somehow?  What about network
> > driver companies (like Intel?)  CPU manufactures?  IB controller
> > companies?  Any company really...
>
> I have no idea how this works outside of intel or graphics, I tend to
> not chat with those people as much as I do with graphics people. Would
> be interesting to know how it is outside of the graphics bubble
> indeed.
>
> Looking just at what intel does, this is a problem everywhere.
> Graphics is a bit worse, simply because our driver stack moves faster
> and is bigger (our driver is bigger than most kernel subsystems in
> their entirety). So we have slightly more fancy tooling and process to
> handle things. But the problem is the exact same thing for all hw
> teams here at Intel. Extrapolating from what I know I'd be surprised
> if this isn't a problem everywhere.
>
> Note that the simplest solution here is to forget about upstream and
> ship on LTS kernels only. Avoids all the pain we have. And that's what
> pretty much everyone in the SoC/gfx space is actually doing, outside
> of very few exceptions like AMD&Intel. Most of the SoC drivers you see
> are reverse-engineered, so naturally they don't have to deal with
> pre-production hw fun simply because they can't even get at the hw
> this early in the cycle.
>
> >> This was an invitation to exchange experience in how to best deal with
> >> the fallout - I do actually know that this is not an intel-only issue
> >> because I chatted with non-Intel people about how they deal with this.
> >> And I thought that figuring out how to do better upstream-first is
> >> very much within the scope of ks/lpc, hence why I suggested Rodrigo
> >> brings this up as a talk proposal.
> >
> > Ok, a "Here is how we work with pre-release hardware and upstream" type
> > of talk, giving suggestions for how to deal with this at a company level
> > would be a nice proposal.  But that was a bit hard to tease out of the
> > original submission here :)
>
> Yeah Rodrigo's text maybe had a bit much intel-lingo in it, without
> more context to explain. Still, I found it a bit suprising that you
> assume a submission from the maintainers of one of the most aggressive
> upstream-first driver teams would have no idea how to do upstream
> development and therefore it must be their self-inflicted pain for not
> doing the right thing ...

Daniel,

Sorry but it is how it is sound from the original proposal. Can you or
Rodrigo share your pain points?

In case this topic will be accepted, I would like to participate too and
share our experience with dealing with upstream-first methodology.

Thanks

>
> Thanks, Daniel
> --
> Daniel Vetter
> Software Engineer, Intel Corporation
> +41 (0) 79 365 57 48 - http://blog.ffwll.ch
> _______________________________________________
> Ksummit-discuss mailing list
> Ksummit-discuss@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/ksummit-discuss

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 801 bytes --]

Re: [Ksummit-discuss] [MAINTAINERS SUMMIT] Challenges in Upstream vs. Embargoed Development in Intel Graphics.

[Rodrigo Vivi]

[-- Attachment #1: Type: text/plain, Size: 8680 bytes --]

On Wed, Sep 5, 2018 at 2:35 AM Leon Romanovsky <leon@kernel.org> wrote:

> On Wed, Sep 05, 2018 at 11:00:12AM +0200, Daniel Vetter wrote:
> > On Wed, Sep 5, 2018 at 10:31 AM, Greg KH <gregkh@linuxfoundation.org>
> wrote:
> > > On Wed, Sep 05, 2018 at 10:17:44AM +0200, Daniel Vetter wrote:
> > >> On Wed, Sep 5, 2018 at 9:48 AM, Greg KH <gregkh@linuxfoundation.org>
> wrote:
> > >> > On Tue, Sep 04, 2018 at 09:49:13PM -0700, Rodrigo Vivi wrote:
> > >> >> On Tue, Sep 4, 2018 at 9:22 PM Leon Romanovsky <leon@kernel.org>
> wrote:
> > >> >> >
> > >> >> > On Tue, Sep 04, 2018 at 12:54:16PM -0700, Rodrigo Vivi wrote:
> > >> >> > > Hi there,
> > >> >> > >
> > >> >> > > I've submitted a proposal for plumber's referred track. There,
> I want to
> > >> >> > > talk
> > >> >> > > about tools and challenges we have on embargo development vs
> upstream one
> > >> >> > > and
> > >> >> > > how to get focus on upstream first and upstream always
> mentality.
> > >> >> > >
> > >> >> > > The name of plumbers proposal talk is:
> > >> >> > > "Unveiling Intel Graphics Internal Development"
> > >> >> > >
> > >> >> > > I'm not sure if the talk will get accepted, but anyway I'd
> like to have the
> > >> >> > > chance to talk to other maintainers to exchange views on
> different ways of
> > >> >> > > maintaining this kind of embargo development including
> challenges, tools,
> > >> >> > > processes, and rules.
> > >> >> > >
> > >> >> > > So I'm interested in hallway tracks of Maintainer / Kernel
> Summit, or maybe
> > >> >> > > a
> > >> >> > > bof session if there's interest.
> > >> >> > >
> > >> >> > > Please let me know if there's interested or if further
> information and/or
> > >> >> > > clarification is needed.
> > >> >> >
> > >> >> > What is "embargo development"? Are you referring to US government
> > >> >> > restrictions or to anything else?
> > >> >>
> > >> >> No. nothing to do with government.
> > >> >> Embargoed by company's temporary restrictions.
> > >> >>
> > >> >> Sorry for not being clear here.
> > >> >
> > >> > Why do we care about something like this?  It sounds like it is an
> Intel
> > >> > issue in that they want to delay pushing stuff upstream.  Why does
> > >> > upstream care about this?
> > >> >
> > >> >> > Also can you please explain why should we know about internal
> Intel
> > >> >> > development flow?
> > >> >>
> > >> >> First of all I don't believe that we are the only one that need to
> > >> >> keep this kind of flow and i915 has a very active development and
> we
> > >> >> are strongly committed with upstream development. Our golden rules
> for
> > >> >> internal development is upstream first, upstream always.
> > >> >
> > >> > Great!  So if this rule runs into opposition to people within your
> > >> > company, doesn't that sound like a meeting with those company
> members is
> > >> > the best solution?
> > >> >
> > >> >> So, maybe sharing some knowledge and lessons we learned on the past
> > >> >> years might be useful to someone else that might still struggle
> with
> > >> >> closed source style of development.
> > >> >
> > >> > Ah, so you want to talk about how to change your process to work
> better
> > >> > with companies that don't like doing upstream-first work?  That
> sounds
> > >> > like a nice talk, but you need to make that a bit more clear here :)
> > >> >
> > >> >> Also we have some challenges on keeping everything updated and
> ready
> > >> >> for upstreaming at any moment.
> > >> >
> > >> > "any moment"?  Don't you know this ahead of time?  If not, that
> sounds
> > >> > like a company problem...
> > >>
> > >> The only companies which do not have this problems (at least in the
> > >> graphics space) are those which don't care one bit about upstream at
> > >> all. So yes, all the problems we have (and everyone else I've talked
> > >> to in gfx) are dealing with are direct consequences of doing
> > >> upstream-first drivers for a proprietary piece of hardware developed
> > >> behind closed doors. And until we have exclusively open source IP
> > >> hardware this problem won't go away.
> > >
> > > Why are graphics companies "special" here somehow?  What about network
> > > driver companies (like Intel?)  CPU manufactures?  IB controller
> > > companies?  Any company really...
> >
> > I have no idea how this works outside of intel or graphics, I tend to
> > not chat with those people as much as I do with graphics people. Would
> > be interesting to know how it is outside of the graphics bubble
> > indeed.
> >
> > Looking just at what intel does, this is a problem everywhere.
> > Graphics is a bit worse, simply because our driver stack moves faster
> > and is bigger (our driver is bigger than most kernel subsystems in
> > their entirety). So we have slightly more fancy tooling and process to
> > handle things. But the problem is the exact same thing for all hw
> > teams here at Intel. Extrapolating from what I know I'd be surprised
> > if this isn't a problem everywhere.
> >
> > Note that the simplest solution here is to forget about upstream and
> > ship on LTS kernels only. Avoids all the pain we have. And that's what
> > pretty much everyone in the SoC/gfx space is actually doing, outside
> > of very few exceptions like AMD&Intel. Most of the SoC drivers you see
> > are reverse-engineered, so naturally they don't have to deal with
> > pre-production hw fun simply because they can't even get at the hw
> > this early in the cycle.
> >
> > >> This was an invitation to exchange experience in how to best deal with
> > >> the fallout - I do actually know that this is not an intel-only issue
> > >> because I chatted with non-Intel people about how they deal with this.
> > >> And I thought that figuring out how to do better upstream-first is
> > >> very much within the scope of ks/lpc, hence why I suggested Rodrigo
> > >> brings this up as a talk proposal.
> > >
> > > Ok, a "Here is how we work with pre-release hardware and upstream" type
> > > of talk, giving suggestions for how to deal with this at a company
> level
> > > would be a nice proposal.  But that was a bit hard to tease out of the
> > > original submission here :)
> >
> > Yeah Rodrigo's text maybe had a bit much intel-lingo in it, without
> > more context to explain. Still, I found it a bit suprising that you
> > assume a submission from the maintainers of one of the most aggressive
> > upstream-first driver teams would have no idea how to do upstream
> > development and therefore it must be their self-inflicted pain for not
> > doing the right thing ...
>
> Daniel,
>
> Sorry but it is how it is sound from the original proposal. Can you or
> Rodrigo share your pain points?
>

My fear of spoiling details here is that it would create a big endless
email discussion
like many we had internally already. But anyway I will share a bit here so
you can have more info to decide if the discussion is appropriated or not
for this summit:

1. Rebasing on top of LTS as Mark and Daniel mentioned creates a future
problem for us. Since drm moves so fast the code on top of LTS will be
likely need to be re-written sooner or later.

2. We are upstreaming everything, so we need to keep patches always ready
for upstream and track the history of each patch itself. So we need patch
of patches.

3. Quality on code review gets impacted because of the upstream focus. It
has the risk of getting forgotten or lower quality reviews like -staging
drivers Takashi mentioned.

4 . If we raise the bar demanding good reviews, once our patches from
internal started appearing on public mailing lists already contained
"Reviewed-by:" we had other issues....

For 1, 2 and 3 we believe we found the solution space and would like to
share experiences and collect feedback. However for the 4th I'd like to be
able to talk to more devs and maintainers face to face because internal
email threads were huge already but we never found a rough internal
consensus on this one.

Thanks,
Rodrigo.


> In case this topic will be accepted, I would like to participate too and
> share our experience with dealing with upstream-first methodology.
>
> Thanks
>
> >
> > Thanks, Daniel
> > --
> > Daniel Vetter
> > Software Engineer, Intel Corporation
> > +41 (0) 79 365 57 48 <+41%2079%20365%2057%2048> - http://blog.ffwll.ch
> > _______________________________________________
> > Ksummit-discuss mailing list
> > Ksummit-discuss@lists.linuxfoundation.org
> > https://lists.linuxfoundation.org/mailman/listinfo/ksummit-discuss
> _______________________________________________
> Ksummit-discuss mailing list
> Ksummit-discuss@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/ksummit-discuss
>

[-- Attachment #2: Type: text/html, Size: 11875 bytes --]

Re: [Ksummit-discuss] [MAINTAINERS SUMMIT] Challenges in Upstream vs. Embargoed Development in Intel Graphics.

[Leon Romanovsky]

[-- Attachment #1: Type: text/plain, Size: 9860 bytes --]

On Wed, Sep 05, 2018 at 03:45:41PM -0700, Rodrigo Vivi wrote:
> On Wed, Sep 5, 2018 at 2:35 AM Leon Romanovsky <leon@kernel.org> wrote:
>
> > On Wed, Sep 05, 2018 at 11:00:12AM +0200, Daniel Vetter wrote:
> > > On Wed, Sep 5, 2018 at 10:31 AM, Greg KH <gregkh@linuxfoundation.org>
> > wrote:
> > > > On Wed, Sep 05, 2018 at 10:17:44AM +0200, Daniel Vetter wrote:
> > > >> On Wed, Sep 5, 2018 at 9:48 AM, Greg KH <gregkh@linuxfoundation.org>
> > wrote:
> > > >> > On Tue, Sep 04, 2018 at 09:49:13PM -0700, Rodrigo Vivi wrote:
> > > >> >> On Tue, Sep 4, 2018 at 9:22 PM Leon Romanovsky <leon@kernel.org>
> > wrote:
> > > >> >> >
> > > >> >> > On Tue, Sep 04, 2018 at 12:54:16PM -0700, Rodrigo Vivi wrote:
> > > >> >> > > Hi there,
> > > >> >> > >
> > > >> >> > > I've submitted a proposal for plumber's referred track. There,
> > I want to
> > > >> >> > > talk
> > > >> >> > > about tools and challenges we have on embargo development vs
> > upstream one
> > > >> >> > > and
> > > >> >> > > how to get focus on upstream first and upstream always
> > mentality.
> > > >> >> > >
> > > >> >> > > The name of plumbers proposal talk is:
> > > >> >> > > "Unveiling Intel Graphics Internal Development"
> > > >> >> > >
> > > >> >> > > I'm not sure if the talk will get accepted, but anyway I'd
> > like to have the
> > > >> >> > > chance to talk to other maintainers to exchange views on
> > different ways of
> > > >> >> > > maintaining this kind of embargo development including
> > challenges, tools,
> > > >> >> > > processes, and rules.
> > > >> >> > >
> > > >> >> > > So I'm interested in hallway tracks of Maintainer / Kernel
> > Summit, or maybe
> > > >> >> > > a
> > > >> >> > > bof session if there's interest.
> > > >> >> > >
> > > >> >> > > Please let me know if there's interested or if further
> > information and/or
> > > >> >> > > clarification is needed.
> > > >> >> >
> > > >> >> > What is "embargo development"? Are you referring to US government
> > > >> >> > restrictions or to anything else?
> > > >> >>
> > > >> >> No. nothing to do with government.
> > > >> >> Embargoed by company's temporary restrictions.
> > > >> >>
> > > >> >> Sorry for not being clear here.
> > > >> >
> > > >> > Why do we care about something like this?  It sounds like it is an
> > Intel
> > > >> > issue in that they want to delay pushing stuff upstream.  Why does
> > > >> > upstream care about this?
> > > >> >
> > > >> >> > Also can you please explain why should we know about internal
> > Intel
> > > >> >> > development flow?
> > > >> >>
> > > >> >> First of all I don't believe that we are the only one that need to
> > > >> >> keep this kind of flow and i915 has a very active development and
> > we
> > > >> >> are strongly committed with upstream development. Our golden rules
> > for
> > > >> >> internal development is upstream first, upstream always.
> > > >> >
> > > >> > Great!  So if this rule runs into opposition to people within your
> > > >> > company, doesn't that sound like a meeting with those company
> > members is
> > > >> > the best solution?
> > > >> >
> > > >> >> So, maybe sharing some knowledge and lessons we learned on the past
> > > >> >> years might be useful to someone else that might still struggle
> > with
> > > >> >> closed source style of development.
> > > >> >
> > > >> > Ah, so you want to talk about how to change your process to work
> > better
> > > >> > with companies that don't like doing upstream-first work?  That
> > sounds
> > > >> > like a nice talk, but you need to make that a bit more clear here :)
> > > >> >
> > > >> >> Also we have some challenges on keeping everything updated and
> > ready
> > > >> >> for upstreaming at any moment.
> > > >> >
> > > >> > "any moment"?  Don't you know this ahead of time?  If not, that
> > sounds
> > > >> > like a company problem...
> > > >>
> > > >> The only companies which do not have this problems (at least in the
> > > >> graphics space) are those which don't care one bit about upstream at
> > > >> all. So yes, all the problems we have (and everyone else I've talked
> > > >> to in gfx) are dealing with are direct consequences of doing
> > > >> upstream-first drivers for a proprietary piece of hardware developed
> > > >> behind closed doors. And until we have exclusively open source IP
> > > >> hardware this problem won't go away.
> > > >
> > > > Why are graphics companies "special" here somehow?  What about network
> > > > driver companies (like Intel?)  CPU manufactures?  IB controller
> > > > companies?  Any company really...
> > >
> > > I have no idea how this works outside of intel or graphics, I tend to
> > > not chat with those people as much as I do with graphics people. Would
> > > be interesting to know how it is outside of the graphics bubble
> > > indeed.
> > >
> > > Looking just at what intel does, this is a problem everywhere.
> > > Graphics is a bit worse, simply because our driver stack moves faster
> > > and is bigger (our driver is bigger than most kernel subsystems in
> > > their entirety). So we have slightly more fancy tooling and process to
> > > handle things. But the problem is the exact same thing for all hw
> > > teams here at Intel. Extrapolating from what I know I'd be surprised
> > > if this isn't a problem everywhere.
> > >
> > > Note that the simplest solution here is to forget about upstream and
> > > ship on LTS kernels only. Avoids all the pain we have. And that's what
> > > pretty much everyone in the SoC/gfx space is actually doing, outside
> > > of very few exceptions like AMD&Intel. Most of the SoC drivers you see
> > > are reverse-engineered, so naturally they don't have to deal with
> > > pre-production hw fun simply because they can't even get at the hw
> > > this early in the cycle.
> > >
> > > >> This was an invitation to exchange experience in how to best deal with
> > > >> the fallout - I do actually know that this is not an intel-only issue
> > > >> because I chatted with non-Intel people about how they deal with this.
> > > >> And I thought that figuring out how to do better upstream-first is
> > > >> very much within the scope of ks/lpc, hence why I suggested Rodrigo
> > > >> brings this up as a talk proposal.
> > > >
> > > > Ok, a "Here is how we work with pre-release hardware and upstream" type
> > > > of talk, giving suggestions for how to deal with this at a company
> > level
> > > > would be a nice proposal.  But that was a bit hard to tease out of the
> > > > original submission here :)
> > >
> > > Yeah Rodrigo's text maybe had a bit much intel-lingo in it, without
> > > more context to explain. Still, I found it a bit suprising that you
> > > assume a submission from the maintainers of one of the most aggressive
> > > upstream-first driver teams would have no idea how to do upstream
> > > development and therefore it must be their self-inflicted pain for not
> > > doing the right thing ...
> >
> > Daniel,
> >
> > Sorry but it is how it is sound from the original proposal. Can you or
> > Rodrigo share your pain points?
> >
>
> My fear of spoiling details here is that it would create a big endless
> email discussion

Thanks for sharing

> like many we had internally already. But anyway I will share a bit here so
> you can have more info to decide if the discussion is appropriated or not
> for this summit:
>
> 1. Rebasing on top of LTS as Mark and Daniel mentioned creates a future
> problem for us. Since drm moves so fast the code on top of LTS will be
> likely need to be re-written sooner or later.
>

We solved it a long time ago by deciding to base our code more or less on
latest -rc* from Linus.

> 2. We are upstreaming everything, so we need to keep patches always ready
> for upstream and track the history of each patch itself. So we need patch
> of patches.

We solved it by internal restructure and put several people to be
responsible for upstreaming (actual sending of patches) and encouraged
actual developers whose patches were send to respond to comments if needed.

BTW, those upstreaming people are regular developers, just more focused.

Once patches are accepted, we rebase our code.

It removed from us need to track patches.

>
> 3. Quality on code review gets impacted because of the upstream focus. It
> has the risk of getting forgotten or lower quality reviews like -staging
> drivers Takashi mentioned.

We (RDMA community) decided do not invest any effort in staging, mainly
because having constant nightmare with lustre.

>
> 4 . If we raise the bar demanding good reviews, once our patches from
> internal started appearing on public mailing lists already contained
> "Reviewed-by:" we had other issues....
>

I don't see any problem with that. People invested time, it is our way
to say our thanks to them.

> For 1, 2 and 3 we believe we found the solution space and would like to
> share experiences and collect feedback. However for the 4th I'd like to be
> able to talk to more devs and maintainers face to face because internal
> email threads were huge already but we never found a rough internal
> consensus on this one.
>
> Thanks,
> Rodrigo.
>
>
> > In case this topic will be accepted, I would like to participate too and
> > share our experience with dealing with upstream-first methodology.
> >
> > Thanks
> >
> > >
> > > Thanks, Daniel
> > > --
> > > Daniel Vetter
> > > Software Engineer, Intel Corporation
> > > +41 (0) 79 365 57 48 <+41%2079%20365%2057%2048> - http://blog.ffwll.ch
> > > _______________________________________________
> > > Ksummit-discuss mailing list
> > > Ksummit-discuss@lists.linuxfoundation.org
> > > https://lists.linuxfoundation.org/mailman/listinfo/ksummit-discuss
> > _______________________________________________
> > Ksummit-discuss mailing list
> > Ksummit-discuss@lists.linuxfoundation.org
> > https://lists.linuxfoundation.org/mailman/listinfo/ksummit-discuss
> >

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 801 bytes --]

Re: [Ksummit-discuss] [MAINTAINERS SUMMIT] Challenges in Upstream vs. Embargoed Development in Intel Graphics.

[Mark Brown]

[-- Attachment #1: Type: text/plain, Size: 1014 bytes --]

On Wed, Sep 05, 2018 at 11:00:12AM +0200, Daniel Vetter wrote:

> Note that the simplest solution here is to forget about upstream and
> ship on LTS kernels only. Avoids all the pain we have. And that's what
> pretty much everyone in the SoC/gfx space is actually doing, outside
> of very few exceptions like AMD&Intel. Most of the SoC drivers you see
> are reverse-engineered, so naturally they don't have to deal with
> pre-production hw fun simply because they can't even get at the hw
> this early in the cycle.

One thing I've done in the past and I know some people do currently is
to use a LTS backport as an integration point for internal testing and
development but simultaneously maintain something against -next as the
primary development platform, upstreaming anything that can be (things
like framework changes or fixes/improvements to already upstream code)
as you go.  Then when the marketing people do their thing just push out
all the patches that were held back.  That seems to work pretty well.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [Ksummit-discuss] [MAINTAINERS SUMMIT] Challenges in Upstream vs. Embargoed Development in Intel Graphics.

[Linus Walleij]

On Wed, Sep 5, 2018 at 11:00 AM Daniel Vetter <daniel.vetter@ffwll.ch> wrote:

> I have no idea how this works outside of intel or graphics, I tend to
> not chat with those people as much as I do with graphics people. Would
> be interesting to know how it is outside of the graphics bubble
> indeed.

More or less any SoC company for routers, handsets, tablets etc
have the same problem.

At one point I was made responsible for such a scenario. The
approach I developed was a bit ad hoc but contained some of this:

- Classify the components into embargoed and non-embargoed
  so anything not affected by embargo can be pushed upstream.
  (OK it's maybe obvious).

- Get management to provide a cut-off date for embargo. Like
  "after this point in time we certainly do not care what code you
  publish pertaining to X" and make that formal so that when that
  day comes developers can simply start sending the code without
  having to ask permission again, because having to do that is
  pointless and bureaucratic. This is a property of the company
  "FOSS-OUT legal process" that is simple but still often
  forgotten. It relieves the developers for the need to hammer
  management for approvals.

- Use internal developers with high upstream experience and
  NDA to make internal reviews and try to anticipate any problems
  that will be seen when posting the code to the community and
  fix it before it happens. Get the code in upstream shape.
  (This is a good reason to hire kernel maintainers and have them
  spend time upstream BTW.)

- Minimize hamming distance to mainline, which means rebase
  internal development as often as possible, track mainline,
  bring in entire branches of development if need be just to not
  deviate, because deviating too much from mainline is inviting
  disaster. This goes counter to the conventional wisdom that
  says you should use stable releases and baselines and LTS
  and what not. I am not a big fan of the latter. Rebase on
  Torvald's -rcN or even linux-next if you are aiming for upstream,
  else you are not really aiming for upstream, but secondary
  goals such as feature completion, "not disturbing development",
  or getting tests to pass cleanly all the time. Get your priorities
  straight: is upstream first really what you want? Then that
  should be priority number one, not feature completion, not
  "not disturbing developers", not passing internal tests.
  If you find upstream first isn't really your number one priority
  then call your strategy upstream second or upstream
  third so that it reflects reality instead of trying to sound good.

The last point was always the most controversial.

Just my €0.01
Linus Walleij

Re: [Ksummit-discuss] [MAINTAINERS SUMMIT] Challenges in Upstream vs. Embargoed Development in Intel Graphics.

[Jani Nikula]

On Thu, 06 Sep 2018, Linus Walleij <linus.walleij@linaro.org> wrote:
> On Wed, Sep 5, 2018 at 11:00 AM Daniel Vetter <daniel.vetter@ffwll.ch> wrote:
>
>> I have no idea how this works outside of intel or graphics, I tend to
>> not chat with those people as much as I do with graphics people. Would
>> be interesting to know how it is outside of the graphics bubble
>> indeed.
>
> More or less any SoC company for routers, handsets, tablets etc
> have the same problem.
>
> At one point I was made responsible for such a scenario. The
> approach I developed was a bit ad hoc but contained some of this:
>
> - Classify the components into embargoed and non-embargoed
>   so anything not affected by embargo can be pushed upstream.
>   (OK it's maybe obvious).

It should be obvious, but given the dual mode of operation, people tend
to err on the side of caution, and you need to constantly have an eye
out for stuff that should be refactored and sent to upstream directly.

> - Get management to provide a cut-off date for embargo. Like
>   "after this point in time we certainly do not care what code you
>   publish pertaining to X" and make that formal so that when that
>   day comes developers can simply start sending the code without
>   having to ask permission again, because having to do that is
>   pointless and bureaucratic. This is a property of the company
>   "FOSS-OUT legal process" that is simple but still often
>   forgotten. It relieves the developers for the need to hammer
>   management for approvals.
>
> - Use internal developers with high upstream experience and
>   NDA to make internal reviews and try to anticipate any problems
>   that will be seen when posting the code to the community and
>   fix it before it happens. Get the code in upstream shape.
>   (This is a good reason to hire kernel maintainers and have them
>   spend time upstream BTW.)

Wholeheartedly agreed.

> - Minimize hamming distance to mainline, which means rebase
>   internal development as often as possible, track mainline,
>   bring in entire branches of development if need be just to not
>   deviate, because deviating too much from mainline is inviting
>   disaster. This goes counter to the conventional wisdom that
>   says you should use stable releases and baselines and LTS
>   and what not. I am not a big fan of the latter. Rebase on
>   Torvald's -rcN or even linux-next if you are aiming for upstream,
>   else you are not really aiming for upstream, but secondary
>   goals such as feature completion, "not disturbing development",
>   or getting tests to pass cleanly all the time. Get your priorities
>   straight: is upstream first really what you want? Then that
>   should be priority number one, not feature completion, not
>   "not disturbing developers", not passing internal tests.
>   If you find upstream first isn't really your number one priority
>   then call your strategy upstream second or upstream
>   third so that it reflects reality instead of trying to sound good.

We're constantly rebasing on our public bleeding edge development
branch, so we're with you here. You also emphasize internal review, but
there's a conflict: if you review the first version and rebase it a
dozen times, it really no longer is the reviewed version. And since you
expect rebases (potentially with conflicts), how polished should you
require the internal patches to be?

These are the kind of problems that I expect Rodrigo to cover in the
presentation, as well as some of our time tested solutions. Basically
ideas how to gear your internal development towards "upstream as soon as
you can talk about it".

BR,
Jani.


-- 
Jani Nikula, Intel Open Source Graphics Center

Re: [Ksummit-discuss] [MAINTAINERS SUMMIT] Challenges in Upstream vs. Embargoed Development in Intel Graphics.

[Mark Brown]

[-- Attachment #1: Type: text/plain, Size: 767 bytes --]

On Thu, Sep 06, 2018 at 01:15:53PM +0300, Jani Nikula wrote:
> On Thu, 06 Sep 2018, Linus Walleij <linus.walleij@linaro.org> wrote:

> > - Classify the components into embargoed and non-embargoed
> >   so anything not affected by embargo can be pushed upstream.
> >   (OK it's maybe obvious).

> It should be obvious, but given the dual mode of operation, people tend
> to err on the side of caution, and you need to constantly have an eye
> out for stuff that should be refactored and sent to upstream directly.

I suspect this is harder for things like the i915 driver where divisions
are less clear but it's usually fairly easy for things like SoCs or more
standalone chips where you're typically working with completely separate
drivers that are easy to segment.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [Ksummit-discuss] [MAINTAINERS SUMMIT] Challenges in Upstream vs. Embargoed Development in Intel Graphics.

[Arnd Bergmann]

On Thu, Sep 6, 2018 at 11:54 AM Linus Walleij <linus.walleij@linaro.org> wrote:
>
> On Wed, Sep 5, 2018 at 11:00 AM Daniel Vetter <daniel.vetter@ffwll.ch> wrote:
>
> > I have no idea how this works outside of intel or graphics, I tend to
> > not chat with those people as much as I do with graphics people. Would
> > be interesting to know how it is outside of the graphics bubble
> > indeed.
>
> More or less any SoC company for routers, handsets, tablets etc
> have the same problem.
>
> At one point I was made responsible for such a scenario. The
> approach I developed was a bit ad hoc but contained some of this:

Thanks for sharing those.

> - Get management to provide a cut-off date for embargo. Like
>   "after this point in time we certainly do not care what code you
>   publish pertaining to X" and make that formal so that when that
>   day comes developers can simply start sending the code without
>   having to ask permission again, because having to do that is
>   pointless and bureaucratic.

I think in the generalized case, you also want the reverse, but
that may be harder: When targetting specific software products
that you want to integrate your code, there should be a deadline
for the latest point by which code needs to be posted in
public. That is mainly what I remember from working on
enterprise hardware in a previous job: you are either based
on a distro schedule (needs to be posted/reviewed/integrated
by date X to have a chance to make it into future product Y),
or on a hardware schedule (needs to be posted X months
ahead of HW product launch to have a reasonable chance
of making it into software products by the time the hardware
makes it into customer's hands).

Having both the earliest date (for information embargo) and
latest date (for software integration) gives you a nice way to
estimate how likely the software is going to make it in time.

       Arnd

Re: [Ksummit-discuss] [MAINTAINERS SUMMIT] Challenges in Upstream vs. Embargoed Development in Intel Graphics.

[Linus Walleij]

On Thu, Sep 6, 2018 at 12:25 PM Arnd Bergmann <arnd@arndb.de> wrote:

> I think in the generalized case, you also want the reverse, but
> that may be harder: When targetting specific software products
> that you want to integrate your code, there should be a deadline
> for the latest point by which code needs to be posted in
> public.

This brings in the process of procurement, as in how companies
making products source their misc hardware like sensors,
touchscreens, displays, FPGAs or whatnot.

Maybe this is obvious.

It happened at one point that we were sourcing hardware from
a third party, and it was pretty complex and I asked procurement
to put a demand on the company to provide upstream support
so we could just grab the latest kernel and use that driver.

I heard other very FOSS-oriented companies ask the same
and is pretty much what I've heard people like Jon Masters
and the Chromebook people say in relation to upstream first
(they can slam me if they disagree) - others also want an
upstream first approach from their component suppliers and
it is going to be part of the procurement process so having
upstream first is going to be a competitive advantage or
even strict requirement for the component vendor.

As it happened in my case the vendor was very unhappy
with this and unused to this kind of requirements. (They have
since changed their attitude so no-one needs to be outed.)

What I realized was that instead of being "hard" on vendors
with this, I could gain more by being let's say "firm".

I required that in order to procure their component, they
should present an upstreaming strategy, and post an initial
patch set for the specific product before we would agree
to procurement. This was more of a gentlemen agreement
than a hard contract clause, but it worked very well to
transform that company, and I think it is a good way, because
being too hard can be counter-productive, I guess it comes
from simple diplomacy, people do not like threats.

Yours,
Linus Walleij

Re: [Ksummit-discuss] [MAINTAINERS SUMMIT] Challenges in Upstream vs. Embargoed Development in Intel Graphics.

[Mark Brown]

[-- Attachment #1: Type: text/plain, Size: 1546 bytes --]

On Thu, Sep 06, 2018 at 12:43:05PM +0200, Linus Walleij wrote:

> It happened at one point that we were sourcing hardware from
> a third party, and it was pretty complex and I asked procurement
> to put a demand on the company to provide upstream support
> so we could just grab the latest kernel and use that driver.

> I heard other very FOSS-oriented companies ask the same
> and is pretty much what I've heard people like Jon Masters
> and the Chromebook people say in relation to upstream first
> (they can slam me if they disagree) - others also want an
> upstream first approach from their component suppliers and
> it is going to be part of the procurement process so having
> upstream first is going to be a competitive advantage or
> even strict requirement for the component vendor.

I've had experience of this from the component vendor side, we managed
to convince some of our customers that our upstream first approach was
good for them and that it should be something they look for in other
vendors.  The basic idea was that as we were working with the system and
getting things reviewed any integration problems were unlikley to be on
our side but rather with drivers from companies that weren't working
with the community and may have misunderstood things and things were
more likely to be robust, and it increased the chances that they'd get
long term support and be able to upgrade their kernel if they wanted to.
They might not ship upstream themselves but they benefited by working
with vendors who were focused on upstream.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [Ksummit-discuss] [MAINTAINERS SUMMIT] Challenges in Upstream vs. Embargoed Development in Intel Graphics.

[Sean Paul]

On Thu, Sep 6, 2018 at 6:43 AM Linus Walleij <linus.walleij@linaro.org> wrote:
>
> On Thu, Sep 6, 2018 at 12:25 PM Arnd Bergmann <arnd@arndb.de> wrote:
>
> > I think in the generalized case, you also want the reverse, but
> > that may be harder: When targetting specific software products
> > that you want to integrate your code, there should be a deadline
> > for the latest point by which code needs to be posted in
> > public.
>
> This brings in the process of procurement, as in how companies
> making products source their misc hardware like sensors,
> touchscreens, displays, FPGAs or whatnot.
>
> Maybe this is obvious.
>
> It happened at one point that we were sourcing hardware from
> a third party, and it was pretty complex and I asked procurement
> to put a demand on the company to provide upstream support
> so we could just grab the latest kernel and use that driver.
>
> I heard other very FOSS-oriented companies ask the same
> and is pretty much what I've heard people like Jon Masters
> and the Chromebook people say in relation to upstream first
> (they can slam me if they disagree) - others also want an
> upstream first approach from their component suppliers and
> it is going to be part of the procurement process so having
> upstream first is going to be a competitive advantage or
> even strict requirement for the component vendor.

Speaking really only for display, but most of this applies to other areas:

Pretty much this. Chromebook display stack must use drm/kms, and the
patches must* go upstream before landing in our kernel. Looking at our
4.14 branch [1], most of the commits there are
UPSTREAM/BACKPORT/FROMGIT/FROMLIST prefixed which means they've landed
upstream or have been reviewed upstream.

We carry downstream patches in a separate working branch shared
between developers and rebase on our 4.14 kernel regularly. Everything
in the working branch should be already sent upstream or being
polished for upstream. Reviews are done on the list. Once a patch
lands upstream, it's backported to our kernel and removed from the
working branch.

Sean

[1]- https://chromium.googlesource.com/chromiumos/third_party/kernel/+log/chromeos-4.14/drivers/gpu/drm

* There are always exceptions

>
> As it happened in my case the vendor was very unhappy
> with this and unused to this kind of requirements. (They have
> since changed their attitude so no-one needs to be outed.)
>
> What I realized was that instead of being "hard" on vendors
> with this, I could gain more by being let's say "firm".
>
> I required that in order to procure their component, they
> should present an upstreaming strategy, and post an initial
> patch set for the specific product before we would agree
> to procurement. This was more of a gentlemen agreement
> than a hard contract clause, but it worked very well to
> transform that company, and I think it is a good way, because
> being too hard can be counter-productive, I guess it comes
> from simple diplomacy, people do not like threats.
>
> Yours,
> Linus Walleij
> _______________________________________________
> Ksummit-discuss mailing list
> Ksummit-discuss@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/ksummit-discuss

Re: [Ksummit-discuss] [MAINTAINERS SUMMIT] Challenges in Upstream vs. Embargoed Development in Intel Graphics.

[Jon Masters]

On 09/06/2018 08:49 AM, Sean Paul wrote:

>> I heard other very FOSS-oriented companies ask the same
>> and is pretty much what I've heard people like Jon Masters
>> and the Chromebook people say in relation to upstream first
>> (they can slam me if they disagree) - others also want an
>> upstream first approach from their component suppliers and
>> it is going to be part of the procurement process so having
>> upstream first is going to be a competitive advantage or
>> even strict requirement for the component vendor.

Thanks for copying me. On our end, I require that the Arm server vendors
post patches and enablement upstream before they will get those patches
into RHEL. Originally, we allowed some wiggle room (e.g. while waiting
for upstream ACPI support) but then went from accepting patches had been
posted to requiring that they also minimally be in linux-next. Next up,
patches will need to be in linux-next or fully merged and in Fedora
before they'll get into RHEL. We're going to make the Arm folks behave
just like the Intel of a few years ago on the Xeon enablement side.
There won't be a choice if they want CentOS (we want RHEL, they want
CentOS, I know reality, and either way I get what I want from them).

Cheers,

Jon.

-- 
Computer Architect | Sent from my Fedora powered laptop

Re: [Ksummit-discuss] [MAINTAINERS SUMMIT] Challenges in Upstream vs. Embargoed Development in Intel Graphics.

[Rodrigo Vivi]

[-- Attachment #1: Type: text/plain, Size: 3791 bytes --]

On Thu, Sep 6, 2018 at 5:50 AM Sean Paul <seanpaul@chromium.org> wrote:

> On Thu, Sep 6, 2018 at 6:43 AM Linus Walleij <linus.walleij@linaro.org>
> wrote:
> >
> > On Thu, Sep 6, 2018 at 12:25 PM Arnd Bergmann <arnd@arndb.de> wrote:
> >
> > > I think in the generalized case, you also want the reverse, but
> > > that may be harder: When targetting specific software products
> > > that you want to integrate your code, there should be a deadline
> > > for the latest point by which code needs to be posted in
> > > public.
> >
> > This brings in the process of procurement, as in how companies
> > making products source their misc hardware like sensors,
> > touchscreens, displays, FPGAs or whatnot.
> >
> > Maybe this is obvious.
> >
> > It happened at one point that we were sourcing hardware from
> > a third party, and it was pretty complex and I asked procurement
> > to put a demand on the company to provide upstream support
> > so we could just grab the latest kernel and use that driver.
> >
> > I heard other very FOSS-oriented companies ask the same
> > and is pretty much what I've heard people like Jon Masters
> > and the Chromebook people say in relation to upstream first
> > (they can slam me if they disagree) - others also want an
> > upstream first approach from their component suppliers and
> > it is going to be part of the procurement process so having
> > upstream first is going to be a competitive advantage or
> > even strict requirement for the component vendor.
>
> Speaking really only for display, but most of this applies to other areas:
>
> Pretty much this. Chromebook display stack must use drm/kms, and the
> patches must* go upstream before landing in our kernel. Looking at our
> 4.14 branch [1], most of the commits there are
> UPSTREAM/BACKPORT/FROMGIT/FROMLIST prefixed which means they've landed
> upstream or have been reviewed upstream.
>

This firm position coming from the product directly is extremely beneficial
for us
and for open source in general.

Thanks a lot for that! :)


>
> We carry downstream patches in a separate working branch shared
> between developers and rebase on our 4.14 kernel regularly. Everything
> in the working branch should be already sent upstream or being
> polished for upstream. Reviews are done on the list. Once a patch
> lands upstream, it's backported to our kernel and removed from the
> working branch.
>
> Sean
>
> [1]-
> https://chromium.googlesource.com/chromiumos/third_party/kernel/+log/chromeos-4.14/drivers/gpu/drm
>
> * There are always exceptions
>
> >
> > As it happened in my case the vendor was very unhappy
> > with this and unused to this kind of requirements. (They have
> > since changed their attitude so no-one needs to be outed.)
> >
> > What I realized was that instead of being "hard" on vendors
> > with this, I could gain more by being let's say "firm".
> >
> > I required that in order to procure their component, they
> > should present an upstreaming strategy, and post an initial
> > patch set for the specific product before we would agree
> > to procurement. This was more of a gentlemen agreement
> > than a hard contract clause, but it worked very well to
> > transform that company, and I think it is a good way, because
> > being too hard can be counter-productive, I guess it comes
> > from simple diplomacy, people do not like threats.
> >
> > Yours,
> > Linus Walleij
> > _______________________________________________
> > Ksummit-discuss mailing list
> > Ksummit-discuss@lists.linuxfoundation.org
> > https://lists.linuxfoundation.org/mailman/listinfo/ksummit-discuss
> _______________________________________________
> Ksummit-discuss mailing list
> Ksummit-discuss@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/ksummit-discuss
>

[-- Attachment #2: Type: text/html, Size: 5318 bytes --]

Re: [Ksummit-discuss] [MAINTAINERS SUMMIT] Challenges in Upstream vs. Embargoed Development in Intel Graphics.

[Rodrigo Vivi]

[-- Attachment #1: Type: text/plain, Size: 4625 bytes --]

On Thu, Sep 6, 2018 at 2:54 AM Linus Walleij <linus.walleij@linaro.org>
wrote:

> On Wed, Sep 5, 2018 at 11:00 AM Daniel Vetter <daniel.vetter@ffwll.ch>
> wrote:
>
> > I have no idea how this works outside of intel or graphics, I tend to
> > not chat with those people as much as I do with graphics people. Would
> > be interesting to know how it is outside of the graphics bubble
> > indeed.
>
> More or less any SoC company for routers, handsets, tablets etc
> have the same problem.
>
> At one point I was made responsible for such a scenario. The
> approach I developed was a bit ad hoc but contained some of this:
>

Thanks a lot for sharing and thanks to the others who commented after
that.


>
> - Classify the components into embargoed and non-embargoed
>   so anything not affected by embargo can be pushed upstream.
>   (OK it's maybe obvious).
>

It is obvious but it is important to highlight, otherwise things might
get stuck in the pipe just because it is unclear and people get afraid
to sign-off something that might be seen as a "leak".


>
> - Get management to provide a cut-off date for embargo. Like
>   "after this point in time we certainly do not care what code you
>   publish pertaining to X" and make that formal so that when that
>   day comes developers can simply start sending the code without
>   having to ask permission again, because having to do that is
>   pointless and bureaucratic. This is a property of the company
>   "FOSS-OUT legal process" that is simple but still often
>   forgotten. It relieves the developers for the need to hammer
>   management for approvals.
>

Yes, on the internal front of the battle we are already doing this
and trying to get even earlier embargo-lifted decisions.


>
> - Use internal developers with high upstream experience and
>   NDA to make internal reviews and try to anticipate any problems
>   that will be seen when posting the code to the community and
>   fix it before it happens. Get the code in upstream shape.
>   (This is a good reason to hire kernel maintainers and have them
>   spend time upstream BTW.)
>

Yeap, we are trying to do the same here.


>
> - Minimize hamming distance to mainline, which means rebase
>   internal development as often as possible, track mainline,
>   bring in entire branches of development if need be just to not
>   deviate, because deviating too much from mainline is inviting
>   disaster. This goes counter to the conventional wisdom that
>   says you should use stable releases and baselines and LTS
>   and what not. I am not a big fan of the latter. Rebase on
>   Torvald's -rcN or even linux-next if you are aiming for upstream,
>   else you are not really aiming for upstream, but secondary
>   goals such as feature completion, "not disturbing development",
>   or getting tests to pass cleanly all the time. Get your priorities
>   straight: is upstream first really what you want? Then that
>   should be priority number one, not feature completion, not
>   "not disturbing developers", not passing internal tests.
>   If you find upstream first isn't really your number one priority
>   then call your strategy upstream second or upstream
>   third so that it reflects reality instead of trying to sound good.
>
> The last point was always the most controversial.
>

It is interesting. In the past I also thought the frequent rebases were
controversial
and maybe kind of insane to keep rebasing. But one of the key lessons we
had is that rebase often is not fast enough still. But keeping it in
constant rebase is easier and cheaper than rebase often. And definitely
cheper than loosing code with the famous "technical debt"
For every week that I kept the BOT disabled and blocked I regretted so
badly. When that happened I had to expend another full week only fixing
conflicts.
If code didn't change much yet, git and patch don't get lost easily. But if
they get lost, wiggle or manual inspection can get it faster.
But when code changed a lot like on -rc1 then the conflict is more a
forward-port as painful the backport and risky.

But of course, a key part of the constant rebase is CI. We constantly
rebase on top of drm-tip that has CI for every patch pre and post merge.
And we also have CI on internal branch for every rebasing point.


>
> Just my €0.01
> Linus Walleij
> _______________________________________________
> Ksummit-discuss mailing list
> Ksummit-discuss@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/ksummit-discuss
>

[-- Attachment #2: Type: text/html, Size: 6192 bytes --]

Re: [Ksummit-discuss] [MAINTAINERS SUMMIT] Challenges in Upstream vs. Embargoed Development in Intel Graphics.

[Mark Brown]

[-- Attachment #1: Type: text/plain, Size: 895 bytes --]

On Wed, Sep 05, 2018 at 10:17:44AM +0200, Daniel Vetter wrote:

> This was an invitation to exchange experience in how to best deal with
> the fallout - I do actually know that this is not an intel-only issue
> because I chatted with non-Intel people about how they deal with this.

It's definitely not an Intel only problem, I know I worked with this
when I was at a chip vendor and I know there are Linaro members with
various approaches to this.

> And I thought that figuring out how to do better upstream-first is
> very much within the scope of ks/lpc, hence why I suggested Rodrigo
> brings this up as a talk proposal.

I agree that there's a valuable discussion to have.  So long as people
are selling physical chips I don't think we can realistically tell
people that they should just upstream everything immediately without
regard for marketing, lead customer relationships and so on.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [Ksummit-discuss] [MAINTAINERS SUMMIT] Challenges in Upstream vs. Embargoed Development in Intel Graphics.

[Greg KH]

On Tue, Sep 04, 2018 at 09:49:13PM -0700, Rodrigo Vivi wrote:
> On Tue, Sep 4, 2018 at 9:22 PM Leon Romanovsky <leon@kernel.org> wrote:
> >
> > On Tue, Sep 04, 2018 at 12:54:16PM -0700, Rodrigo Vivi wrote:
> > > Hi there,
> > >
> > > I've submitted a proposal for plumber's referred track. There, I want to
> > > talk
> > > about tools and challenges we have on embargo development vs upstream one
> > > and
> > > how to get focus on upstream first and upstream always mentality.
> > >
> > > The name of plumbers proposal talk is:
> > > "Unveiling Intel Graphics Internal Development"
> > >
> > > I'm not sure if the talk will get accepted, but anyway I'd like to have the
> > > chance to talk to other maintainers to exchange views on different ways of
> > > maintaining this kind of embargo development including challenges, tools,
> > > processes, and rules.
> > >
> > > So I'm interested in hallway tracks of Maintainer / Kernel Summit, or maybe
> > > a
> > > bof session if there's interest.
> > >
> > > Please let me know if there's interested or if further information and/or
> > > clarification is needed.
> >
> > What is "embargo development"? Are you referring to US government
> > restrictions or to anything else?
> 
> No. nothing to do with government.
> Embargoed by company's temporary restrictions.
> 
> Sorry for not being clear here.

Why do we care about something like this?  It sounds like it is an Intel
issue in that they want to delay pushing stuff upstream.  Why does
upstream care about this?

> > Also can you please explain why should we know about internal Intel
> > development flow?
> 
> First of all I don't believe that we are the only one that need to
> keep this kind of flow and i915 has a very active development and we
> are strongly committed with upstream development. Our golden rules for
> internal development is upstream first, upstream always.

Great!  So if this rule runs into opposition to people within your
company, doesn't that sound like a meeting with those company members is
the best solution?

> So, maybe sharing some knowledge and lessons we learned on the past
> years might be useful to someone else that might still struggle with
> closed source style of development.

Ah, so you want to talk about how to change your process to work better
with companies that don't like doing upstream-first work?  That sounds
like a nice talk, but you need to make that a bit more clear here :)

> Also we have some challenges on keeping everything updated and ready
> for upstreaming at any moment.

"any moment"?  Don't you know this ahead of time?  If not, that sounds
like a company problem...

thanks,

greg k-h

[Ksummit-discuss] [MAINTAINERS SUMMIT] Challenges in Upstream vs. Embargoed Development in Intel Graphics.

[Rodrigo Vivi]

Hi there,

I've submitted a proposal for plumber's referred track. There, I want to talk
about tools and challenges we have on embargo development vs upstream one and
how to get focus on upstream first and upstream always mentality.

The name of plumbers proposal talk is:
"Unveiling Intel Graphics Internal Development"

I'm not sure if the talk will get accepted, but anyway I'd like to have the
chance to talk to other maintainers to exchange views on different ways of
maintaining this kind of embargo development including challenges, tools,
processes, and rules.

So I'm interested in hallway tracks of Maintainer / Kernel Summit, or maybe a
bof session if there's interest.

Please let me know if there's interested or if further information and/or
clarification is needed.

Thanks in advance,
Rodrigo.

Re: [Ksummit-discuss] [MAINTAINERS SUMMIT] Challenges in Upstream vs. Embargoed Development in Intel Graphics.

[Rodrigo Vivi]

[-- Attachment #1: Type: text/plain, Size: 1336 bytes --]

sorry for the duplication and for the noise.
This was the original one that had been blocked because I  wasn't yet
subscribed but it was now approved by moderator generating the duplication.
Sorry,
Rodrigo.

On Thu, Sep 6, 2018 at 12:37 PM Rodrigo Vivi <rodrigo.vivi@intel.com> wrote:

> Hi there,
>
> I've submitted a proposal for plumber's referred track. There, I want to
> talk
> about tools and challenges we have on embargo development vs upstream one
> and
> how to get focus on upstream first and upstream always mentality.
>
> The name of plumbers proposal talk is:
> "Unveiling Intel Graphics Internal Development"
>
> I'm not sure if the talk will get accepted, but anyway I'd like to have the
> chance to talk to other maintainers to exchange views on different ways of
> maintaining this kind of embargo development including challenges, tools,
> processes, and rules.
>
> So I'm interested in hallway tracks of Maintainer / Kernel Summit, or
> maybe a
> bof session if there's interest.
>
> Please let me know if there's interested or if further information and/or
> clarification is needed.
>
> Thanks in advance,
> Rodrigo.
> _______________________________________________
> Ksummit-discuss mailing list
> Ksummit-discuss@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/ksummit-discuss
>

[-- Attachment #2: Type: text/html, Size: 1922 bytes --]