Announcement of SSE requirement change in dpdk

Announcement of SSE requirement change in dpdk

[Neil Horman]

Can anyone point out to me when and where the change to require SSE4.2 was
dicussed?  The first I saw of it was when the commit to the release notes went
in on August 3, and I can find no prior mention of it, save for the patches that
went in separately in the prior weeks.

Neil

Re: Announcement of SSE requirement change in dpdk

[Bruce Richardson]

On Wed, Aug 09, 2017 at 04:21:32PM -0400, Neil Horman wrote:
> Can anyone point out to me when and where the change to require SSE4.2 was
> dicussed?  The first I saw of it was when the commit to the release notes went
> in on August 3, and I can find no prior mention of it, save for the patches that
> went in separately in the prior weeks.
> 
> Neil
> 
There was no real widespread discussion of it, if that's what you are
looking for. I made the proposal via patch, and it was reviewed and
acked by a number of folks, with nobody raising any objections at the
time. Possibly it was a change that should have been more widely
publicised ahead of time, but I'm not sure what form that publicization
should have taken, since all tech discussion happens on the dev mailing
list anyway.
Not that I'm planning any similar changes, but for the future, what do
you think the process for changes like this should be - and what changes
would classify for it? If we have a process problem, let's try and fix
it.

Regards,
/Bruce.

Re: Announcement of SSE requirement change in dpdk

[Jay Rolette]

On Fri, Aug 11, 2017 at 3:29 PM, Bruce Richardson <
bruce.richardson@intel.com> wrote:

> On Wed, Aug 09, 2017 at 04:21:32PM -0400, Neil Horman wrote:
> > Can anyone point out to me when and where the change to require SSE4.2
> was
> > dicussed?  The first I saw of it was when the commit to the release
> notes went
> > in on August 3, and I can find no prior mention of it, save for the
> patches that
> > went in separately in the prior weeks.
> >
> > Neil
> >
> There was no real widespread discussion of it, if that's what you are
> looking for. I made the proposal via patch, and it was reviewed and
> acked by a number of folks, with nobody raising any objections at the
> time. Possibly it was a change that should have been more widely
> publicised ahead of time, but I'm not sure what form that publicization
> should have taken, since all tech discussion happens on the dev mailing
> list anyway.
> Not that I'm planning any similar changes, but for the future, what do
> you think the process for changes like this should be - and what changes
> would classify for it? If we have a process problem, let's try and fix
> it.
>
> Regards,
> /Bruce.
>

I'd suggest removing support for any hardware (CPUs or NICs) should require
a much "louder" announcement.

This particular one didn't hit my company, but when you have hardware out
in the field that you are expected to provide on-going software updates
for, these sort of changes are painful. The LTS tree helps considerably
with this, but definitely deserves more publicity than usual.

$0.02
Jay

Re: Announcement of SSE requirement change in dpdk

[Neil Horman]

On Fri, Aug 11, 2017 at 09:29:24PM +0100, Bruce Richardson wrote:
> On Wed, Aug 09, 2017 at 04:21:32PM -0400, Neil Horman wrote:
> > Can anyone point out to me when and where the change to require SSE4.2 was
> > dicussed?  The first I saw of it was when the commit to the release notes went
> > in on August 3, and I can find no prior mention of it, save for the patches that
> > went in separately in the prior weeks.
> > 
> > Neil
> > 
> There was no real widespread discussion of it, if that's what you are
> looking for. I made the proposal via patch, and it was reviewed and
> acked by a number of folks, with nobody raising any objections at the
I had a feeling that was the case, and yes, that does concern me somewhat.  In
this particular case I think its ok, because I can't really imagine anyone using
older atom processors, but I think it could become problematic in the future If
that support line moves too far into territory in which theres downstream
support issues (with things like OVS or other tree-external applications)

> time. Possibly it was a change that should have been more widely
> publicised ahead of time, but I'm not sure what form that publicization
> should have taken, since all tech discussion happens on the dev mailing
> list anyway.
> Not that I'm planning any similar changes, but for the future, what do
> you think the process for changes like this should be - and what changes
> would classify for it? If we have a process problem, let's try and fix
> it.
> 

I don't rightly know, to be honest.  DPDK is a little unique in this situation,
since user libraries are built to just access the lowest common denominator of a
given arch.  And in many ways, so is the kernel.  I'm open to suggestions, but I
think so some sort of plan would be a good idea.  These are just off the top of
my head, and likely have drawbacks, but just to get some conversation started:

1) Use extendend ISA instructions opportunistically
	By this I mean  to say, we could implement an alternatives system,
simmilar to what we have in the kernel, which can do dynamic instruction
replacement based on a run time test.  For example, you can write two versions
of a function, one which impements its method with sse4 and another version
which does the same thing using core isa instructions).  If sse4 is available at
runtime, the sse4 variant is mapped in, else the other version is.
	This is something we sort of talked about before, and while theres been
general support in its philosophy, its the sort of thing that takes alot of
work, and it is only used in those cases where you know you can use the
acceleration.

2) Limit where you introduce hardware deprecation
	Perhaps hardware deprecation can be announced in the same way ABI
deprecation is, and then introduced at a later date (I would make an opening
argument for the next LTS release).  Using the LTS release as a deprecation
point is nice because it lets downstream consumers standardize on a release
without having to worry about hardware support going away.

Just my $0.02.  food for thought
Neil

> Regards,
> /Bruce.
> 

Re: Announcement of SSE requirement change in dpdk

[Bruce Richardson]

On Sat, Aug 12, 2017 at 02:19:45PM -0400, Neil Horman wrote:
> On Fri, Aug 11, 2017 at 09:29:24PM +0100, Bruce Richardson wrote:
> > On Wed, Aug 09, 2017 at 04:21:32PM -0400, Neil Horman wrote:
> > > Can anyone point out to me when and where the change to require SSE4.2 was
> > > dicussed?  The first I saw of it was when the commit to the release notes went
> > > in on August 3, and I can find no prior mention of it, save for the patches that
> > > went in separately in the prior weeks.
> > > 
> > > Neil
> > > 
> > There was no real widespread discussion of it, if that's what you are
> > looking for. I made the proposal via patch, and it was reviewed and
> > acked by a number of folks, with nobody raising any objections at the
> I had a feeling that was the case, and yes, that does concern me somewhat.  In
> this particular case I think its ok, because I can't really imagine anyone using
> older atom processors, but I think it could become problematic in the future If
> that support line moves too far into territory in which theres downstream
> support issues (with things like OVS or other tree-external applications)
> 
> > time. Possibly it was a change that should have been more widely
> > publicised ahead of time, but I'm not sure what form that publicization
> > should have taken, since all tech discussion happens on the dev mailing
> > list anyway.
> > Not that I'm planning any similar changes, but for the future, what do
> > you think the process for changes like this should be - and what changes
> > would classify for it? If we have a process problem, let's try and fix
> > it.
> > 
> 
> I don't rightly know, to be honest.  DPDK is a little unique in this situation,
> since user libraries are built to just access the lowest common denominator of a
> given arch.  And in many ways, so is the kernel.  I'm open to suggestions, but I
> think so some sort of plan would be a good idea.  These are just off the top of
> my head, and likely have drawbacks, but just to get some conversation started:
> 
> 1) Use extendend ISA instructions opportunistically
> 	By this I mean  to say, we could implement an alternatives system,
> simmilar to what we have in the kernel, which can do dynamic instruction
> replacement based on a run time test.  For example, you can write two versions
> of a function, one which impements its method with sse4 and another version
> which does the same thing using core isa instructions).  If sse4 is available at
> runtime, the sse4 variant is mapped in, else the other version is.
> 	This is something we sort of talked about before, and while theres been
> general support in its philosophy, its the sort of thing that takes alot of
> work, and it is only used in those cases where you know you can use the
> acceleration.
> 
> 2) Limit where you introduce hardware deprecation
> 	Perhaps hardware deprecation can be announced in the same way ABI
> deprecation is, and then introduced at a later date (I would make an opening
> argument for the next LTS release).  Using the LTS release as a deprecation
> point is nice because it lets downstream consumers standardize on a release
> without having to worry about hardware support going away.
> 
> Just my $0.02.  food for thought
> Neil
> 
I think the ABI deprecation policy suggestion is a good one, where if we
want to drop support for some HW that was otherwise supported, we should
announce it at least one release in advance to make sure everyone is
aware of it.

/Bruce

Re: Announcement of SSE requirement change in dpdk

[Neil Horman]

On Mon, Aug 14, 2017 at 10:32:15AM +0100, Bruce Richardson wrote:
> On Sat, Aug 12, 2017 at 02:19:45PM -0400, Neil Horman wrote:
> > On Fri, Aug 11, 2017 at 09:29:24PM +0100, Bruce Richardson wrote:
> > > On Wed, Aug 09, 2017 at 04:21:32PM -0400, Neil Horman wrote:
> > > > Can anyone point out to me when and where the change to require SSE4.2 was
> > > > dicussed?  The first I saw of it was when the commit to the release notes went
> > > > in on August 3, and I can find no prior mention of it, save for the patches that
> > > > went in separately in the prior weeks.
> > > > 
> > > > Neil
> > > > 
> > > There was no real widespread discussion of it, if that's what you are
> > > looking for. I made the proposal via patch, and it was reviewed and
> > > acked by a number of folks, with nobody raising any objections at the
> > I had a feeling that was the case, and yes, that does concern me somewhat.  In
> > this particular case I think its ok, because I can't really imagine anyone using
> > older atom processors, but I think it could become problematic in the future If
> > that support line moves too far into territory in which theres downstream
> > support issues (with things like OVS or other tree-external applications)
> > 
> > > time. Possibly it was a change that should have been more widely
> > > publicised ahead of time, but I'm not sure what form that publicization
> > > should have taken, since all tech discussion happens on the dev mailing
> > > list anyway.
> > > Not that I'm planning any similar changes, but for the future, what do
> > > you think the process for changes like this should be - and what changes
> > > would classify for it? If we have a process problem, let's try and fix
> > > it.
> > > 
> > 
> > I don't rightly know, to be honest.  DPDK is a little unique in this situation,
> > since user libraries are built to just access the lowest common denominator of a
> > given arch.  And in many ways, so is the kernel.  I'm open to suggestions, but I
> > think so some sort of plan would be a good idea.  These are just off the top of
> > my head, and likely have drawbacks, but just to get some conversation started:
> > 
> > 1) Use extendend ISA instructions opportunistically
> > 	By this I mean  to say, we could implement an alternatives system,
> > simmilar to what we have in the kernel, which can do dynamic instruction
> > replacement based on a run time test.  For example, you can write two versions
> > of a function, one which impements its method with sse4 and another version
> > which does the same thing using core isa instructions).  If sse4 is available at
> > runtime, the sse4 variant is mapped in, else the other version is.
> > 	This is something we sort of talked about before, and while theres been
> > general support in its philosophy, its the sort of thing that takes alot of
> > work, and it is only used in those cases where you know you can use the
> > acceleration.
> > 
> > 2) Limit where you introduce hardware deprecation
> > 	Perhaps hardware deprecation can be announced in the same way ABI
> > deprecation is, and then introduced at a later date (I would make an opening
> > argument for the next LTS release).  Using the LTS release as a deprecation
> > point is nice because it lets downstream consumers standardize on a release
> > without having to worry about hardware support going away.
> > 
> > Just my $0.02.  food for thought
> > Neil
> > 
> I think the ABI deprecation policy suggestion is a good one, where if we
> want to drop support for some HW that was otherwise supported, we should
> announce it at least one release in advance to make sure everyone is
> aware of it.
> 

Ok, I can agree with that. Are we also agreed on limiting hardware deprecation
to LTS release points?
Neil

> /Bruce
> 

Re: Announcement of SSE requirement change in dpdk

[Bruce Richardson]

On Mon, Aug 14, 2017 at 06:50:40AM -0400, Neil Horman wrote:
> On Mon, Aug 14, 2017 at 10:32:15AM +0100, Bruce Richardson wrote:
> > On Sat, Aug 12, 2017 at 02:19:45PM -0400, Neil Horman wrote:
> > > On Fri, Aug 11, 2017 at 09:29:24PM +0100, Bruce Richardson wrote:
> > > > On Wed, Aug 09, 2017 at 04:21:32PM -0400, Neil Horman wrote:
> > > > > Can anyone point out to me when and where the change to require SSE4.2 was
> > > > > dicussed?  The first I saw of it was when the commit to the release notes went
> > > > > in on August 3, and I can find no prior mention of it, save for the patches that
> > > > > went in separately in the prior weeks.
> > > > > 
> > > > > Neil
> > > > > 
> > > > There was no real widespread discussion of it, if that's what you are
> > > > looking for. I made the proposal via patch, and it was reviewed and
> > > > acked by a number of folks, with nobody raising any objections at the
> > > I had a feeling that was the case, and yes, that does concern me somewhat.  In
> > > this particular case I think its ok, because I can't really imagine anyone using
> > > older atom processors, but I think it could become problematic in the future If
> > > that support line moves too far into territory in which theres downstream
> > > support issues (with things like OVS or other tree-external applications)
> > > 
> > > > time. Possibly it was a change that should have been more widely
> > > > publicised ahead of time, but I'm not sure what form that publicization
> > > > should have taken, since all tech discussion happens on the dev mailing
> > > > list anyway.
> > > > Not that I'm planning any similar changes, but for the future, what do
> > > > you think the process for changes like this should be - and what changes
> > > > would classify for it? If we have a process problem, let's try and fix
> > > > it.
> > > > 
> > > 
> > > I don't rightly know, to be honest.  DPDK is a little unique in this situation,
> > > since user libraries are built to just access the lowest common denominator of a
> > > given arch.  And in many ways, so is the kernel.  I'm open to suggestions, but I
> > > think so some sort of plan would be a good idea.  These are just off the top of
> > > my head, and likely have drawbacks, but just to get some conversation started:
> > > 
> > > 1) Use extendend ISA instructions opportunistically
> > > 	By this I mean  to say, we could implement an alternatives system,
> > > simmilar to what we have in the kernel, which can do dynamic instruction
> > > replacement based on a run time test.  For example, you can write two versions
> > > of a function, one which impements its method with sse4 and another version
> > > which does the same thing using core isa instructions).  If sse4 is available at
> > > runtime, the sse4 variant is mapped in, else the other version is.
> > > 	This is something we sort of talked about before, and while theres been
> > > general support in its philosophy, its the sort of thing that takes alot of
> > > work, and it is only used in those cases where you know you can use the
> > > acceleration.
> > > 
> > > 2) Limit where you introduce hardware deprecation
> > > 	Perhaps hardware deprecation can be announced in the same way ABI
> > > deprecation is, and then introduced at a later date (I would make an opening
> > > argument for the next LTS release).  Using the LTS release as a deprecation
> > > point is nice because it lets downstream consumers standardize on a release
> > > without having to worry about hardware support going away.
> > > 
> > > Just my $0.02.  food for thought
> > > Neil
> > > 
> > I think the ABI deprecation policy suggestion is a good one, where if we
> > want to drop support for some HW that was otherwise supported, we should
> > announce it at least one release in advance to make sure everyone is
> > aware of it.
> > 
> 
> Ok, I can agree with that. Are we also agreed on limiting hardware deprecation
> to LTS release points?
> Neil
> 
To be clear, you think any hardware deprecation should be done as part
of the LTS release, rather than just after one? I would have thought the
latter, so as to keep legacy HW support around for as long as possible,
but I won't have a problem either way.

If you think it's a good policy to have a fixed point at which any HW
deprecations happen, I don't have an objection to that, but I'm curious
as to whether others think it may be too restrictive. It's not something
we've had a lot of up till now to know how big a deal such a restriction
might be.

+techboard: I suggest the tech board take an agenda item to ratify and
assign owner to document a HW deprecation policy, based on this thread,
at it's next meeting.

/Bruce

Re: Announcement of SSE requirement change in dpdk

[Neil Horman]

On Mon, Aug 14, 2017 at 11:58:21AM +0100, Bruce Richardson wrote:
> On Mon, Aug 14, 2017 at 06:50:40AM -0400, Neil Horman wrote:
> > On Mon, Aug 14, 2017 at 10:32:15AM +0100, Bruce Richardson wrote:
> > > On Sat, Aug 12, 2017 at 02:19:45PM -0400, Neil Horman wrote:
> > > > On Fri, Aug 11, 2017 at 09:29:24PM +0100, Bruce Richardson wrote:
> > > > > On Wed, Aug 09, 2017 at 04:21:32PM -0400, Neil Horman wrote:
> > > > > > Can anyone point out to me when and where the change to require SSE4.2 was
> > > > > > dicussed?  The first I saw of it was when the commit to the release notes went
> > > > > > in on August 3, and I can find no prior mention of it, save for the patches that
> > > > > > went in separately in the prior weeks.
> > > > > > 
> > > > > > Neil
> > > > > > 
> > > > > There was no real widespread discussion of it, if that's what you are
> > > > > looking for. I made the proposal via patch, and it was reviewed and
> > > > > acked by a number of folks, with nobody raising any objections at the
> > > > I had a feeling that was the case, and yes, that does concern me somewhat.  In
> > > > this particular case I think its ok, because I can't really imagine anyone using
> > > > older atom processors, but I think it could become problematic in the future If
> > > > that support line moves too far into territory in which theres downstream
> > > > support issues (with things like OVS or other tree-external applications)
> > > > 
> > > > > time. Possibly it was a change that should have been more widely
> > > > > publicised ahead of time, but I'm not sure what form that publicization
> > > > > should have taken, since all tech discussion happens on the dev mailing
> > > > > list anyway.
> > > > > Not that I'm planning any similar changes, but for the future, what do
> > > > > you think the process for changes like this should be - and what changes
> > > > > would classify for it? If we have a process problem, let's try and fix
> > > > > it.
> > > > > 
> > > > 
> > > > I don't rightly know, to be honest.  DPDK is a little unique in this situation,
> > > > since user libraries are built to just access the lowest common denominator of a
> > > > given arch.  And in many ways, so is the kernel.  I'm open to suggestions, but I
> > > > think so some sort of plan would be a good idea.  These are just off the top of
> > > > my head, and likely have drawbacks, but just to get some conversation started:
> > > > 
> > > > 1) Use extendend ISA instructions opportunistically
> > > > 	By this I mean  to say, we could implement an alternatives system,
> > > > simmilar to what we have in the kernel, which can do dynamic instruction
> > > > replacement based on a run time test.  For example, you can write two versions
> > > > of a function, one which impements its method with sse4 and another version
> > > > which does the same thing using core isa instructions).  If sse4 is available at
> > > > runtime, the sse4 variant is mapped in, else the other version is.
> > > > 	This is something we sort of talked about before, and while theres been
> > > > general support in its philosophy, its the sort of thing that takes alot of
> > > > work, and it is only used in those cases where you know you can use the
> > > > acceleration.
> > > > 
> > > > 2) Limit where you introduce hardware deprecation
> > > > 	Perhaps hardware deprecation can be announced in the same way ABI
> > > > deprecation is, and then introduced at a later date (I would make an opening
> > > > argument for the next LTS release).  Using the LTS release as a deprecation
> > > > point is nice because it lets downstream consumers standardize on a release
> > > > without having to worry about hardware support going away.
> > > > 
> > > > Just my $0.02.  food for thought
> > > > Neil
> > > > 
> > > I think the ABI deprecation policy suggestion is a good one, where if we
> > > want to drop support for some HW that was otherwise supported, we should
> > > announce it at least one release in advance to make sure everyone is
> > > aware of it.
> > > 
> > 
> > Ok, I can agree with that. Are we also agreed on limiting hardware deprecation
> > to LTS release points?
> > Neil
> > 
> To be clear, you think any hardware deprecation should be done as part
> of the LTS release, rather than just after one? I would have thought the
> latter, so as to keep legacy HW support around for as long as possible,
> but I won't have a problem either way.
> 
Thats what I was proposing, yes, that we should limit HW deprecation to LTS
releases, as that allows people using an LTS release to be guaranteed hardware
support for the lifetime of the release, though I'm not really married to the
idea, I'm just using it as a starting point to discuss the decision.

I do think we need to state that LTS releases maintain hardware compatibility as
well as ABI, but if we're comfortable with just adhering to that statement and
doing the backport work needed if it comes up, I'm also fine with hardware
deprecation on the "one release ahead" ABI schedule.

> If you think it's a good policy to have a fixed point at which any HW
> deprecations happen, I don't have an objection to that, but I'm curious
> as to whether others think it may be too restrictive. It's not something
> we've had a lot of up till now to know how big a deal such a restriction
> might be.
> 
I do think a deprecation schedule is warranted, but I'm flexible on the cadence.
I too was hoping we could get other opinions on the matter regarding how
restrictive this needs to be.

> +techboard: I suggest the tech board take an agenda item to ratify and
> assign owner to document a HW deprecation policy, based on this thread,
> at it's next meeting.
> 
That would be great, please!

Best
Neil

> /Bruce
>