Re: [PATCH 6/6] Drop flex_arrays

[Neil Horman <nhorman@tuxdriver.com>]

On Mon, Dec 17, 2018 at 07:50:11AM -0500, Kent Overstreet wrote:
> On Thu, Dec 13, 2018 at 01:09:17PM -0500, Neil Horman wrote:
> > On Thu, Dec 13, 2018 at 08:45:33AM -0800, Matthew Wilcox wrote:
> > > On Thu, Dec 13, 2018 at 10:51:49AM -0500, Neil Horman wrote:
> > > > On Thu, Dec 13, 2018 at 06:41:11AM -0800, Matthew Wilcox wrote:
> > > > > On Thu, Dec 13, 2018 at 09:30:47PM +0900, Xin Long wrote:
> > > > > > On Sat, Sep 8, 2018 at 1:57 AM Kent Overstreet
> > > > > > <kent.overstreet@gmail.com> wrote:
> > > > > > >
> > > > > > > All existing users have been converted to generic radix trees
> > > > > > NAK, SCTP is still using flex_arrays,
> > > > > > # grep flex_array net/sctp/*
> > > > > > 
> > > > > > This patch will break the build.
> > > > > 
> > > > > sctp added that user after this patch was sent.  Please stop adding
> > > > > flexarray users!
> > > > > 
> > > > > This particular user should probably have just used kvmalloc.
> > > > > 
> > > > 
> > > > No, I don't think thats right.
> > > > 
> > > > This appears to have been sent on September 7th.  Commit
> > > > 0d493b4d0be352b5e361e4fa0bc3efe952d8b10e, which added the use of flex_arrays to
> > > > sctp, seems to have been merged on August 10th, a month prior.
> > > 
> > > Are you seriously suggesting anybody sending cleanups needs to be
> > > monitoring every single email list to see if anybody has added a new user?
> > > Removing the flexarray has been advertised since May.
> > > https://lkml.org/lkml/2018/5/22/1142
> > > 
> > I don't see how thats any more egregious than everyone else having to monitor
> > for removals of code thats in the tree at some indeterminate future.  The long and the short of it
> > is that a new flex_array user was added in the intervening 7 months that this
> > patch has been waiting to go in, and it will now break if merged.  I'm sorry we
> > started using it during that time, but it got missed by everyone in the chain
> > that merged it, and hasn't been noticed in the 4 months since.  It is what it
> > is, and now it needs to be undone. 
> > 
> > > > regardless, however, sctp has a current in-tree use of flex_arrays, and merging
> > > > this patch will break the build without a respin.
> > > 
> > > Great.  I await your patch to replace the flexarray usage.
> > Sure, we'll get to it as soon as we can, or, if you are in a hurry, you can
> > replace the same usage, like you've done for all the other users in this series.
> 
> This is really my fault for slacking on getting generic-radix-trees in, and
> given that the sctp code has been merged I'll do the conversion.
> 
Thank you, I appreciate that.

> However.
> 
> Looking at the sctp code, honestly, wtf is going on here.
> 
> sctp_send_add_streams() calls sctp_stream_alloc_out() when it wants to make the
> out flex_array bigger - ok, this makes sense, you're using a flex_array because
> you want something resizable.
> 
> But wait, look what it actually does - it unconditionally frees the old flex
> array and preallocates a new one and copies the contents of the old one over.
> 
> Without, as far as I can tell, any locking whatsoever.
> 
> Was this code tested? Reviewed?
> 
Yup, both sides are protected by the socket lock for which the sctp connection
is associated.  Its locked in the sctp_setsockopt function, which is the path
through which we update/reallocate these flex arrays, and its also locked on
transmit in sctp_sendmsg, and on receive in sctp_rcv (via bh_lock_sock)

Neil