[GIT PULL] flexible-array member conversion patches for 5.8-rc2

[GIT PULL] flexible-array member conversion patches for 5.8-rc2

[Gustavo A. R. Silva]

The following changes since commit b3a9e3b9622ae10064826dccb4f7a52bd88c7407:

  Linux 5.8-rc1 (2020-06-14 12:45:04 -0700)

are available in the Git repository at:

  git://git.kernel.org/pub/scm/linux/kernel/git/gustavoars/linux.git tags/flex-array-conversions-5.8-rc2

for you to fetch changes up to 4caeb3acd21ffa2496fc3e94398019d4b3415523:

  w1: Replace zero-length array with flexible-array (2020-06-15 15:20:19 -0500)

----------------------------------------------------------------
flexible-array member conversion patches for 5.8-rc2

Hi Linus,

Please, pull the following patches that replace zero-length arrays with
flexible-array members.

Notice that all of these patches have been baking in linux-next for
two development cycles now.

There is a regular need in the kernel to provide a way to declare having a
dynamically sized set of trailing elements in a structure. Kernel code should
always use “flexible array members”[1] for these cases. The older style of
one-element or zero-length arrays should no longer be used.

In older C code, dynamically sized trailing elements were done by specifying
a one-element array at the end of a structure:

struct something {
        size_t count;
        struct foo items[1];
};

This led to fragile size calculations via sizeof() (which would need to remove
the size of the single trailing element to get a correct size of the “header”).
A GNU C extension[2] was introduced to allow for zero-length arrays, to avoid
these kinds of size problems:

struct something {
        size_t count;
        struct foo items[0];
};

But this led to other problems, and didn’t solve some problems shared by both
styles, like not being able to detect when such an array is accidentally being
used _not_ at the end of a structure (which could happen directly, or when
such a struct was in unions, structs of structs, etc).

C99 introduced “flexible array members”, which lacks a numeric size for the
array declaration entirely:

struct something {
        size_t count;
        struct foo items[];
};

This is the way the kernel expects dynamically sized trailing elements to be
declared. It allows the compiler to generate errors when the flexible array
does not occur last in the structure, which helps to prevent some kind of
undefined behavior[3] bugs from being inadvertently introduced to the codebase.
It also allows the compiler to correctly analyze array sizes (via sizeof(),
CONFIG_FORTIFY_SOURCE, and CONFIG_UBSAN_BOUNDS). For instance, there is no
mechanism that warns us that the following application of the sizeof() operator
to a zero-length array always results in zero:

struct something {
        size_t count;
        struct foo items[0];
};

struct something *instance;

instance = kmalloc(struct_size(instance, items, count), GFP_KERNEL);
instance->count = count;

size = sizeof(instance->items) * instance->count;
memcpy(instance->items, source, size);

At the last line of code above, size turns out to be zero, when one might have
thought it represents the total size in bytes of the dynamic memory recently
allocated for the trailing array items. Here are a couple examples of this
issue[4][5]. Instead, flexible array members have incomplete type, and so the
sizeof() operator may not be applied[2], so any misuse of such operators will
be immediately noticed at build time.

With respect to one-element arrays, one has to be acutely aware that such
arrays occupy at least as much space as a single object of the type[2], hence they
contribute to the size of the enclosing structure. This is prone to error every
time people want to calculate the total size of dynamic memory to allocate for
a structure containing an array of this kind as a member:

struct something {
        size_t count;
        struct foo items[1];
};

struct something *instance;

instance = kmalloc(struct_size(instance, items, count - 1), GFP_KERNEL);
instance->count = count;

size = sizeof(instance->items) * instance->count;
memcpy(instance->items, source, size);

In the example above, we had to remember to calculate count - 1 when using the
struct_size() helper, otherwise we would have –unintentionally– allocated memory
for one too many items objects. The cleanest and least error-prone way to
implement this is through the use of a flexible array member:

struct something {
        size_t count;
        struct foo items[];
};

struct something *instance;

instance = kmalloc(struct_size(instance, items, count), GFP_KERNEL);
instance->count = count;

size = sizeof(instance->items[0]) * instance->count;
memcpy(instance->items, source, size);

Thanks
--
Gustavo

[1] https://en.wikipedia.org/wiki/Flexible_array_member
[2] https://gcc.gnu.org/onlinedocs/gcc/Zero-Length.html
[3] https://git.kernel.org/linus/76497732932f15e7323dc805e8ea8dc11bb587cf
[4] https://git.kernel.org/linus/f2cd32a443da694ac4e28fbf4ac6f9d5cc63a539
[5] https://git.kernel.org/linus/ab91c2a89f86be2898cee208d492816ec238b2cf

----------------------------------------------------------------
Gustavo A. R. Silva (41):
      firmware: dmi-sysfs: Replace zero-length array with flexible-array member
      firmware: google: memconsole: Replace zero-length array with flexible-array member
      firmware: google: vpd: Replace zero-length array with flexible-array member
      aio: Replace zero-length array with flexible-array
      ARM: tegra: Replace zero-length array with flexible-array
      dmaengine: Replace zero-length array with flexible-array
      can: peak_canfd: Replace zero-length array with flexible-array
      can: Replace zero-length array with flexible-array
      crypto: Replace zero-length array with flexible-array
      drbd: Replace zero-length array with flexible-array
      drm/edid: Replace zero-length array with flexible-array
      cb710: Replace zero-length array with flexible-array
      firewire: ohci: Replace zero-length array with flexible-array
      FS-Cache: Replace zero-length array with flexible-array
      ia64: kernel: unwind_i.h: Replace zero-length array with flexible-array
      samples: mei: Replace zero-length array with flexible-array
      ibft: Replace zero-length array with flexible-array
      jffs2: Replace zero-length array with flexible-array
      KVM: Replace zero-length array with flexible-array
      kexec: Replace zero-length array with flexible-array
      keys: encrypted-type: Replace zero-length array with flexible-array
      kprobes: Replace zero-length array with flexible-array
      libata: Replace zero-length array with flexible-array
      tools/testing/nvdimm: Replace zero-length array with flexible-array
      block: Replace zero-length array with flexible-array
      oprofile: Replace zero-length array with flexible-array
      firmware: pcdp: Replace zero-length array with flexible-array
      media: pwc: Replace zero-length array with flexible-array
      rapidio: Replace zero-length array with flexible-array
      RxRPC: Replace zero-length array with flexible-array
      phy: samsung: Replace zero-length array with flexible-array
      sctp: Replace zero-length array with flexible-array
      ima: Replace zero-length array with flexible-array
      ASoC: SOF: Replace zero-length array with flexible-array
      Squashfs: Replace zero-length array with flexible-array
      stm class: Replace zero-length array with flexible-array
      dmaengine: tegra-apb: Replace zero-length array with flexible-array
      tifm: Replace zero-length array with flexible-array
      soc: ti: Replace zero-length array with flexible-array
      tracing/probe: Replace zero-length array with flexible-array
      w1: Replace zero-length array with flexible-array

 arch/ia64/kernel/unwind_i.h                   |  2 +-
 block/partitions/ldm.h                        |  2 +-
 drivers/amba/tegra-ahb.c                      |  2 +-
 drivers/block/drbd/drbd_int.h                 |  2 +-
 drivers/block/drbd/drbd_protocol.h            |  8 +++---
 drivers/crypto/chelsio/chcr_crypto.h          |  8 +++---
 drivers/dma/milbeaut-hdmac.c                  |  2 +-
 drivers/dma/milbeaut-xdmac.c                  |  2 +-
 drivers/dma/moxart-dma.c                      |  2 +-
 drivers/dma/tegra20-apb-dma.c                 |  2 +-
 drivers/dma/ti/edma.c                         |  2 +-
 drivers/dma/ti/k3-udma.c                      |  2 +-
 drivers/dma/timb_dma.c                        |  2 +-
 drivers/firewire/core-cdev.c                  |  2 +-
 drivers/firewire/core-transaction.c           |  2 +-
 drivers/firewire/core.h                       |  2 +-
 drivers/firewire/nosy.c                       |  2 +-
 drivers/firewire/ohci.c                       |  2 +-
 drivers/firmware/dmi-sysfs.c                  |  2 +-
 drivers/firmware/google/memconsole-coreboot.c |  2 +-
 drivers/firmware/google/vpd.c                 |  2 +-
 drivers/firmware/iscsi_ibft.c                 |  2 +-
 drivers/firmware/pcdp.h                       |  2 +-
 drivers/hwtracing/stm/policy.c                |  2 +-
 drivers/hwtracing/stm/stm.h                   |  4 +--
 drivers/media/usb/pwc/pwc.h                   |  2 +-
 drivers/net/can/peak_canfd/peak_pciefd_main.c |  4 +--
 drivers/oprofile/cpu_buffer.h                 |  2 +-
 drivers/phy/samsung/phy-samsung-usb2.h        |  2 +-
 drivers/rapidio/rio-scan.c                    |  2 +-
 drivers/soc/ti/knav_qmss.h                    |  2 +-
 drivers/w1/w1_netlink.h                       |  4 +--
 fs/aio.c                                      |  2 +-
 fs/jffs2/nodelist.h                           |  2 +-
 fs/jffs2/summary.h                            |  4 +--
 fs/squashfs/squashfs_fs.h                     | 16 ++++++------
 include/drm/drm_displayid.h                   |  2 +-
 include/keys/encrypted-type.h                 |  2 +-
 include/keys/rxrpc-type.h                     |  4 +--
 include/linux/can/skb.h                       |  2 +-
 include/linux/cb710.h                         |  2 +-
 include/linux/dmaengine.h                     |  4 +--
 include/linux/fscache-cache.h                 |  2 +-
 include/linux/kexec.h                         |  2 +-
 include/linux/kprobes.h                       |  2 +-
 include/linux/kvm_host.h                      |  2 +-
 include/linux/libata.h                        |  2 +-
 include/linux/sctp.h                          | 36 +++++++++++++--------------
 include/linux/tifm.h                          |  2 +-
 kernel/trace/trace_probe.h                    |  2 +-
 samples/mei/mei-amt-version.c                 |  2 +-
 security/integrity/integrity.h                |  4 +--
 sound/soc/sof/probe.h                         |  8 +++---
 tools/testing/nvdimm/test/nfit_test.h         |  6 ++---
 54 files changed, 96 insertions(+), 96 deletions(-)

Re: [GIT PULL] flexible-array member conversion patches for 5.8-rc2

[Linus Torvalds]

On Mon, Jun 15, 2020 at 1:34 PM Gustavo A. R. Silva
<gustavoars@kernel.org> wrote:
>
> flexible-array member conversion patches for 5.8-rc2

Gaah. I merged this, and even spent some time trying to make the merge
message look better, and then when I started looking at the end
result, I ended up unpulling it.

It's all been recently rebased, which I might be willing to ignore for
something like this, but all these one-liner patches then come with a
boiler-plate commit message that is something like 50 lines long and
is actively misleading as it talks about "this tree-wide patch" and
just repeats the generic issues that were presented (better) in the
merge message anyway.

The individual commit messages literally add nothing specific to that
commit AT ALL except for the very first line, which is also
boiler-plate but at least mentions the subsystem.

End result: when I do "git log" after having merged this, the result
is basically two *thousand* lines of repeated noise.

Some repetition is fine. But 2000 lines of repeated boiler-plate makes
me go "that's actively detrimental to any use of 'git log'" and I
undid my pull.

               Linus

Re: [GIT PULL] flexible-array member conversion patches for 5.8-rc2

[Gustavo A. R. Silva]

On Mon, Jun 15, 2020 at 04:07:28PM -0700, Linus Torvalds wrote:
> On Mon, Jun 15, 2020 at 1:34 PM Gustavo A. R. Silva
> <gustavoars@kernel.org> wrote:
> >
> > flexible-array member conversion patches for 5.8-rc2
> 
> Gaah. I merged this, and even spent some time trying to make the merge
> message look better, and then when I started looking at the end
> result, I ended up unpulling it.
> 
> It's all been recently rebased, which I might be willing to ignore for
> something like this, but all these one-liner patches then come with a
> boiler-plate commit message that is something like 50 lines long and
> is actively misleading as it talks about "this tree-wide patch" and
> just repeats the generic issues that were presented (better) in the
> merge message anyway.
> 
> The individual commit messages literally add nothing specific to that
> commit AT ALL except for the very first line, which is also
> boiler-plate but at least mentions the subsystem.
> 
> End result: when I do "git log" after having merged this, the result
> is basically two *thousand* lines of repeated noise.
> 
> Some repetition is fine. But 2000 lines of repeated boiler-plate makes
> me go "that's actively detrimental to any use of 'git log'" and I
> undid my pull.
> 

Yep, you're right. We made the mistake of not having something to point
people to, from the beginning; which we are trying to fix by adding the
following documentation to deprecated.rst:

https://lore.kernel.org/lkml/20200608213711.GA22271@embeddedor/

That's the reason why the log for these sorts of patches ended up being
so repetitive: as a way to compensate for not having proper documentation
in place from the beginning.

And yes, it was my mistake not to remove part of the log from the changelog
text when I turned the treewide patch into separate per-subsystem patches
--for they to be properly tested by the 0-day CI Kernel Test Service.

I will modify the changelog text of the separate patches and send you the
pull-request once again.

BTW, the commit message is the exact same text we are planning to add to
the deprecated.rst file, which will be used for both zero-length and
one-element array conversions. It might be better to to remove the part
for the one-element arrays, in this case.

Thanks
--
Gustavo