Olof, Arnd, I've put together a script to move the dts files and update the makefiles. It doesn't handle files not following a common prefix which isn't many and some includes within the dts files will need some fixups by hand. MAINTAINERS will also need updating. A few questions: Do we want to move absolutely everything to subdirs? There's quite a few platforms with only 1-2 platforms. I haven't added these to the list yet, but can. Do any vendors need another level of directories? davinci, omap, nspire, etc. for TI for example. What to do with armv7m.dtsi? I guess it should remain and we just fixup the include. There may be a few other cross vendor things. Sub-arch maintainers, 'vendor_map' below is the mapping of file prefix to new subdirectory (the SoC vendor prefix). Please comment if there are any issues. Rob 8<----------------------------------------------------------- #!/usr/bin/env python3 import os import re from git import Git import glob vendor_map = { 'imx': 'fsl', 'ls': 'fsl', 'vf': 'fsl', 'qcom': 'qcom', 'am3' : 'ti', 'am4' : 'ti', 'am5' : 'ti', 'da' : 'ti', 'dm' : 'ti', 'dra' : 'ti', 'keystone' : 'ti', 'omap' : 'ti', 'nspire' : 'ti', 'armada' : 'marvell', 'berlin' : 'marvell', 'dove' : 'marvell', 'kirkwood' : 'marvell', 'orion' : 'marvell', 'pxa' : 'marvell', 'mvebu' : 'marvell', 'mmp2' : 'marvell', 'arm-' : 'arm', 'integ' : 'arm', 've' : 'arm', 'aspeed' : 'aspeed', 'at91' : 'atmel', 'sama' : 'atmel', 'bcm' : 'brcm', 'exynos' : 'samsung', 's3c' : 'samsung', 's5p' : 'samsung', 'gemini' : 'cortina', 'hi3' : 'hisilicon', 'hip' : 'hisilicon', 'hisi' : 'hisilicon', 'mt' : 'mediatek', 'meson' : 'amlogic', 'owl' : 'actions', 'r7' : 'renesas', 'r8' : 'renesas', 'r9' : 'renesas', 'rk' : 'rockchip', 'socfpga' : 'altera', 'st' : 'st', 'spear' : 'st', 'sun' : 'allwinner', 'tegra' : 'nvidia', 'zynq' : 'xilinx', 'wm' : 'wm', 'uniph' : 'socionext', 'zx' : 'zte', } if __name__ == "__main__": g = Git('.') g.checkout("HEAD", "arch/arm/boot/dts/Makefile") dts_make = open("arch/arm/boot/dts/Makefile", "r").read() # make entries 1 line make2 = re.sub(r'\\\n', '', dts_make) for k,v in vendor_map.items(): for f in glob.iglob("arch/arm/boot/dts/" + k + "*.*"): new_dir = "arch/arm/boot/dts/" + v + "/" base = os.path.splitext(os.path.basename(f))[0] os.makedirs(new_dir, exist_ok=True) g.mv(f, new_dir) # Remove the file from the makefile dts_make = re.sub('.*' + base + r'\.dtb.*\\\n', '', dts_make) dts_make = re.sub('.*' + base + r'\.dtb', '', dts_make) # extract the matching makefile entry reg = re.search(r'.*' + base + r'.*', make2) if not reg: continue entry = reg.group(0) make2 = re.sub(r'.*' + base + r'.*', '', make2) if entry: makefile = open(new_dir + 'Makefile', 'a+') print(entry, file=makefile) makefile.close() for d in sorted(glob.iglob("arch/arm/boot/dts/*/")): dts_make += 'subdir-y += ' + d.split(os.path.sep)[-2] + '\n' # Add license and sort entries of sub-dir makefile vendor_make = '# SPDX-License-Identifier: GPL-2.0\n' make_lines = open(d + 'Makefile', 'r').readlines() for l in sorted(make_lines): vendor_make += l vendor_make = re.sub(r'\t', r'\\\n\t', vendor_make) f = open(d + 'Makefile', 'w') f.write(vendor_make) f.close() g.add(d + 'Makefile') # Remove entries with no dtbs left dts_make = re.sub(r'.*\+= \\\n\n', '', dts_make) open("arch/arm/boot/dts/Makefile", "w").write(dts_make) g.add("arch/arm/boot/dts/Makefile") _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Hello Rob, On 04/12/2018 12:36:49-0600, Rob Herring wrote: > Olof, Arnd, > > I've put together a script to move the dts files and update the > makefiles. It doesn't handle files not following a common prefix which > isn't many and some includes within the dts files will need some fixups > by hand. > > MAINTAINERS will also need updating. > > A few questions: > > Do we want to move absolutely everything to subdirs? There's quite a > few platforms with only 1-2 platforms. I haven't added these to the > list yet, but can. > > Do any vendors need another level of directories? davinci, omap, nspire, > etc. for TI for example. > > What to do with armv7m.dtsi? I guess it should remain and we just fixup > the include. There may be a few other cross vendor things. > > > Sub-arch maintainers, > 'vendor_map' below is the mapping of file prefix to new subdirectory > (the SoC vendor prefix). Please comment if there are any issues. > > Rob > > 8<----------------------------------------------------------- > #!/usr/bin/env python3 > > import os > import re > from git import Git > import glob > > vendor_map = { > 'imx': 'fsl', > 'ls': 'fsl', > 'vf': 'fsl', > 'qcom': 'qcom', > 'am3' : 'ti', > 'am4' : 'ti', > 'am5' : 'ti', > 'da' : 'ti', > 'dm' : 'ti', > 'dra' : 'ti', > 'keystone' : 'ti', > 'omap' : 'ti', > 'nspire' : 'ti', > 'armada' : 'marvell', > 'berlin' : 'marvell', > 'dove' : 'marvell', > 'kirkwood' : 'marvell', > 'orion' : 'marvell', > 'pxa' : 'marvell', > 'mvebu' : 'marvell', > 'mmp2' : 'marvell', > 'arm-' : 'arm', > 'integ' : 'arm', > 've' : 'arm', > 'aspeed' : 'aspeed', > 'at91' : 'atmel', > 'sama' : 'atmel', Unfortunately, we have many boards without a proper prefix. Wouldn't it be better to use arch/arm/boot/dts/Makefile and map dtb-$(CONFIG_*) to a vendor directory? This way you would be sure to not miss any. This would also ease the creation of subdirs if we decide to do that. -- Alexandre Belloni, Bootlin Embedded Linux and Kernel engineering https://bootlin.com _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
On Tue, Dec 4, 2018 at 12:47 PM Alexandre Belloni <alexandre.belloni@bootlin.com> wrote: > > Hello Rob, > > On 04/12/2018 12:36:49-0600, Rob Herring wrote: > > Olof, Arnd, > > > > I've put together a script to move the dts files and update the > > makefiles. It doesn't handle files not following a common prefix which > > isn't many and some includes within the dts files will need some fixups > > by hand. > > > > MAINTAINERS will also need updating. > > > > A few questions: > > > > Do we want to move absolutely everything to subdirs? There's quite a > > few platforms with only 1-2 platforms. I haven't added these to the > > list yet, but can. > > > > Do any vendors need another level of directories? davinci, omap, nspire, > > etc. for TI for example. > > > > What to do with armv7m.dtsi? I guess it should remain and we just fixup > > the include. There may be a few other cross vendor things. > > > > > > Sub-arch maintainers, > > 'vendor_map' below is the mapping of file prefix to new subdirectory > > (the SoC vendor prefix). Please comment if there are any issues. > > > > Rob > > > > 8<----------------------------------------------------------- > > #!/usr/bin/env python3 > > > > import os > > import re > > from git import Git > > import glob > > > > vendor_map = { > > 'imx': 'fsl', > > 'ls': 'fsl', > > 'vf': 'fsl', > > 'qcom': 'qcom', > > 'am3' : 'ti', > > 'am4' : 'ti', > > 'am5' : 'ti', > > 'da' : 'ti', > > 'dm' : 'ti', > > 'dra' : 'ti', > > 'keystone' : 'ti', > > 'omap' : 'ti', > > 'nspire' : 'ti', > > 'armada' : 'marvell', > > 'berlin' : 'marvell', > > 'dove' : 'marvell', > > 'kirkwood' : 'marvell', > > 'orion' : 'marvell', > > 'pxa' : 'marvell', > > 'mvebu' : 'marvell', > > 'mmp2' : 'marvell', > > 'arm-' : 'arm', > > 'integ' : 'arm', > > 've' : 'arm', > > 'aspeed' : 'aspeed', > > 'at91' : 'atmel', > > 'sama' : 'atmel', > > Unfortunately, we have many boards without a proper prefix. Wouldn't it > be better to use arch/arm/boot/dts/Makefile and map dtb-$(CONFIG_*) to a > vendor directory? Only 13 boards by my count Atmel. And 2 boards each for omap3 and Renesas. That's manageable to do by hand or just add to the map. Though in hindsight I probably would have used the config. > This way you would be sure to not miss any. This would also ease the > creation of subdirs if we decide to do that. _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
On Tue, Dec 04, 2018 at 12:36:49PM -0600, Rob Herring wrote: > Olof, Arnd, > > I've put together a script to move the dts files and update the > makefiles. It doesn't handle files not following a common prefix which > isn't many and some includes within the dts files will need some fixups > by hand. > > MAINTAINERS will also need updating. > > A few questions: > > Do we want to move absolutely everything to subdirs? There's quite a > few platforms with only 1-2 platforms. I haven't added these to the > list yet, but can. > > Do any vendors need another level of directories? davinci, omap, nspire, > etc. for TI for example. > > What to do with armv7m.dtsi? I guess it should remain and we just fixup > the include. There may be a few other cross vendor things. > > > Sub-arch maintainers, > 'vendor_map' below is the mapping of file prefix to new subdirectory > (the SoC vendor prefix). Please comment if there are any issues. > > Rob > > 8<----------------------------------------------------------- > #!/usr/bin/env python3 > > import os > import re > from git import Git > import glob > > vendor_map = { > 'imx': 'fsl', > 'ls': 'fsl', > 'vf': 'fsl', > 'qcom': 'qcom', > 'am3' : 'ti', > 'am4' : 'ti', > 'am5' : 'ti', > 'da' : 'ti', > 'dm' : 'ti', > 'dra' : 'ti', > 'keystone' : 'ti', > 'omap' : 'ti', > 'nspire' : 'ti', > 'armada' : 'marvell', > 'berlin' : 'marvell', > 'dove' : 'marvell', > 'kirkwood' : 'marvell', > 'orion' : 'marvell', > 'pxa' : 'marvell', > 'mvebu' : 'marvell', > 'mmp2' : 'marvell', > 'arm-' : 'arm', > 'integ' : 'arm', > 've' : 'arm', > 'aspeed' : 'aspeed', > 'at91' : 'atmel', > 'sama' : 'atmel', > 'bcm' : 'brcm', > 'exynos' : 'samsung', > 's3c' : 'samsung', > 's5p' : 'samsung', > 'gemini' : 'cortina', > 'hi3' : 'hisilicon', > 'hip' : 'hisilicon', > 'hisi' : 'hisilicon', > 'mt' : 'mediatek', > 'meson' : 'amlogic', > 'owl' : 'actions', > 'r7' : 'renesas', > 'r8' : 'renesas', > 'r9' : 'renesas', The above 'r' prefixes could be a bit longer if desired. e.g.: 'r7s' 'r8a' 'r9a' > 'rk' : 'rockchip', > 'socfpga' : 'altera', > 'st' : 'st', > 'spear' : 'st', > 'sun' : 'allwinner', > 'tegra' : 'nvidia', > 'zynq' : 'xilinx', > 'wm' : 'wm', > 'uniph' : 'socionext', > 'zx' : 'zte', For renesas I believe you also want: 'emev2' 'gr-peach' 'iwg20d-q7' 'sh' > } > > if __name__ == "__main__": > g = Git('.') > > g.checkout("HEAD", "arch/arm/boot/dts/Makefile") > dts_make = open("arch/arm/boot/dts/Makefile", "r").read() > > # make entries 1 line > make2 = re.sub(r'\\\n', '', dts_make) > > for k,v in vendor_map.items(): > for f in glob.iglob("arch/arm/boot/dts/" + k + "*.*"): > new_dir = "arch/arm/boot/dts/" + v + "/" > base = os.path.splitext(os.path.basename(f))[0] > > os.makedirs(new_dir, exist_ok=True) > g.mv(f, new_dir) > > # Remove the file from the makefile > dts_make = re.sub('.*' + base + r'\.dtb.*\\\n', '', dts_make) > dts_make = re.sub('.*' + base + r'\.dtb', '', dts_make) > > # extract the matching makefile entry > reg = re.search(r'.*' + base + r'.*', make2) > if not reg: > continue > > entry = reg.group(0) > make2 = re.sub(r'.*' + base + r'.*', '', make2) > if entry: > makefile = open(new_dir + 'Makefile', 'a+') > print(entry, file=makefile) > makefile.close() > > > for d in sorted(glob.iglob("arch/arm/boot/dts/*/")): > dts_make += 'subdir-y += ' + d.split(os.path.sep)[-2] + '\n' > > # Add license and sort entries of sub-dir makefile > vendor_make = '# SPDX-License-Identifier: GPL-2.0\n' > make_lines = open(d + 'Makefile', 'r').readlines() > for l in sorted(make_lines): > vendor_make += l > > vendor_make = re.sub(r'\t', r'\\\n\t', vendor_make) > > f = open(d + 'Makefile', 'w') > f.write(vendor_make) > f.close() > g.add(d + 'Makefile') > > # Remove entries with no dtbs left > dts_make = re.sub(r'.*\+= \\\n\n', '', dts_make) > > open("arch/arm/boot/dts/Makefile", "w").write(dts_make) > g.add("arch/arm/boot/dts/Makefile") > _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Rob, Am 04.12.18 um 19:36 schrieb Rob Herring: > I've put together a script to move the dts files and update the > makefiles. It doesn't handle files not following a common prefix which > isn't many and some includes within the dts files will need some fixups > by hand. > > MAINTAINERS will also need updating. > > A few questions: > > Do we want to move absolutely everything to subdirs? This refactoring is a terrible idea! While it would've been nice to have more structure from the start, bootloaders like U-Boot expect a flat structure for arm .dtb files now. If you start installing them into subdirs instead, they won't find the files anymore under the hardcoded name. Doing this only for new platforms would be much less invasive and allow to prepare bootloaders accordingly. Alternatively, white-list which ones are safe to move around. But don't just script a refactoring because it looks nicer in the source tree, without testing what side effects this can have for board/distro users of the compiled files in practice. We already had that discussion for arm64 because Debian chose to ignore the kernel-installed subdirectories and installed .dtb files into a flat directory, which collided with openSUSE sticking to the kernel choice. This topic becomes even more important with EBBR: There is neither a mechanism in place to sync .dts files into U-Boot or EDK2 source trees, nor are capsule updates implemented in U-Boot for easily deploying such bootloaders with new .dts sources or paths yet. And I can assure you that just getting users to dd the right bootloader can be difficult... Since DT forward and backward compatibility is often being neglected, for example with optional properties or renamed compatibles that break booting with previous drivers, new kernel versions often need updated Device Trees to make use of new/enhanced drivers. Therefore it is unfortunately often enough a necessity to load newer kernel-based .dtb files matching the kernel (as opposed to the dream of kernel-independent hardware descriptions) when working with the latest -rc or -next kernels at least. For examples of DTs needing updates, look no further than Linaro's 96boards - in case of hikey960/EDK2 GRUB is another layer where .dtb paths may be hardcoded, ditto for arm; and Armada was an example where the upstream bindings for the network IP changed incompatibly. DT overlays are another topic that is not making any progress upstream according to the ELCE BoF, so beyond the Raspberry Pi the only known working way to apply them is to write a U-Boot boot.scr script, which can either reuse $fdtcontroladdr DT or use the filename $fdtfile or hardcode one, the latter two of which would break with your renaming. So expect people to be using .dtb files, expect them to be affected by file movements to subdirectories here, and don't expect each user to understand or be able to fix things themselves if they fall apart as result of your changes and they suddenly no longer have Ethernet/Wifi. Regards, Andreas -- SUSE Linux GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany GF: Felix Imendörffer, Jane Smithard, Graham Norton HRB 21284 (AG Nürnberg) _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
On Tue, Dec 4, 2018 at 7:22 PM Andreas Färber <afaerber@suse.de> wrote: > > Rob, > > Am 04.12.18 um 19:36 schrieb Rob Herring: > > I've put together a script to move the dts files and update the > > makefiles. It doesn't handle files not following a common prefix which > > isn't many and some includes within the dts files will need some fixups > > by hand. > > > > MAINTAINERS will also need updating. > > > > A few questions: > > > > Do we want to move absolutely everything to subdirs? > > This refactoring is a terrible idea! How do you really feel? > While it would've been nice to have more structure from the start, > bootloaders like U-Boot expect a flat structure for arm .dtb files now. > If you start installing them into subdirs instead, they won't find the > files anymore under the hardcoded name. > > Doing this only for new platforms would be much less invasive and allow > to prepare bootloaders accordingly. That was my suggestion where this started for the new RDA platform. Olof preferred to move everything and that's my desire too. > Alternatively, white-list which ones > are safe to move around. I'd prefer to know which ones the distros don't want moved. That should be easier to figure out. We also need that anyways in context of what platforms we care about compatibility. Another option is dtbs_install target could flatten the installed dtbs. That is the only part the distros should depend on. > But don't just script a refactoring because it > looks nicer in the source tree, without testing what side effects this > can have for board/distro users of the compiled files in practice. > We already had that discussion for arm64 because Debian chose to ignore > the kernel-installed subdirectories and installed .dtb files into a flat > directory, which collided with openSUSE sticking to the kernel choice. So everyone already deals with subdirs or not with arm and arm64 already, seems like they can deal with this. I will raise the topic on the cross-distro list though. > This topic becomes even more important with EBBR: There is neither a > mechanism in place to sync .dts files into U-Boot or EDK2 source trees, > nor are capsule updates implemented in U-Boot for easily deploying such > bootloaders with new .dts sources or paths yet. EBBR actually says firmware (including dtbs) goes in directories named by vendor. > And I can assure you > that just getting users to dd the right bootloader can be difficult... > Since DT forward and backward compatibility is often being neglected, > for example with optional properties or renamed compatibles that break > booting with previous drivers, new kernel versions often need updated > Device Trees to make use of new/enhanced drivers. Therefore it is > unfortunately often enough a necessity to load newer kernel-based .dtb > files matching the kernel (as opposed to the dream of kernel-independent > hardware descriptions) when working with the latest -rc or -next kernels > at least. For examples of DTs needing updates, look no further than > Linaro's 96boards - in case of hikey960/EDK2 GRUB is another layer where > .dtb paths may be hardcoded, ditto for arm; and Armada was an example > where the upstream bindings for the network IP changed incompatibly. Compatibility is an issue, yes, but that really has nothing to do with this. > DT overlays are another topic that is not making any progress upstream > according to the ELCE BoF, so beyond the Raspberry Pi the only known > working way to apply them is to write a U-Boot boot.scr script, which > can either reuse $fdtcontroladdr DT or use the filename $fdtfile or > hardcode one, the latter two of which would break with your renaming. DT overlays also have nothing to do with this as there aren't any in the kernel. I'm not inclined to take any either with a flat tree. We're already at 1800+ files. Rob _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
On Wed, Dec 5, 2018 at 3:37 AM Rob Herring <robh@kernel.org> wrote: > > Olof, Arnd, > > I've put together a script to move the dts files and update the > makefiles. It doesn't handle files not following a common prefix which > isn't many and some includes within the dts files will need some fixups > by hand. > > MAINTAINERS will also need updating. > > A few questions: > > Do we want to move absolutely everything to subdirs? There's quite a > few platforms with only 1-2 platforms. I haven't added these to the > list yet, but can. > > Do any vendors need another level of directories? davinci, omap, nspire, > etc. for TI for example. > > What to do with armv7m.dtsi? I guess it should remain and we just fixup > the include. There may be a few other cross vendor things. > > > Sub-arch maintainers, > 'vendor_map' below is the mapping of file prefix to new subdirectory > (the SoC vendor prefix). Please comment if there are any issues. > > Rob > > 8<----------------------------------------------------------- > #!/usr/bin/env python3 > > import os > import re > from git import Git > import glob > > vendor_map = { > 'imx': 'fsl', > 'ls': 'fsl', > 'vf': 'fsl', > 'qcom': 'qcom', > 'am3' : 'ti', > 'am4' : 'ti', > 'am5' : 'ti', > 'da' : 'ti', > 'dm' : 'ti', > 'dra' : 'ti', > 'keystone' : 'ti', > 'omap' : 'ti', > 'nspire' : 'ti', > 'armada' : 'marvell', > 'berlin' : 'marvell', > 'dove' : 'marvell', > 'kirkwood' : 'marvell', > 'orion' : 'marvell', > 'pxa' : 'marvell', > 'mvebu' : 'marvell', > 'mmp2' : 'marvell', > 'arm-' : 'arm', > 'integ' : 'arm', > 've' : 'arm', > 'aspeed' : 'aspeed', > 'at91' : 'atmel', > 'sama' : 'atmel', > 'bcm' : 'brcm', > 'exynos' : 'samsung', > 's3c' : 'samsung', > 's5p' : 'samsung', > 'gemini' : 'cortina', > 'hi3' : 'hisilicon', > 'hip' : 'hisilicon', > 'hisi' : 'hisilicon', > 'mt' : 'mediatek', > 'meson' : 'amlogic', > 'owl' : 'actions', > 'r7' : 'renesas', > 'r8' : 'renesas', > 'r9' : 'renesas', > 'rk' : 'rockchip', > 'socfpga' : 'altera', > 'st' : 'st', > 'spear' : 'st', > 'sun' : 'allwinner', > 'tegra' : 'nvidia', > 'zynq' : 'xilinx', > 'wm' : 'wm', > 'uniph' : 'socionext', I tested this script, and I confirmed all of my DT files were moved to the socionext subdirectory. However, this would break arm64 build because some of my arm64 DT files include arm32 ones. I think it is common to share base platform boards between arm and arm64. Of course, It will be pretty easy to fix it up by hand later, though. > 'zx' : 'zte', > } > > if __name__ == "__main__": > g = Git('.') > > g.checkout("HEAD", "arch/arm/boot/dts/Makefile") > dts_make = open("arch/arm/boot/dts/Makefile", "r").read() > > # make entries 1 line > make2 = re.sub(r'\\\n', '', dts_make) > > for k,v in vendor_map.items(): > for f in glob.iglob("arch/arm/boot/dts/" + k + "*.*"): > new_dir = "arch/arm/boot/dts/" + v + "/" > base = os.path.splitext(os.path.basename(f))[0] > > os.makedirs(new_dir, exist_ok=True) > g.mv(f, new_dir) > > # Remove the file from the makefile > dts_make = re.sub('.*' + base + r'\.dtb.*\\\n', '', dts_make) > dts_make = re.sub('.*' + base + r'\.dtb', '', dts_make) > > # extract the matching makefile entry > reg = re.search(r'.*' + base + r'.*', make2) > if not reg: > continue > > entry = reg.group(0) > make2 = re.sub(r'.*' + base + r'.*', '', make2) > if entry: > makefile = open(new_dir + 'Makefile', 'a+') > print(entry, file=makefile) > makefile.close() > > > for d in sorted(glob.iglob("arch/arm/boot/dts/*/")): > dts_make += 'subdir-y += ' + d.split(os.path.sep)[-2] + '\n' > > # Add license and sort entries of sub-dir makefile > vendor_make = '# SPDX-License-Identifier: GPL-2.0\n' > make_lines = open(d + 'Makefile', 'r').readlines() > for l in sorted(make_lines): > vendor_make += l > > vendor_make = re.sub(r'\t', r'\\\n\t', vendor_make) > > f = open(d + 'Makefile', 'w') > f.write(vendor_make) > f.close() > g.add(d + 'Makefile') > > # Remove entries with no dtbs left > dts_make = re.sub(r'.*\+= \\\n\n', '', dts_make) > > open("arch/arm/boot/dts/Makefile", "w").write(dts_make) > g.add("arch/arm/boot/dts/Makefile") -- Best Regards Masahiro Yamada _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Hi Rob, On Tue, 4 Dec 2018 12:36:49 -0600 Rob Herring <robh@kernel.org> wrote: > Olof, Arnd, > > I've put together a script to move the dts files and update the > makefiles. It doesn't handle files not following a common prefix which > isn't many and some includes within the dts files will need some fixups > by hand. > > MAINTAINERS will also need updating. > > A few questions: > > Do we want to move absolutely everything to subdirs? There's quite a > few platforms with only 1-2 platforms. I haven't added these to the > list yet, but can. > > Do any vendors need another level of directories? davinci, omap, nspire, > etc. for TI for example. > > What to do with armv7m.dtsi? I guess it should remain and we just fixup > the include. There may be a few other cross vendor things. > > > Sub-arch maintainers, > 'vendor_map' below is the mapping of file prefix to new subdirectory > (the SoC vendor prefix). Please comment if there are any issues. > > Rob > > 8<----------------------------------------------------------- > #!/usr/bin/env python3 > > import os > import re > from git import Git > import glob > > vendor_map = { > 'imx': 'fsl', > 'ls': 'fsl', > 'vf': 'fsl', > 'qcom': 'qcom', > 'am3' : 'ti', > 'am4' : 'ti', > 'am5' : 'ti', > 'da' : 'ti', > 'dm' : 'ti', > 'dra' : 'ti', > 'keystone' : 'ti', > 'omap' : 'ti', > 'nspire' : 'ti', > 'armada' : 'marvell', > 'berlin' : 'marvell', Now, berlin SoC is synaptics' SoC ;) Thanks _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
On 04/12/2018 at 19:36, Rob Herring wrote: > Olof, Arnd, > > I've put together a script to move the dts files and update the > makefiles. It doesn't handle files not following a common prefix which > isn't many and some includes within the dts files will need some fixups > by hand. > > MAINTAINERS will also need updating. > > A few questions: > > Do we want to move absolutely everything to subdirs? There's quite a > few platforms with only 1-2 platforms. I haven't added these to the > list yet, but can. > > Do any vendors need another level of directories? davinci, omap, nspire, > etc. for TI for example. > > What to do with armv7m.dtsi? I guess it should remain and we just fixup > the include. There may be a few other cross vendor things. > > > Sub-arch maintainers, > 'vendor_map' below is the mapping of file prefix to new subdirectory > (the SoC vendor prefix). Please comment if there are any issues. It'll certainly break some of our scripts (CI, building scripts, overlay makefiles, etc.). We already experienced it when adding the "dts" directory. But if it's needed... > 8<----------------------------------------------------------- > #!/usr/bin/env python3 > > import os > import re > from git import Git > import glob > > vendor_map = { [..] > 'integ' : 'arm', > 've' : 'arm', > 'aspeed' : 'aspeed', > 'at91' : 'atmel', As it's a new directory, what about creating a "microchip" one as it's our vendor's name now. > 'sama' : 'atmel', If desired, you can extend to sama5d for the current files. Adding the entry 'sam9' : "microchip", can be future proof... Best regards, -- Nicolas Ferre _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
On Wed, Dec 5, 2018 at 7:02 AM Jisheng Zhang <Jisheng.Zhang@synaptics.com> wrote: > Rob Herring <robh@kernel.org> wrote: > > 'armada' : 'marvell', > > 'berlin' : 'marvell', > > Now, berlin SoC is synaptics' SoC ;) This illustrates perfectly the artificial nature of using vendor names as prefixes with DT properties, prefix names, directories etc. Companies start out purporting to be some eternal entity and the next day they buy each other left and right and license their hardware IP to whoever wants it. It actually makes much more sense to organize these files by the SoC family name, because that doesn't change when the SoC is sold to another company. omap/* containing all OMAP platforms, msm/* for all Qualcomm SoCs etc. SoC names/codenames are at least eternal once they have been manufactured and we can keep them together no matter what vendor currently controls it. However I think there was a fork in the road ages ago when someone or something decided to use vendor prefixes for DT properties leading to this situation that we can no longer back out of. It has the side effect of splitting DTS files with the same SoC in two different folders marvell/* and synaptics/* it's a bit meh. Yours, Linus Walleij _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
On Wed, 5 Dec 2018 09:19:49 +0100 Linus Walleij wrote: > On Wed, Dec 5, 2018 at 7:02 AM Jisheng Zhang wrote: > > Rob Herring wrote: > > > > 'armada' : 'marvell', > > > 'berlin' : 'marvell', > > > > Now, berlin SoC is synaptics' SoC ;) > > This illustrates perfectly the artificial nature of using vendor names > as prefixes with DT properties, prefix names, directories etc. > > Companies start out purporting to be some eternal entity and the > next day they buy each other left and right and license their > hardware IP to whoever wants it. > > It actually makes much more sense to organize these files by > the SoC family name, because that doesn't change when the > SoC is sold to another company. If the SoC is sold to another company, then case1: The original SoC family is renamed to another family. case2: Based on the original SoC, a newer SoC family comes out. I'm not sure it's still fine to put the new or renamed SoCs' files into the original SoC directory. Another issue is: who will be the maintainer of new or renamed SoC family? Thanks, Jisheng > > omap/* containing all OMAP platforms, msm/* for all Qualcomm > SoCs etc. SoC names/codenames are at least eternal once they > have been manufactured and we can keep them together > no matter what vendor currently controls it. > > However I think there was a fork in the road ages ago when > someone or something decided to use vendor prefixes for > DT properties leading to this situation that we can no longer > back out of. > > It has the side effect of splitting DTS files with the same SoC > in two different folders marvell/* and synaptics/* > it's a bit meh. > > Yours, > Linus Walleij _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
On Wed, Dec 5, 2018 at 9:34 AM Jisheng Zhang <Jisheng.Zhang@synaptics.com> wrote: > [Me] > > It actually makes much more sense to organize these files by > > the SoC family name, because that doesn't change when the > > SoC is sold to another company. > > If the SoC is sold to another company, then > > case1: The original SoC family is renamed to another family. > > case2: Based on the original SoC, a newer SoC family comes out. > > I'm not sure it's still fine to put the new or renamed SoCs' files into the > original SoC directory. > > Another issue is: who will be the maintainer of new or renamed SoC family? Of course marketing prefer to associate the acquired company's SoC with itself and rename it. But that doesn't account for the case where the same SoC is produced by the new owner under the old name. Example: The Gemini SoC arch/arm/boot/dts/gemini-* https://dflund.se/~triad/krad/gemini/ This SoC was produced by StorLink Semiconductor in Taiwan early 2000s. At some point between 2000 and 2008 the company changed its name and/or got restructured and was renamed Storm Semiconductor. The masks for the packaging was retained so the chips still said "StorLink". Then in 2008 Cortina Systems acquired Storm Semiconductor eventually changed the masks and and renamed the identical chips from "SLnnnn" to "CSnnnn" but the chip inside the package was still the same. The SoC is still codenamed "Gemini", that is the only constant. And now we will put these devicetrees into folder...? storlink/ storm/ cortina/ If we look at the chip packageing, it should be different folders depending on what supplier was used when the device was manufactured. This is why the Vendor scheme is not really working. Another issue is illustrated again with the Synaptics device: an SoC produced exclusively for Synaptics (IIUC) but with a intimate relationship to a very easily identifed SoC. This naming nomenclature is a half-measure and a can of worms, that is my only point. Yours, Linus Walleij _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
On 05. 12. 18 5:18, Masahiro Yamada wrote: > On Wed, Dec 5, 2018 at 3:37 AM Rob Herring <robh@kernel.org> wrote: >> >> Olof, Arnd, >> >> I've put together a script to move the dts files and update the >> makefiles. It doesn't handle files not following a common prefix which >> isn't many and some includes within the dts files will need some fixups >> by hand. >> >> MAINTAINERS will also need updating. >> >> A few questions: >> >> Do we want to move absolutely everything to subdirs? There's quite a >> few platforms with only 1-2 platforms. I haven't added these to the >> list yet, but can. >> >> Do any vendors need another level of directories? davinci, omap, nspire, >> etc. for TI for example. >> >> What to do with armv7m.dtsi? I guess it should remain and we just fixup >> the include. There may be a few other cross vendor things. >> >> >> Sub-arch maintainers, >> 'vendor_map' below is the mapping of file prefix to new subdirectory >> (the SoC vendor prefix). Please comment if there are any issues. >> >> Rob >> >> 8<----------------------------------------------------------- >> #!/usr/bin/env python3 >> >> import os >> import re >> from git import Git >> import glob >> >> vendor_map = { >> 'imx': 'fsl', >> 'ls': 'fsl', >> 'vf': 'fsl', >> 'qcom': 'qcom', >> 'am3' : 'ti', >> 'am4' : 'ti', >> 'am5' : 'ti', >> 'da' : 'ti', >> 'dm' : 'ti', >> 'dra' : 'ti', >> 'keystone' : 'ti', >> 'omap' : 'ti', >> 'nspire' : 'ti', >> 'armada' : 'marvell', >> 'berlin' : 'marvell', >> 'dove' : 'marvell', >> 'kirkwood' : 'marvell', >> 'orion' : 'marvell', >> 'pxa' : 'marvell', >> 'mvebu' : 'marvell', >> 'mmp2' : 'marvell', >> 'arm-' : 'arm', >> 'integ' : 'arm', >> 've' : 'arm', >> 'aspeed' : 'aspeed', >> 'at91' : 'atmel', >> 'sama' : 'atmel', >> 'bcm' : 'brcm', >> 'exynos' : 'samsung', >> 's3c' : 'samsung', >> 's5p' : 'samsung', >> 'gemini' : 'cortina', >> 'hi3' : 'hisilicon', >> 'hip' : 'hisilicon', >> 'hisi' : 'hisilicon', >> 'mt' : 'mediatek', >> 'meson' : 'amlogic', >> 'owl' : 'actions', >> 'r7' : 'renesas', >> 'r8' : 'renesas', >> 'r9' : 'renesas', >> 'rk' : 'rockchip', >> 'socfpga' : 'altera', >> 'st' : 'st', >> 'spear' : 'st', >> 'sun' : 'allwinner', >> 'tegra' : 'nvidia', >> 'zynq' : 'xilinx', >> 'wm' : 'wm', >> 'uniph' : 'socionext', > > > I tested this script, and > I confirmed all of my DT files were moved to the socionext subdirectory. > > However, this would break arm64 build > because some of my arm64 DT files include arm32 ones. > > I think it is common to share base platform boards between arm and arm64. +1 Just a note here that for Zynq and ZynqMP there should be doable to run Microblaze in programmable logic with reusing fixed IPs. There could be also other soft core CPUs which could do this on Xilinx devices. It means maybe moving this out of arm/arm64 to more generic location would be one more step to consider as the part of this move. Thanks, Michal _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
On Wed, Dec 5, 2018 at 2:20 AM Linus Walleij <linus.walleij@linaro.org> wrote: > > On Wed, Dec 5, 2018 at 7:02 AM Jisheng Zhang > <Jisheng.Zhang@synaptics.com> wrote: > > Rob Herring <robh@kernel.org> wrote: > > > > 'armada' : 'marvell', > > > 'berlin' : 'marvell', > > > > Now, berlin SoC is synaptics' SoC ;) But we're not changing the names of existing SoCs and at least so far there aren't new 32-bit berlin SoCs. > This illustrates perfectly the artificial nature of using vendor names > as prefixes with DT properties, prefix names, directories etc. > > Companies start out purporting to be some eternal entity and the > next day they buy each other left and right and license their > hardware IP to whoever wants it. > > It actually makes much more sense to organize these files by > the SoC family name, because that doesn't change when the > SoC is sold to another company. Uh, no. i.MX23 is a Sigmatel chip STMP?? which shared absolutely nothing with other i.MX chips except for the derivatives that followed. IIRC, STMP chips were even partially upstream. Marketing names change at the whim of marketing and don't even require a sale of a company. > omap/* containing all OMAP platforms, msm/* for all Qualcomm > SoCs etc. SoC names/codenames are at least eternal once they > have been manufactured and we can keep them together > no matter what vendor currently controls it. Where do TI amXXXX chips go? Not all QC chips are 'msm' and I think that name is abandoned now. No solution is going to be perfect. We are already use vendors for arm64 (except for the oddball exynos), so if we change arm, we shouldn't do something different. EBBR is also going with vendor for firmware directories in the EFI system partition. Speak up if you want to change that before 1.0. > However I think there was a fork in the road ages ago when > someone or something decided to use vendor prefixes for > DT properties leading to this situation that we can no longer > back out of. It was simpler times... > It has the side effect of splitting DTS files with the same SoC > in two different folders marvell/* and synaptics/* > it's a bit meh. All I really care about here is things are organized by maintainers. Someone care to write a script to ensure all 1800 files have a maintainer attached to them (that is not me)? Rob _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
On 04/12/2018 19:36, Rob Herring wrote: > Olof, Arnd, > > I've put together a script to move the dts files and update the > makefiles. It doesn't handle files not following a common prefix which > isn't many and some includes within the dts files will need some fixups > by hand. > > MAINTAINERS will also need updating. > > A few questions: > > Do we want to move absolutely everything to subdirs? There's quite a > few platforms with only 1-2 platforms. I haven't added these to the > list yet, but can. > > Do any vendors need another level of directories? davinci, omap, nspire, > etc. for TI for example. > > What to do with armv7m.dtsi? I guess it should remain and we just fixup > the include. There may be a few other cross vendor things. > > > Sub-arch maintainers, > 'vendor_map' below is the mapping of file prefix to new subdirectory > (the SoC vendor prefix). Please comment if there are any issues. > > Rob > > 8<----------------------------------------------------------- > #!/usr/bin/env python3 > > import os > import re > from git import Git > import glob > > vendor_map = { > 'imx': 'fsl', > 'ls': 'fsl', > 'vf': 'fsl', > 'qcom': 'qcom', > 'am3' : 'ti', > 'am4' : 'ti', > 'am5' : 'ti', > 'da' : 'ti', > 'dm' : 'ti', > 'dra' : 'ti', > 'keystone' : 'ti', > 'omap' : 'ti', > 'nspire' : 'ti', > 'armada' : 'marvell', > 'berlin' : 'marvell', > 'dove' : 'marvell', > 'kirkwood' : 'marvell', > 'orion' : 'marvell', > 'pxa' : 'marvell', > 'mvebu' : 'marvell', > 'mmp2' : 'marvell', > 'arm-' : 'arm', > 'integ' : 'arm', > 've' : 'arm', > 'aspeed' : 'aspeed', > 'at91' : 'atmel', > 'sama' : 'atmel', > 'bcm' : 'brcm', > 'exynos' : 'samsung', > 's3c' : 'samsung', > 's5p' : 'samsung', > 'gemini' : 'cortina', > 'hi3' : 'hisilicon', > 'hip' : 'hisilicon', > 'hisi' : 'hisilicon', > 'mt' : 'mediatek', > 'meson' : 'amlogic', > 'owl' : 'actions', > 'r7' : 'renesas', > 'r8' : 'renesas', > 'r9' : 'renesas', > 'rk' : 'rockchip', > 'socfpga' : 'altera', > 'st' : 'st', > 'spear' : 'st', > 'sun' : 'allwinner', > 'tegra' : 'nvidia', > 'zynq' : 'xilinx', > 'wm' : 'wm', > 'uniph' : 'socionext', > 'zx' : 'zte', 'ox8' : 'oxsemi', I volunteer as a tribute ! This is a low-priority platform, you can move them in a separate directory if you want. Neil > } > > if __name__ == "__main__": > g = Git('.') > > g.checkout("HEAD", "arch/arm/boot/dts/Makefile") > dts_make = open("arch/arm/boot/dts/Makefile", "r").read() > > # make entries 1 line > make2 = re.sub(r'\\\n', '', dts_make) > > for k,v in vendor_map.items(): > for f in glob.iglob("arch/arm/boot/dts/" + k + "*.*"): > new_dir = "arch/arm/boot/dts/" + v + "/" > base = os.path.splitext(os.path.basename(f))[0] > > os.makedirs(new_dir, exist_ok=True) > g.mv(f, new_dir) > > # Remove the file from the makefile > dts_make = re.sub('.*' + base + r'\.dtb.*\\\n', '', dts_make) > dts_make = re.sub('.*' + base + r'\.dtb', '', dts_make) > > # extract the matching makefile entry > reg = re.search(r'.*' + base + r'.*', make2) > if not reg: > continue > > entry = reg.group(0) > make2 = re.sub(r'.*' + base + r'.*', '', make2) > if entry: > makefile = open(new_dir + 'Makefile', 'a+') > print(entry, file=makefile) > makefile.close() > > > for d in sorted(glob.iglob("arch/arm/boot/dts/*/")): > dts_make += 'subdir-y += ' + d.split(os.path.sep)[-2] + '\n' > > # Add license and sort entries of sub-dir makefile > vendor_make = '# SPDX-License-Identifier: GPL-2.0\n' > make_lines = open(d + 'Makefile', 'r').readlines() > for l in sorted(make_lines): > vendor_make += l > > vendor_make = re.sub(r'\t', r'\\\n\t', vendor_make) > > f = open(d + 'Makefile', 'w') > f.write(vendor_make) > f.close() > g.add(d + 'Makefile') > > # Remove entries with no dtbs left > dts_make = re.sub(r'.*\+= \\\n\n', '', dts_make) > > open("arch/arm/boot/dts/Makefile", "w").write(dts_make) > g.add("arch/arm/boot/dts/Makefile") > > _______________________________________________ > linux-arm-kernel mailing list > linux-arm-kernel@lists.infradead.org > http://lists.infradead.org/mailman/listinfo/linux-arm-kernel > _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
[-- Attachment #1.1: Type: text/plain, Size: 3061 bytes --] On Tue, Dec 04, 2018 at 10:17:15PM -0600, Rob Herring wrote: > On Tue, Dec 4, 2018 at 7:22 PM Andreas Färber <afaerber@suse.de> wrote: > > > > Rob, > > > > Am 04.12.18 um 19:36 schrieb Rob Herring: > > > I've put together a script to move the dts files and update the > > > makefiles. It doesn't handle files not following a common prefix which > > > isn't many and some includes within the dts files will need some fixups > > > by hand. > > > > > > MAINTAINERS will also need updating. > > > > > > A few questions: > > > > > > Do we want to move absolutely everything to subdirs? > > > > This refactoring is a terrible idea! > > How do you really feel? > > > While it would've been nice to have more structure from the start, > > bootloaders like U-Boot expect a flat structure for arm .dtb files now. > > If you start installing them into subdirs instead, they won't find the > > files anymore under the hardcoded name. > > > > Doing this only for new platforms would be much less invasive and allow > > to prepare bootloaders accordingly. > > That was my suggestion where this started for the new RDA platform. > Olof preferred to move everything and that's my desire too. I think we might really be stuck with "new stuff only in subdirs". There's a whole huge world out there that depends on these layouts and it's almost but not quite like the API vs ABI thing. You can change the kernel API at whim but this is something with a lot of external users, and those folks aren't supposed to get broken. > > Alternatively, white-list which ones > > are safe to move around. > > I'd prefer to know which ones the distros don't want moved. That > should be easier to figure out. We also need that anyways in context > of what platforms we care about compatibility. > > Another option is dtbs_install target could flatten the installed > dtbs. That is the only part the distros should depend on. But it's not just "the distros" that'll get bit by this too. Or rather, it's the "big distros" and all the OpenEmbedded/buildroot/etc ones too that follow off the built-in logic to find the DTB that'll get bit too with a rename. [snip] > > DT overlays are another topic that is not making any progress upstream > > according to the ELCE BoF, so beyond the Raspberry Pi the only known > > working way to apply them is to write a U-Boot boot.scr script, which > > can either reuse $fdtcontroladdr DT or use the filename $fdtfile or > > hardcode one, the latter two of which would break with your renaming. > > DT overlays also have nothing to do with this as there aren't any in > the kernel. I'm not inclined to take any either with a flat tree. > We're already at 1800+ files. I'm going to do my best to not hijack the thread to be about overlays but that's a whole chicken and egg type problem too. But yes, it's not exactly related to moving or not existing dts/dtb files but something about where / how those are to be stored and maintained really does need to be done. -- Tom [-- Attachment #1.2: signature.asc --] [-- Type: application/pgp-signature, Size: 819 bytes --] [-- Attachment #2: Type: text/plain, Size: 176 bytes --] _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
On Tue, Dec 4, 2018 at 12:37 PM Rob Herring <robh@kernel.org> wrote: > > Olof, Arnd, > > I've put together a script to move the dts files and update the > makefiles. It doesn't handle files not following a common prefix which > isn't many and some includes within the dts files will need some fixups > by hand. > > MAINTAINERS will also need updating. > > A few questions: > > Do we want to move absolutely everything to subdirs? There's quite a > few platforms with only 1-2 platforms. I haven't added these to the > list yet, but can. > > Do any vendors need another level of directories? davinci, omap, nspire, > etc. for TI for example. > > What to do with armv7m.dtsi? I guess it should remain and we just fixup > the include. There may be a few other cross vendor things. > > > Sub-arch maintainers, > 'vendor_map' below is the mapping of file prefix to new subdirectory > (the SoC vendor prefix). Please comment if there are any issues. > > Rob > > 8<----------------------------------------------------------- > #!/usr/bin/env python3 > > import os > import re > from git import Git > import glob > > vendor_map = { > 'imx': 'fsl', > 'ls': 'fsl', > 'vf': 'fsl', Probably we can update "fsl" to "nxp" now since you are at it. :) We didn't try to update this previously because for the past two years we were thinking that it will need to be updated to qcom again soon. > 'qcom': 'qcom', > 'am3' : 'ti', > 'am4' : 'ti', > 'am5' : 'ti', > 'da' : 'ti', > 'dm' : 'ti', > 'dra' : 'ti', > 'keystone' : 'ti', > 'omap' : 'ti', > 'nspire' : 'ti', > 'armada' : 'marvell', > 'berlin' : 'marvell', > 'dove' : 'marvell', > 'kirkwood' : 'marvell', > 'orion' : 'marvell', > 'pxa' : 'marvell', > 'mvebu' : 'marvell', > 'mmp2' : 'marvell', > 'arm-' : 'arm', > 'integ' : 'arm', > 've' : 'arm', > 'aspeed' : 'aspeed', > 'at91' : 'atmel', > 'sama' : 'atmel', > 'bcm' : 'brcm', > 'exynos' : 'samsung', > 's3c' : 'samsung', > 's5p' : 'samsung', > 'gemini' : 'cortina', > 'hi3' : 'hisilicon', > 'hip' : 'hisilicon', > 'hisi' : 'hisilicon', > 'mt' : 'mediatek', > 'meson' : 'amlogic', > 'owl' : 'actions', > 'r7' : 'renesas', > 'r8' : 'renesas', > 'r9' : 'renesas', > 'rk' : 'rockchip', > 'socfpga' : 'altera', > 'st' : 'st', > 'spear' : 'st', > 'sun' : 'allwinner', > 'tegra' : 'nvidia', > 'zynq' : 'xilinx', > 'wm' : 'wm', > 'uniph' : 'socionext', > 'zx' : 'zte', > } > > if __name__ == "__main__": > g = Git('.') > > g.checkout("HEAD", "arch/arm/boot/dts/Makefile") > dts_make = open("arch/arm/boot/dts/Makefile", "r").read() > > # make entries 1 line > make2 = re.sub(r'\\\n', '', dts_make) > > for k,v in vendor_map.items(): > for f in glob.iglob("arch/arm/boot/dts/" + k + "*.*"): > new_dir = "arch/arm/boot/dts/" + v + "/" > base = os.path.splitext(os.path.basename(f))[0] > > os.makedirs(new_dir, exist_ok=True) > g.mv(f, new_dir) > > # Remove the file from the makefile > dts_make = re.sub('.*' + base + r'\.dtb.*\\\n', '', dts_make) > dts_make = re.sub('.*' + base + r'\.dtb', '', dts_make) > > # extract the matching makefile entry > reg = re.search(r'.*' + base + r'.*', make2) > if not reg: > continue > > entry = reg.group(0) > make2 = re.sub(r'.*' + base + r'.*', '', make2) > if entry: > makefile = open(new_dir + 'Makefile', 'a+') > print(entry, file=makefile) > makefile.close() > > > for d in sorted(glob.iglob("arch/arm/boot/dts/*/")): > dts_make += 'subdir-y += ' + d.split(os.path.sep)[-2] + '\n' > > # Add license and sort entries of sub-dir makefile > vendor_make = '# SPDX-License-Identifier: GPL-2.0\n' > make_lines = open(d + 'Makefile', 'r').readlines() > for l in sorted(make_lines): > vendor_make += l > > vendor_make = re.sub(r'\t', r'\\\n\t', vendor_make) > > f = open(d + 'Makefile', 'w') > f.write(vendor_make) > f.close() > g.add(d + 'Makefile') > > # Remove entries with no dtbs left > dts_make = re.sub(r'.*\+= \\\n\n', '', dts_make) > > open("arch/arm/boot/dts/Makefile", "w").write(dts_make) > g.add("arch/arm/boot/dts/Makefile") _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
On Wed, Dec 5, 2018 at 4:02 PM Rob Herring <robh@kernel.org> wrote: > We are already use vendors for arm64 (except for the oddball exynos), > so if we change arm, we shouldn't do something different. EBBR is also > going with vendor for firmware directories in the EFI system > partition. Speak up if you want to change that before 1.0. I don't think there is a silver bullet for this, except "use vendor or whatever makes most sense", like if in doubt use vendor but no need to overspecify. Rough consensus, not perfect consensus. > All I really care about here is things are organized by maintainers. > Someone care to write a script to ensure all 1800 files have a > maintainer attached to them (that is not me)? OK why not let the maintainers choose the name of the subdir rather than insist to set it to $vendor? Just consistency? For the stuff I maintain please use: 'arm-' : 'arm', 'integ' : 'arm', 've' : 'arm', These are fine. Drop this - 'gemini' : 'cortina', Just use the "gemini" prefix. Because this was three different vendors so let's stick with the SoC codename. And for 'ste' I would use 'st-ericsson' or 'stericsson' (also a valid vendor) so it is readable. It's no super-big deal so if you think my view is just annoying I am not going to persist, I will just get over it. Thanks, Linus Walleij _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Am 05.12.18 um 05:17 schrieb Rob Herring: > On Tue, Dec 4, 2018 at 7:22 PM Andreas Färber <afaerber@suse.de> wrote: >> >> Rob, >> >> Am 04.12.18 um 19:36 schrieb Rob Herring: >>> I've put together a script to move the dts files and update the >>> makefiles. It doesn't handle files not following a common prefix which >>> isn't many and some includes within the dts files will need some fixups >>> by hand. >>> >>> MAINTAINERS will also need updating. >>> >>> A few questions: >>> >>> Do we want to move absolutely everything to subdirs? >> >> This refactoring is a terrible idea! > > How do you really feel? > >> While it would've been nice to have more structure from the start, >> bootloaders like U-Boot expect a flat structure for arm .dtb files now. >> If you start installing them into subdirs instead, they won't find the >> files anymore under the hardcoded name. >> >> Doing this only for new platforms would be much less invasive and allow >> to prepare bootloaders accordingly. > > That was my suggestion where this started for the new RDA platform. > Olof preferred to move everything and that's my desire too. > >> Alternatively, white-list which ones >> are safe to move around. > > I'd prefer to know which ones the distros don't want moved. That > should be easier to figure out. We also need that anyways in context > of what platforms we care about compatibility. > > Another option is dtbs_install target could flatten the installed > dtbs. That is the only part the distros should depend on. I'd be okay with distinguishing source vs. install location. Due to the issue I mention below (and more) we can't use install_dtbs for openSUSE and had to reimplement it, which we'd need to (and can) adjust. >> But don't just script a refactoring because it >> looks nicer in the source tree, without testing what side effects this >> can have for board/distro users of the compiled files in practice. >> We already had that discussion for arm64 because Debian chose to ignore >> the kernel-installed subdirectories and installed .dtb files into a flat >> directory, which collided with openSUSE sticking to the kernel choice. > > So everyone already deals with subdirs or not with arm and arm64 > already, seems like they can deal with this. I will raise the topic on > the cross-distro list though. Sounds like you're twisting words... The keyword was "hardcoded" paths - one way or another (not "and") depending on the kernel choices being flat for arm, vendor subdir for arm64. >> This topic becomes even more important with EBBR: There is neither a >> mechanism in place to sync .dts files into U-Boot or EDK2 source trees, >> nor are capsule updates implemented in U-Boot for easily deploying such >> bootloaders with new .dts sources or paths yet. > > EBBR actually says firmware (including dtbs) goes in directories named > by vendor. Fine, but unrelated. >> And I can assure you >> that just getting users to dd the right bootloader can be difficult... >> Since DT forward and backward compatibility is often being neglected, >> for example with optional properties or renamed compatibles that break >> booting with previous drivers, new kernel versions often need updated >> Device Trees to make use of new/enhanced drivers. Therefore it is >> unfortunately often enough a necessity to load newer kernel-based .dtb >> files matching the kernel (as opposed to the dream of kernel-independent >> hardware descriptions) when working with the latest -rc or -next kernels >> at least. For examples of DTs needing updates, look no further than >> Linaro's 96boards - in case of hikey960/EDK2 GRUB is another layer where >> .dtb paths may be hardcoded, ditto for arm; and Armada was an example >> where the upstream bindings for the network IP changed incompatibly. > > Compatibility is an issue, yes, but that really has nothing to do with this. > >> DT overlays are another topic that is not making any progress upstream >> according to the ELCE BoF, so beyond the Raspberry Pi the only known >> working way to apply them is to write a U-Boot boot.scr script, which >> can either reuse $fdtcontroladdr DT or use the filename $fdtfile or >> hardcode one, the latter two of which would break with your renaming. > > DT overlays also have nothing to do with this as there aren't any in > the kernel. I'm not inclined to take any either with a flat tree. > We're already at 1800+ files. Read again: a) Breaking DT changes and b) the desire to use Overlays instead of replacing the bootloaders for each change are _reasons_ why people depend on .dtb filenames from the kernel source tree for their boot flow today. Nothing to do with downstream .dtbo files. For example, remember when I reported that the kernel didn't compile DTs with -@? No reaction except for Frank asking to be CC'ed - was it ever fixed??? Do EDK2's or U-Boot's built-in DTs compile with -@ today? Raspberry Pi overlays in U-Boot work because it switched to passing the DT through from the proprietary firmware. Point being that while it would be nice to get a current, compatible DT via UEFI tables and ignore .dtb filenames outside of a few bootloaders, in reality we're not quite there yet for all platforms. I see no problem (except for naming choices) moving new targets like RDA to subfolders because we can then hardcode it the new way; I also assume deeply embedded targets like stm32f4 or targets with no mainline bootloaders yet like owl-s500 could be refactored. I do see problems refactoring widely used SBC targets like sunXi though. Regards, Andreas -- SUSE Linux GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany GF: Felix Imendörffer, Jane Smithard, Graham Norton HRB 21284 (AG Nürnberg) _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Hello, On Wed, Dec 05, 2018 at 09:01:59AM -0600, Rob Herring wrote: > i.MX23 is a Sigmatel chip STMP?? I think the Freescale i.MX23 didn't exist at Sigmatel back then. AFAIK this is a new design using IP from Sigmatel after the aquisition. Best regards Uwe -- Pengutronix e.K. | Uwe Kleine-König | Industrial Linux Solutions | http://www.pengutronix.de/ | _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
On Thu, Dec 6, 2018 at 7:39 AM Uwe Kleine-König <u.kleine-koenig@pengutronix.de> wrote: > > Hello, > > On Wed, Dec 05, 2018 at 09:01:59AM -0600, Rob Herring wrote: > > i.MX23 is a Sigmatel chip STMP?? > > I think the Freescale i.MX23 didn't exist at Sigmatel back then. AFAIK > this is a new design using IP from Sigmatel after the aquisition. It is not. I was in the i.MX group which Sigmatel was merged into at the time. Purely marketing rebranding. Rob _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
On 06/12/2018 07:58:24-0600, Rob Herring wrote: > On Thu, Dec 6, 2018 at 7:39 AM Uwe Kleine-König > <u.kleine-koenig@pengutronix.de> wrote: > > > > Hello, > > > > On Wed, Dec 05, 2018 at 09:01:59AM -0600, Rob Herring wrote: > > > i.MX23 is a Sigmatel chip STMP?? > > > > I think the Freescale i.MX23 didn't exist at Sigmatel back then. AFAIK > > this is a new design using IP from Sigmatel after the aquisition. > > It is not. I was in the i.MX group which Sigmatel was merged into at > the time. Purely marketing rebranding. > Wouldn't it be easier to name the directory to the corresponding mach-* entry? So, imx23 and imx28 would go to mxs/, other imx in imx/. And this also solves the Marvell mess with the Synaptics Socs going in berlin/ and the other ones in mvebu/. -- Alexandre Belloni, Bootlin Embedded Linux and Kernel engineering https://bootlin.com _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
On Thu, Dec 6, 2018 at 3:05 PM Alexandre Belloni <alexandre.belloni@bootlin.com> wrote: > On 06/12/2018 07:58:24-0600, Rob Herring wrote: > > On Thu, Dec 6, 2018 at 7:39 AM Uwe Kleine-König > > <u.kleine-koenig@pengutronix.de> wrote: > > > > > > Hello, > > > > > > On Wed, Dec 05, 2018 at 09:01:59AM -0600, Rob Herring wrote: > > > > i.MX23 is a Sigmatel chip STMP?? > > > > > > I think the Freescale i.MX23 didn't exist at Sigmatel back then. AFAIK > > > this is a new design using IP from Sigmatel after the aquisition. > > > > It is not. I was in the i.MX group which Sigmatel was merged into at > > the time. Purely marketing rebranding. > > > > Wouldn't it be easier to name the directory to the corresponding mach-* > entry? > > So, imx23 and imx28 would go to mxs/, other imx in imx/. And this also > solves the Marvell mess with the Synaptics Socs going in berlin/ and the > other ones in mvebu/. I like this idea. We discussed merging all ARM reference design mach-* to one dir if I just name that mach-arm then we get a convergence to the vendor name in some organic way. Yours, Linus Walleij _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
On Thu, Dec 6, 2018 at 8:30 AM Linus Walleij <linus.walleij@linaro.org> wrote: > > On Thu, Dec 6, 2018 at 3:05 PM Alexandre Belloni > <alexandre.belloni@bootlin.com> wrote: > > On 06/12/2018 07:58:24-0600, Rob Herring wrote: > > > On Thu, Dec 6, 2018 at 7:39 AM Uwe Kleine-König > > > <u.kleine-koenig@pengutronix.de> wrote: > > > > > > > > Hello, > > > > > > > > On Wed, Dec 05, 2018 at 09:01:59AM -0600, Rob Herring wrote: > > > > > i.MX23 is a Sigmatel chip STMP?? > > > > > > > > I think the Freescale i.MX23 didn't exist at Sigmatel back then. AFAIK > > > > this is a new design using IP from Sigmatel after the aquisition. > > > > > > It is not. I was in the i.MX group which Sigmatel was merged into at > > > the time. Purely marketing rebranding. > > > > > > > Wouldn't it be easier to name the directory to the corresponding mach-* > > entry? > > > > So, imx23 and imx28 would go to mxs/, other imx in imx/. And this also > > solves the Marvell mess with the Synaptics Socs going in berlin/ and the > > other ones in mvebu/. > > I like this idea. Fine by me though I think marvell gets more complicated than that. I'll try a pass at that at least for the cases with a mixture of families. > We discussed merging all ARM reference design mach-* to one dir > if I just name that mach-arm then we get a convergence to the > vendor name in some organic way. TBC, you want .../boot/dts/arm/* for all the ARM, Ltd boards? Just making sure as you were arguing against vendor names. :) Rob _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
On Thu, Dec 6, 2018 at 7:32 AM Andreas Färber <afaerber@suse.de> wrote: > > Am 05.12.18 um 05:17 schrieb Rob Herring: > > On Tue, Dec 4, 2018 at 7:22 PM Andreas Färber <afaerber@suse.de> wrote: > >> > >> Rob, > >> > >> Am 04.12.18 um 19:36 schrieb Rob Herring: > >>> I've put together a script to move the dts files and update the > >>> makefiles. It doesn't handle files not following a common prefix which > >>> isn't many and some includes within the dts files will need some fixups > >>> by hand. > >>> > >>> MAINTAINERS will also need updating. > >>> > >>> A few questions: > >>> > >>> Do we want to move absolutely everything to subdirs? > >> > >> This refactoring is a terrible idea! > > > > How do you really feel? > > > >> While it would've been nice to have more structure from the start, > >> bootloaders like U-Boot expect a flat structure for arm .dtb files now. > >> If you start installing them into subdirs instead, they won't find the > >> files anymore under the hardcoded name. > >> > >> Doing this only for new platforms would be much less invasive and allow > >> to prepare bootloaders accordingly. > > > > That was my suggestion where this started for the new RDA platform. > > Olof preferred to move everything and that's my desire too. > > > >> Alternatively, white-list which ones > >> are safe to move around. > > > > I'd prefer to know which ones the distros don't want moved. That > > should be easier to figure out. We also need that anyways in context > > of what platforms we care about compatibility. > > > > Another option is dtbs_install target could flatten the installed > > dtbs. That is the only part the distros should depend on. > > I'd be okay with distinguishing source vs. install location. Due to the > issue I mention below (and more) we can't use install_dtbs for openSUSE > and had to reimplement it, which we'd need to (and can) adjust. What would be needed for dtbs_install to work? arm64 needs to support a flat install? If it doesn't work for Debian or openSUSE, I'm not sure why we have it. So I'd like to make it work. > >> But don't just script a refactoring because it > >> looks nicer in the source tree, without testing what side effects this > >> can have for board/distro users of the compiled files in practice. > >> We already had that discussion for arm64 because Debian chose to ignore > >> the kernel-installed subdirectories and installed .dtb files into a flat > >> directory, which collided with openSUSE sticking to the kernel choice. > > > > So everyone already deals with subdirs or not with arm and arm64 > > already, seems like they can deal with this. I will raise the topic on > > the cross-distro list though. > > Sounds like you're twisting words... The keyword was "hardcoded" paths - > one way or another (not "and") depending on the kernel choices being > flat for arm, vendor subdir for arm64. > > >> This topic becomes even more important with EBBR: There is neither a > >> mechanism in place to sync .dts files into U-Boot or EDK2 source trees, > >> nor are capsule updates implemented in U-Boot for easily deploying such > >> bootloaders with new .dts sources or paths yet. > > > > EBBR actually says firmware (including dtbs) goes in directories named > > by vendor. > > Fine, but unrelated. If the distros want dtbs in a flat dir and EBBR says otherwise, then it is related. > >> And I can assure you > >> that just getting users to dd the right bootloader can be difficult... > >> Since DT forward and backward compatibility is often being neglected, > >> for example with optional properties or renamed compatibles that break > >> booting with previous drivers, new kernel versions often need updated > >> Device Trees to make use of new/enhanced drivers. Therefore it is > >> unfortunately often enough a necessity to load newer kernel-based .dtb > >> files matching the kernel (as opposed to the dream of kernel-independent > >> hardware descriptions) when working with the latest -rc or -next kernels > >> at least. For examples of DTs needing updates, look no further than > >> Linaro's 96boards - in case of hikey960/EDK2 GRUB is another layer where > >> .dtb paths may be hardcoded, ditto for arm; and Armada was an example > >> where the upstream bindings for the network IP changed incompatibly. > > > > Compatibility is an issue, yes, but that really has nothing to do with this. > > > >> DT overlays are another topic that is not making any progress upstream > >> according to the ELCE BoF, so beyond the Raspberry Pi the only known > >> working way to apply them is to write a U-Boot boot.scr script, which > >> can either reuse $fdtcontroladdr DT or use the filename $fdtfile or > >> hardcode one, the latter two of which would break with your renaming. > > > > DT overlays also have nothing to do with this as there aren't any in > > the kernel. I'm not inclined to take any either with a flat tree. > > We're already at 1800+ files. > > Read again: a) Breaking DT changes and b) the desire to use Overlays > instead of replacing the bootloaders for each change are _reasons_ why > people depend on .dtb filenames from the kernel source tree for their > boot flow today. Nothing to do with downstream .dtbo files. > > For example, remember when I reported that the kernel didn't compile DTs > with -@? No reaction except for Frank asking to be CC'ed - was it ever > fixed??? Do EDK2's or U-Boot's built-in DTs compile with -@ today? IIRC, Frank objected to changing this globally because it will bloat all dtbs. And then no one did the work to make it a per dtb option. Maybe that was the same issue in another thread. > Raspberry Pi overlays in U-Boot work because it switched to passing the > DT through from the proprietary firmware. > Point being that while it would be nice to get a current, compatible DT > via UEFI tables and ignore .dtb filenames outside of a few bootloaders, > in reality we're not quite there yet for all platforms. > > I see no problem (except for naming choices) moving new targets like RDA > to subfolders because we can then hardcode it the new way; I also assume > deeply embedded targets like stm32f4 or targets with no mainline > bootloaders yet like owl-s500 could be refactored. > I do see problems refactoring widely used SBC targets like sunXi though. > > Regards, > Andreas > > -- > SUSE Linux GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany > GF: Felix Imendörffer, Jane Smithard, Graham Norton > HRB 21284 (AG Nürnberg) _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
[-- Attachment #1.1: Type: text/plain, Size: 1279 bytes --] On Thu, Dec 06, 2018 at 01:06:43PM -0600, Rob Herring wrote: > On Thu, Dec 6, 2018 at 7:32 AM Andreas Färber <afaerber@suse.de> wrote: > > I'd be okay with distinguishing source vs. install location. Due to the > > issue I mention below (and more) we can't use install_dtbs for openSUSE > > and had to reimplement it, which we'd need to (and can) adjust. > What would be needed for dtbs_install to work? arm64 needs to support > a flat install? If it doesn't work for Debian or openSUSE, I'm not > sure why we have it. So I'd like to make it work. Correct me if I'm wrong but as far as the flat vs directory thing goes isn't the issue that this winds up being a rename for an existing 32 bit system? If you just install the dtbs in the default location then a bootloader or whatever that is hard coded to look for foo-bar.dtb won't see the new foo/foo-bar.dtb (or whatever) and will continue to use the old binary. It's not the fact that that it's in a directory, it's the fact that the bootloader sees the name it needs to look for change (if it's looking on a filesystem at all). This isn't a problem for arm64 as the location isn't changing, it's used directories from day one. The issues with the existing install_dtbs sounded unrelated to this. [-- Attachment #1.2: signature.asc --] [-- Type: application/pgp-signature, Size: 488 bytes --] [-- Attachment #2: Type: text/plain, Size: 176 bytes --] _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
[-- Attachment #1.1: Type: text/plain, Size: 6547 bytes --] On Thu, Dec 06, 2018 at 01:06:43PM -0600, Rob Herring wrote: > On Thu, Dec 6, 2018 at 7:32 AM Andreas Färber <afaerber@suse.de> wrote: > > > > Am 05.12.18 um 05:17 schrieb Rob Herring: > > > On Tue, Dec 4, 2018 at 7:22 PM Andreas Färber <afaerber@suse.de> wrote: > > >> > > >> Rob, > > >> > > >> Am 04.12.18 um 19:36 schrieb Rob Herring: > > >>> I've put together a script to move the dts files and update the > > >>> makefiles. It doesn't handle files not following a common prefix which > > >>> isn't many and some includes within the dts files will need some fixups > > >>> by hand. > > >>> > > >>> MAINTAINERS will also need updating. > > >>> > > >>> A few questions: > > >>> > > >>> Do we want to move absolutely everything to subdirs? > > >> > > >> This refactoring is a terrible idea! > > > > > > How do you really feel? > > > > > >> While it would've been nice to have more structure from the start, > > >> bootloaders like U-Boot expect a flat structure for arm .dtb files now. > > >> If you start installing them into subdirs instead, they won't find the > > >> files anymore under the hardcoded name. > > >> > > >> Doing this only for new platforms would be much less invasive and allow > > >> to prepare bootloaders accordingly. > > > > > > That was my suggestion where this started for the new RDA platform. > > > Olof preferred to move everything and that's my desire too. > > > > > >> Alternatively, white-list which ones > > >> are safe to move around. > > > > > > I'd prefer to know which ones the distros don't want moved. That > > > should be easier to figure out. We also need that anyways in context > > > of what platforms we care about compatibility. > > > > > > Another option is dtbs_install target could flatten the installed > > > dtbs. That is the only part the distros should depend on. > > > > I'd be okay with distinguishing source vs. install location. Due to the > > issue I mention below (and more) we can't use install_dtbs for openSUSE > > and had to reimplement it, which we'd need to (and can) adjust. > > What would be needed for dtbs_install to work? arm64 needs to support > a flat install? If it doesn't work for Debian or openSUSE, I'm not > sure why we have it. So I'd like to make it work. > > > >> But don't just script a refactoring because it > > >> looks nicer in the source tree, without testing what side effects this > > >> can have for board/distro users of the compiled files in practice. > > >> We already had that discussion for arm64 because Debian chose to ignore > > >> the kernel-installed subdirectories and installed .dtb files into a flat > > >> directory, which collided with openSUSE sticking to the kernel choice. > > > > > > So everyone already deals with subdirs or not with arm and arm64 > > > already, seems like they can deal with this. I will raise the topic on > > > the cross-distro list though. > > > > Sounds like you're twisting words... The keyword was "hardcoded" paths - > > one way or another (not "and") depending on the kernel choices being > > flat for arm, vendor subdir for arm64. > > > > >> This topic becomes even more important with EBBR: There is neither a > > >> mechanism in place to sync .dts files into U-Boot or EDK2 source trees, > > >> nor are capsule updates implemented in U-Boot for easily deploying such > > >> bootloaders with new .dts sources or paths yet. > > > > > > EBBR actually says firmware (including dtbs) goes in directories named > > > by vendor. > > > > Fine, but unrelated. > > If the distros want dtbs in a flat dir and EBBR says otherwise, then > it is related. > > > >> And I can assure you > > >> that just getting users to dd the right bootloader can be difficult... > > >> Since DT forward and backward compatibility is often being neglected, > > >> for example with optional properties or renamed compatibles that break > > >> booting with previous drivers, new kernel versions often need updated > > >> Device Trees to make use of new/enhanced drivers. Therefore it is > > >> unfortunately often enough a necessity to load newer kernel-based .dtb > > >> files matching the kernel (as opposed to the dream of kernel-independent > > >> hardware descriptions) when working with the latest -rc or -next kernels > > >> at least. For examples of DTs needing updates, look no further than > > >> Linaro's 96boards - in case of hikey960/EDK2 GRUB is another layer where > > >> .dtb paths may be hardcoded, ditto for arm; and Armada was an example > > >> where the upstream bindings for the network IP changed incompatibly. > > > > > > Compatibility is an issue, yes, but that really has nothing to do with this. > > > > > >> DT overlays are another topic that is not making any progress upstream > > >> according to the ELCE BoF, so beyond the Raspberry Pi the only known > > >> working way to apply them is to write a U-Boot boot.scr script, which > > >> can either reuse $fdtcontroladdr DT or use the filename $fdtfile or > > >> hardcode one, the latter two of which would break with your renaming. > > > > > > DT overlays also have nothing to do with this as there aren't any in > > > the kernel. I'm not inclined to take any either with a flat tree. > > > We're already at 1800+ files. > > > > Read again: a) Breaking DT changes and b) the desire to use Overlays > > instead of replacing the bootloaders for each change are _reasons_ why > > people depend on .dtb filenames from the kernel source tree for their > > boot flow today. Nothing to do with downstream .dtbo files. > > > > For example, remember when I reported that the kernel didn't compile DTs > > with -@? No reaction except for Frank asking to be CC'ed - was it ever > > fixed??? Do EDK2's or U-Boot's built-in DTs compile with -@ today? > > IIRC, Frank objected to changing this globally because it will bloat > all dtbs. And then no one did the work to make it a per dtb option. > Maybe that was the same issue in another thread. Being the author of one of the patches to pass in -@ so we could have overlays work out of the box, no, there was some other problem with making it only happen for some sub-set of DTBs. AFAIK the current answer is the one of a few years ago that no, if you want symbols for overlays to work you pass in ..whatever it is.. so that -@ is passed in. In the end it felt like there was more concern over the core concept than anything else and I moved along due to other pressing concerns. -- Tom [-- Attachment #1.2: signature.asc --] [-- Type: application/pgp-signature, Size: 819 bytes --] [-- Attachment #2: Type: text/plain, Size: 176 bytes --] _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
On Thu, Dec 6, 2018 at 12:07 PM Mark Brown <broonie@kernel.org> wrote: > > On Thu, Dec 06, 2018 at 01:06:43PM -0600, Rob Herring wrote: > > On Thu, Dec 6, 2018 at 7:32 AM Andreas Färber <afaerber@suse.de> wrote: > > > > I'd be okay with distinguishing source vs. install location. Due to the > > > issue I mention below (and more) we can't use install_dtbs for openSUSE > > > and had to reimplement it, which we'd need to (and can) adjust. > > > What would be needed for dtbs_install to work? arm64 needs to support > > a flat install? If it doesn't work for Debian or openSUSE, I'm not > > sure why we have it. So I'd like to make it work. > > Correct me if I'm wrong but as far as the flat vs directory thing goes > isn't the issue that this winds up being a rename for an existing 32 bit > system? If you just install the dtbs in the default location then a > bootloader or whatever that is hard coded to look for foo-bar.dtb won't > see the new foo/foo-bar.dtb (or whatever) and will continue to use the > old binary. It's not the fact that that it's in a directory, it's the > fact that the bootloader sees the name it needs to look for change (if > it's looking on a filesystem at all). This isn't a problem for arm64 as > the location isn't changing, it's used directories from day one. Yeah, install needs to remain flat even if the dts files move into subdirectories. It will be painful for everybody if the install location moves. > The issues with the existing install_dtbs sounded unrelated to this. Agreed. -Olof _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
On Thu, Dec 6, 2018 at 5:57 PM Rob Herring <robh@kernel.org> wrote: > On Thu, Dec 6, 2018 at 8:30 AM Linus Walleij <linus.walleij@linaro.org> wrote: > > We discussed merging all ARM reference design mach-* to one dir > > if I just name that mach-arm then we get a convergence to the > > vendor name in some organic way. > > TBC, you want .../boot/dts/arm/* for all the ARM, Ltd boards? Just > making sure as you were arguing against vendor names. :) I am not against vendor names, but I am also for SoC names because I think whatever makes most sense should be the rule, so both/and not either/or. One does not exclude the other. It's just a name. In this case what I meant was that while we (me and Arnd) originally discussed merging it all into mach-versatile (how would you know) if I instead merge it all into mach-arm, we get a 1:1 correspondence between mach-dir and vendor name and DTS dir so everyone is happy. Yours, Linus Walleij _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
On Thu, Dec 6, 2018 at 2:07 PM Mark Brown <broonie@kernel.org> wrote: > > On Thu, Dec 06, 2018 at 01:06:43PM -0600, Rob Herring wrote: > > On Thu, Dec 6, 2018 at 7:32 AM Andreas Färber <afaerber@suse.de> wrote: > > > > I'd be okay with distinguishing source vs. install location. Due to the > > > issue I mention below (and more) we can't use install_dtbs for openSUSE > > > and had to reimplement it, which we'd need to (and can) adjust. > > > What would be needed for dtbs_install to work? arm64 needs to support > > a flat install? If it doesn't work for Debian or openSUSE, I'm not > > sure why we have it. So I'd like to make it work. > > Correct me if I'm wrong but as far as the flat vs directory thing goes > isn't the issue that this winds up being a rename for an existing 32 bit > system? If you just install the dtbs in the default location then a > bootloader or whatever that is hard coded to look for foo-bar.dtb won't > see the new foo/foo-bar.dtb (or whatever) and will continue to use the > old binary. It's not the fact that that it's in a directory, it's the > fact that the bootloader sees the name it needs to look for change (if > it's looking on a filesystem at all). Correct. > This isn't a problem for arm64 as > the location isn't changing, it's used directories from day one. The kernel may have used directories, but that's not what the distros did according to Andreas: > We already had that discussion for arm64 because Debian chose to ignore > the kernel-installed subdirectories and installed .dtb files into a flat > directory, which collided with openSUSE sticking to the kernel choice. So are the distros different or who changed to align? That's not clear from this thread. > The issues with the existing install_dtbs sounded unrelated to this. Maybe, what are the issues? We can't change the source layout transparently if dtbs_install is not being used. My question here is whether a flat install is useful on arm64. We can either have a kconfig variable that arm32 sets to do flat installs or it could be some command line make variable and then any user can pick what they want for any arch. Rob _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
[-- Attachment #1.1: Type: text/plain, Size: 395 bytes --] On Fri, Dec 07, 2018 at 08:57:06AM -0600, Rob Herring wrote: > On Thu, Dec 6, 2018 at 2:07 PM Mark Brown <broonie@kernel.org> wrote: > > The issues with the existing install_dtbs sounded unrelated to this. > Maybe, what are the issues? We can't change the source layout > transparently if dtbs_install is not being used. I thought that was the thing with adding -@ so overlays could be used? [-- Attachment #1.2: signature.asc --] [-- Type: application/pgp-signature, Size: 488 bytes --] [-- Attachment #2: Type: text/plain, Size: 176 bytes --] _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
On Fri, Dec 7, 2018 at 9:16 AM Mark Brown <broonie@kernel.org> wrote: > > On Fri, Dec 07, 2018 at 08:57:06AM -0600, Rob Herring wrote: > > On Thu, Dec 6, 2018 at 2:07 PM Mark Brown <broonie@kernel.org> wrote: > > > > The issues with the existing install_dtbs sounded unrelated to this. > > > Maybe, what are the issues? We can't change the source layout > > transparently if dtbs_install is not being used. > > I thought that was the thing with adding -@ so overlays could be used? I don't think so as that is during building, not install. Any user can set '-@' with 'make DTC_FLAGS="-@" ...' already. The issue with that was changing the default globally and no way to set per platform. Now that I think about, moving the sources to subdirs may allow setting DTC_FLAGS per subdir which may be good enough. Rob _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
On Tue, Dec 4, 2018 at 12:36 PM Rob Herring <robh@kernel.org> wrote: > > Olof, Arnd, > > I've put together a script to move the dts files and update the > makefiles. It doesn't handle files not following a common prefix which > isn't many and some includes within the dts files will need some fixups > by hand. > > MAINTAINERS will also need updating. > > A few questions: > > Do we want to move absolutely everything to subdirs? There's quite a > few platforms with only 1-2 platforms. I haven't added these to the > list yet, but can. > > Do any vendors need another level of directories? davinci, omap, nspire, > etc. for TI for example. > > What to do with armv7m.dtsi? I guess it should remain and we just fixup > the include. There may be a few other cross vendor things. > > > Sub-arch maintainers, > 'vendor_map' below is the mapping of file prefix to new subdirectory > (the SoC vendor prefix). Please comment if there are any issues. Here's an updated mapping filled out with the rest of the platforms and using SoC family names in some cases as discussed. The move is completely scripted now including include fixups (though any new includes could break things). So mainly just need to bikeshed the directory mapping. Not sure if marvell should be split up more or not. I split out pxa2xx and pxa3xx, but then there's other pxa chips I think aren't really related. TI is still all one directory except nspire. I was going to split out davinci too, but it's only a couple of files. Sub-arch maintainers need to chime in with what they want. A branch is here including a fix to dtbs_install: git://git.kernel.org/pub/scm/linux/kernel/git/robh/linux.git arm-dts-move vendor_map = { 'alphascale' : 'alphascale', 'alpine' : 'alpine', 'artpec' : 'axis', 'atlas' : 'sirf', 'prima' : 'sirf', 'axm' : 'lsi', 'cx9' : 'cnxt', 'ecx' : 'calxeda', 'highbank' : 'calxeda', 'efm' : 'efm32', 'ep7' : 'cirrus', 'mxs': 'mxs', 'imx23': 'mxs', 'imx28': 'mxs', 'imx': 'imx', 'ls': 'fsl', 'vf': 'fsl', 'qcom': 'qcom', 'am3' : 'ti', 'am4' : 'ti', 'am5' : 'ti', 'dra' : 'ti', 'keystone' : 'ti', 'omap' : 'ti', 'compulab' : 'ti', 'logicpd' : 'ti', 'elpida' : 'ti', 'motorola-cpcap' : 'ti', 'twl' : 'ti', 'da' : 'ti', 'dm' : 'ti', 'nspire' : 'nspire', 'armada' : 'marvell', 'dove' : 'marvell', 'kirkwood' : 'marvell', 'orion' : 'marvell', 'mvebu' : 'marvell', 'mmp2' : 'marvell', 'berlin' : 'berlin', 'pxa2' : 'pxa', 'pxa3' : 'pxa', 'pxa' : 'marvell', 'arm-' : 'arm', 'integ' : 'arm', 'mps' : 'arm', 've' : 'arm', 'aspeed' : 'aspeed', 'at91' : 'at91', 'sama' : 'at91', 'usb_' : 'at91', 'tny_' : 'at91', 'mpa1600' : 'at91', 'animeo_ip' : 'at91', 'aks-cdu' : 'at91', 'ethernut5' : 'at91', 'evk-pro3' : 'at91', 'pm9g45' : 'at91', 'ge86' : 'at91', 'bcm' : 'brcm', 'exynos' : 'exynos', 's3c' : 'samsung', 's5p' : 'samsung', 'gemini' : 'gemini', 'hi3' : 'hisilicon', 'hip' : 'hisilicon', 'hisi' : 'hisilicon', 'mt' : 'mediatek', 'meson' : 'meson', 'moxa' : 'moxa', 'nuvo' : 'nuvoton', 'lpc' : 'lpc', 'owl' : 'actions', 'ox8' : 'oxsemi', 'picox' : 'picoxcell', 'r7' : 'renesas', 'r8' : 'renesas', 'r9' : 'renesas', 'emev2' : 'renesas', 'sh73a' : 'renesas', 'gr-' : 'renesas', 'iwg' : 'renesas', 'rk' : 'rockchip', 'rv11' : 'rockchip', 'socfpga' : 'socfpga', 'stm' : 'stm32', 'sti' : 'sti', 'st-pin' : 'sti', 'ste' : 'st-ericsson', 'spear' : 'spear', 'sun' : 'allwinner', 'axp' : 'allwinner', 'tango' : 'sigma', 'tegra' : 'nvidia', 'uniph' : 'socionext', 'vt8500' : 'vt8500', 'wm8' : 'vt8500', 'xen' : 'xen', 'zx' : 'zte', 'zynq' : 'xilinx', } _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
* Linus Walleij <linus.walleij@linaro.org> [181206 22:12]: > On Thu, Dec 6, 2018 at 5:57 PM Rob Herring <robh@kernel.org> wrote: > > On Thu, Dec 6, 2018 at 8:30 AM Linus Walleij <linus.walleij@linaro.org> wrote: > > > > We discussed merging all ARM reference design mach-* to one dir > > > if I just name that mach-arm then we get a convergence to the > > > vendor name in some organic way. > > > > TBC, you want .../boot/dts/arm/* for all the ARM, Ltd boards? Just > > making sure as you were arguing against vendor names. :) > > I am not against vendor names, but I am also for SoC > names because I think whatever makes most sense should > be the rule, so both/and not either/or. One does not exclude > the other. It's just a name. > > In this case what > I meant was that while we (me and Arnd) originally discussed > merging it all into mach-versatile (how would you know) > if I instead merge it all into mach-arm, we get a 1:1 correspondence > between mach-dir and vendor name and DTS dir so everyone > is happy. With the number of trade names we've already seen with the TI SoCs, I'd probably prefer arch/arm/boot/dts/ti. What used to be omap is now dra7 and am437x and so on. And for timing, doing this just before -rc1 gets tagged seems like a good time to do it. At least I have still pending large dts changes waiting that I'd rather not send a pull request out for until next week :) Regards, Tony _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
On Fri, 7 Dec 2018 at 23:33, Rob Herring <robh@kernel.org> wrote: > > On Tue, Dec 4, 2018 at 12:36 PM Rob Herring <robh@kernel.org> wrote: > > > > Olof, Arnd, > > > > I've put together a script to move the dts files and update the > > makefiles. It doesn't handle files not following a common prefix which > > isn't many and some includes within the dts files will need some fixups > > by hand. > > > > MAINTAINERS will also need updating. > > > > A few questions: > > > > Do we want to move absolutely everything to subdirs? There's quite a > > few platforms with only 1-2 platforms. I haven't added these to the > > list yet, but can. > > > > Do any vendors need another level of directories? davinci, omap, nspire, > > etc. for TI for example. > > > > What to do with armv7m.dtsi? I guess it should remain and we just fixup > > the include. There may be a few other cross vendor things. > > > > > > Sub-arch maintainers, > > 'vendor_map' below is the mapping of file prefix to new subdirectory > > (the SoC vendor prefix). Please comment if there are any issues. > > Here's an updated mapping filled out with the rest of the platforms > and using SoC family names in some cases as discussed. The move is > completely scripted now including include fixups (though any new > includes could break things). So mainly just need to bikeshed the > directory mapping. Not sure if marvell should be split up more or not. > I split out pxa2xx and pxa3xx, but then there's other pxa chips I > think aren't really related. TI is still all one directory except > nspire. I was going to split out davinci too, but it's only a couple > of files. Sub-arch maintainers need to chime in with what they want. > > A branch is here including a fix to dtbs_install: > git://git.kernel.org/pub/scm/linux/kernel/git/robh/linux.git arm-dts-move > > vendor_map = { > 'alphascale' : 'alphascale', > 'alpine' : 'alpine', > 'artpec' : 'axis', > 'atlas' : 'sirf', > 'prima' : 'sirf', > 'axm' : 'lsi', > 'cx9' : 'cnxt', > 'ecx' : 'calxeda', > 'highbank' : 'calxeda', > 'efm' : 'efm32', > 'ep7' : 'cirrus', > 'mxs': 'mxs', > 'imx23': 'mxs', > 'imx28': 'mxs', > 'imx': 'imx', > 'ls': 'fsl', > 'vf': 'fsl', > 'qcom': 'qcom', > 'am3' : 'ti', > 'am4' : 'ti', > 'am5' : 'ti', > 'dra' : 'ti', > 'keystone' : 'ti', > 'omap' : 'ti', > 'compulab' : 'ti', > 'logicpd' : 'ti', > 'elpida' : 'ti', > 'motorola-cpcap' : 'ti', > 'twl' : 'ti', > 'da' : 'ti', > 'dm' : 'ti', > 'nspire' : 'nspire', > 'armada' : 'marvell', > 'dove' : 'marvell', > 'kirkwood' : 'marvell', > 'orion' : 'marvell', > 'mvebu' : 'marvell', > 'mmp2' : 'marvell', > 'berlin' : 'berlin', > 'pxa2' : 'pxa', > 'pxa3' : 'pxa', > 'pxa' : 'marvell', > 'arm-' : 'arm', > 'integ' : 'arm', > 'mps' : 'arm', > 've' : 'arm', > 'aspeed' : 'aspeed', > 'at91' : 'at91', > 'sama' : 'at91', > 'usb_' : 'at91', > 'tny_' : 'at91', > 'mpa1600' : 'at91', > 'animeo_ip' : 'at91', > 'aks-cdu' : 'at91', > 'ethernut5' : 'at91', > 'evk-pro3' : 'at91', > 'pm9g45' : 'at91', > 'ge86' : 'at91', > 'bcm' : 'brcm', > 'exynos' : 'exynos', > 's3c' : 'samsung', > 's5p' : 'samsung', Since Exynos is Samsung, I would prefer consistency here so let's put everything under vendor - samsung. I understand that It will be different than arm64 but that's the problem of choosing non-vendor name for arm64 at first place. So let's go with: 'exynos' : 'samsung', 's3c' : 'samsung', 's5p' : 'samsung', Best regards, Krzysztof > 'gemini' : 'gemini', > 'hi3' : 'hisilicon', > 'hip' : 'hisilicon', > 'hisi' : 'hisilicon', > 'mt' : 'mediatek', > 'meson' : 'meson', > 'moxa' : 'moxa', > 'nuvo' : 'nuvoton', > 'lpc' : 'lpc', > 'owl' : 'actions', > 'ox8' : 'oxsemi', > 'picox' : 'picoxcell', > 'r7' : 'renesas', > 'r8' : 'renesas', > 'r9' : 'renesas', > 'emev2' : 'renesas', > 'sh73a' : 'renesas', > 'gr-' : 'renesas', > 'iwg' : 'renesas', > 'rk' : 'rockchip', > 'rv11' : 'rockchip', > 'socfpga' : 'socfpga', > 'stm' : 'stm32', > 'sti' : 'sti', > 'st-pin' : 'sti', > 'ste' : 'st-ericsson', > 'spear' : 'spear', > 'sun' : 'allwinner', > 'axp' : 'allwinner', > 'tango' : 'sigma', > 'tegra' : 'nvidia', > 'uniph' : 'socionext', > 'vt8500' : 'vt8500', > 'wm8' : 'vt8500', > 'xen' : 'xen', > 'zx' : 'zte', > 'zynq' : 'xilinx', > } _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
On Tue, 2018-12-04 at 12:36 -0600, Rob Herring wrote: > I've put together a script to move the dts files and update the > makefiles. Anything here which the split-devicetree tree[0] needs to care about? I think this is all just about file movement contained entirely underneath arch/arm/boot/dts, in which case the answer *should* be "no". Ian. [0] https://git.kernel.org/pub/scm/linux/kernel/git/devicetree/devicetree-rebasing.git/ _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
On Fri, Dec 7, 2018 at 11:33 PM Rob Herring <robh@kernel.org> wrote: > Here's an updated mapping filled out with the rest of the platforms > and using SoC family names in some cases as discussed. This looks really good to me! Acked-by: Linus Walleij <linus.walleij@linaro.org> >The move is > completely scripted now including include fixups (though any new > includes could break things). So mainly just need to bikeshed the > directory mapping. I did have a serious minute to think over whether this was a bikeshedding type discussion and the outcome is as unimportant as the color of some bikeshed. Went as far as to re-read Parsons (hilarious!) original essay and the first chapter of Wittgensteins Tractatus. I concluded it was more profound than that for several reasons. Yours, Linus Walleij _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
On Fri, Dec 07, 2018 at 04:33:41PM -0600, Rob Herring wrote: > On Tue, Dec 4, 2018 at 12:36 PM Rob Herring <robh@kernel.org> wrote: > > > > Olof, Arnd, > > > > I've put together a script to move the dts files and update the > > makefiles. It doesn't handle files not following a common prefix which > > isn't many and some includes within the dts files will need some fixups > > by hand. > > > > MAINTAINERS will also need updating. > > > > A few questions: > > > > Do we want to move absolutely everything to subdirs? There's quite a > > few platforms with only 1-2 platforms. I haven't added these to the > > list yet, but can. > > > > Do any vendors need another level of directories? davinci, omap, nspire, > > etc. for TI for example. > > > > What to do with armv7m.dtsi? I guess it should remain and we just fixup > > the include. There may be a few other cross vendor things. > > > > > > Sub-arch maintainers, > > 'vendor_map' below is the mapping of file prefix to new subdirectory > > (the SoC vendor prefix). Please comment if there are any issues. > > Here's an updated mapping filled out with the rest of the platforms > and using SoC family names in some cases as discussed. The move is > completely scripted now including include fixups (though any new > includes could break things). So mainly just need to bikeshed the > directory mapping. Not sure if marvell should be split up more or not. > I split out pxa2xx and pxa3xx, but then there's other pxa chips I > think aren't really related. TI is still all one directory except > nspire. I was going to split out davinci too, but it's only a couple > of files. Sub-arch maintainers need to chime in with what they want. I think we can debate this forever if we want to, and of course when product families change owners (or when companies merge or change names) it will be tempting to rename or move files. But I think we'll treat this the same way as we do with bindings and other aspects of the kernel: some platforms will have legacy names that might not describe their current state but documents the lineage of the product. For that, it might be slightly easier to use product family names instead of company ditto, but I really don't care enough to request changes. Also, since the subdirectory doesn't influence install path on 32-bit, renames shouldn't be the end of the world (but I'm likely to turn down those requests beyond the possible initial "we miscategorized this platform" kind). -Olof _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel