[PATCH 0/3] x86 disk image and modules initramfs generation

[PATCH 0/3] x86 disk image and modules initramfs generation

[H. Peter Anvin]

From: "H. Peter Anvin" (Intel) <hpa@zytor.com>

When compiling on a different machine than the runtime target,
including but not limited to simulators, it is rather handy to be able
to produce a bootable image. The scripts for that in x86 are
relatively old, and assume a BIOS system.

This adds a build target to generate a hdimage which can be booted
either from BIOS or EFI, and modernizes the genimage.sh script
including adding the ability to add an arbitrary number of initramfs
files (limited only by the length of the command line.)

Possibly more controversial, at least from a Kbuild design perspective
(as usual I'm the guy who wants to do something with Kbuild which it
seems it was never really designed to do), is add the ability to
create an initramfs image which includes all the built modules. Some
distributions cannot be easily booted without modules in initramfs,
and this creates an image which can be added to initramfs to provide
the kernel modules, as finalized by "make modules_install".

The final patch put these two together, and allows the modules
initramfs to be included in the x86 boot image.

---

Fixes in v2:

1/3: correct command line generation for multiple initrds
2/3: no change
3/3: append, don't overwrite FDINITRD when adding modules.img

 Makefile                     |  17 ++-
 arch/x86/Makefile            |   8 +-
 arch/x86/boot/.gitignore     |   1 +
 arch/x86/boot/Makefile       |  62 +++++----
 arch/x86/boot/genimage.sh    | 294 +++++++++++++++++++++++++++++++------------
 arch/x86/boot/mtools.conf.in |   3 +
 usr/.gitignore               |   3 +
 usr/Kconfig                  |  31 ++---
 usr/Makefile                 |  39 +++++-
 9 files changed, 331 insertions(+), 127 deletions(-)

[PATCH v2 1/3] x86/boot: Modernize genimage script; hdimage support; remove bzlilo

[H. Peter Anvin]

From: "H. Peter Anvin (Intel)" <hpa@zytor.com>

The image generation scripts in arch/x86/boot are pretty out of date,
except for the isoimage target. Update and clean up the
genimage.sh script, and make it support an arbitrary number of
initramfs files in the image.

Add a "hdimage" target, which can be booted by either BIOS or
EFI (if the kernel is compiled with the EFI stub.) For EFI to be able
to pass the command line to the kernel, we need the EFI shell, but the
firmware builtin EFI shell, if it even exists, is pretty much always
the last resort boot option, so search for OVMF or EDK2 and explicitly
include a copy of the EFI shell.

To make this all work, use bash features in the script.  Furthermore,
this version of the script makes use of some mtools features,
especially mpartition, that might not exist in very old version of
mtools, but given all the other dependencies on this script this
doesn't seem such a big deal.

Signed-off-by: H. Peter Anvin (Intel) <hpa@zytor.com>
---
 arch/x86/Makefile            |   5 +-
 arch/x86/boot/.gitignore     |   1 +
 arch/x86/boot/Makefile       |  44 +++---
 arch/x86/boot/genimage.sh    | 294 ++++++++++++++++++++++++++---------
 arch/x86/boot/mtools.conf.in |   3 +
 5 files changed, 247 insertions(+), 100 deletions(-)

diff --git a/arch/x86/Makefile b/arch/x86/Makefile
index 9a85eae37b17..943f26a32834 100644
--- a/arch/x86/Makefile
+++ b/arch/x86/Makefile
@@ -247,7 +247,7 @@ drivers-$(CONFIG_FB) += arch/x86/video/
 
 boot := arch/x86/boot
 
-BOOT_TARGETS = bzdisk fdimage fdimage144 fdimage288 isoimage
+BOOT_TARGETS = bzdisk fdimage fdimage144 fdimage288 hdimage isoimage
 
 PHONY += bzImage $(BOOT_TARGETS)
 
@@ -305,8 +305,9 @@ define archhelp
   echo  '  fdimage		- Create 1.4MB boot floppy image (arch/x86/boot/fdimage)'
   echo  '  fdimage144		- Create 1.4MB boot floppy image (arch/x86/boot/fdimage)'
   echo  '  fdimage288		- Create 2.8MB boot floppy image (arch/x86/boot/fdimage)'
+  echo  '  hdimage		- Create a BIOS/EFI hard disk image (arch/x86/boot/hdimage)'
   echo  '  isoimage		- Create a boot CD-ROM image (arch/x86/boot/image.iso)'
-  echo  '			  bzdisk/fdimage*/isoimage also accept:'
+  echo  '			  bzdisk/fdimage*/hdimage/isoimage also accept:'
   echo  '			  FDARGS="..."  arguments for the booted kernel'
   echo  '                  	  FDINITRD=file initrd for the booted kernel'
   echo  ''
diff --git a/arch/x86/boot/.gitignore b/arch/x86/boot/.gitignore
index 9cc7f1357b9b..1189be057ebd 100644
--- a/arch/x86/boot/.gitignore
+++ b/arch/x86/boot/.gitignore
@@ -11,3 +11,4 @@ setup.elf
 fdimage
 mtools.conf
 image.iso
+hdimage
diff --git a/arch/x86/boot/Makefile b/arch/x86/boot/Makefile
index fe605205b4ce..dfbc26a8e924 100644
--- a/arch/x86/boot/Makefile
+++ b/arch/x86/boot/Makefile
@@ -29,7 +29,7 @@ KCOV_INSTRUMENT		:= n
 SVGA_MODE	:= -DSVGA_MODE=NORMAL_VGA
 
 targets		:= vmlinux.bin setup.bin setup.elf bzImage
-targets		+= fdimage fdimage144 fdimage288 image.iso mtools.conf
+targets		+= fdimage fdimage144 fdimage288 image.iso hdimage
 subdir-		:= compressed
 
 setup-y		+= a20.o bioscall.o cmdline.o copy.o cpu.o cpuflags.o cpucheck.o
@@ -115,47 +115,49 @@ $(obj)/compressed/vmlinux: FORCE
 	$(Q)$(MAKE) $(build)=$(obj)/compressed $@
 
 # Set this if you want to pass append arguments to the
-# bzdisk/fdimage/isoimage kernel
+# bzdisk/fdimage/hdimage/isoimage kernel
 FDARGS =
-# Set this if you want an initrd included with the
-# bzdisk/fdimage/isoimage kernel
+# Set this if you want one or more initrds included in the image
 FDINITRD =
 
-image_cmdline = default linux $(FDARGS) $(if $(FDINITRD),initrd=initrd.img,)
+imgdeps = $(obj)/bzImage $(obj)/mtools.conf $(src)/genimage.sh
 
 $(obj)/mtools.conf: $(src)/mtools.conf.in
 	sed -e 's|@OBJ@|$(obj)|g' < $< > $@
 
+targets += mtools.conf
+
+# genimage.sh requires bash, but it also has a bunch of other
+# external dependencies.
 quiet_cmd_genimage = GENIMAGE $3
-cmd_genimage = sh $(srctree)/$(src)/genimage.sh $2 $3 $(obj)/bzImage \
-			$(obj)/mtools.conf '$(image_cmdline)' $(FDINITRD)
+cmd_genimage = $(BASH) $(srctree)/$(src)/genimage.sh $2 $3 $(obj)/bzImage \
+		$(obj)/mtools.conf '$(FDARGS)' $(FDINITRD)
 
-PHONY += bzdisk fdimage fdimage144 fdimage288 isoimage bzlilo install
+PHONY += bzdisk fdimage fdimage144 fdimage288 hdimage isoimage install
 
 # This requires write access to /dev/fd0
-bzdisk: $(obj)/bzImage $(obj)/mtools.conf
+# All images require syslinux to be installed; hdimage also requires
+# EDK2/OVMF if the kernel is compiled with the EFI stub.
+bzdisk: $(imgdeps)
 	$(call cmd,genimage,bzdisk,/dev/fd0)
 
-# These require being root or having syslinux 2.02 or higher installed
-fdimage fdimage144: $(obj)/bzImage $(obj)/mtools.conf
+fdimage fdimage144: $(imgdeps)
 	$(call cmd,genimage,fdimage144,$(obj)/fdimage)
 	@$(kecho) 'Kernel: $(obj)/fdimage is ready'
 
-fdimage288: $(obj)/bzImage $(obj)/mtools.conf
+fdimage288: $(imgdeps)
 	$(call cmd,genimage,fdimage288,$(obj)/fdimage)
 	@$(kecho) 'Kernel: $(obj)/fdimage is ready'
 
-isoimage: $(obj)/bzImage
+hdimage: $(imgdeps)
+	$(call cmd,genimage,hdimage,$(obj)/hdimage)
+	@$(kecho) 'Kernel: $(obj)/hdimage is ready'
+
+isoimage: $(imgdeps)
 	$(call cmd,genimage,isoimage,$(obj)/image.iso)
 	@$(kecho) 'Kernel: $(obj)/image.iso is ready'
 
-bzlilo:
-	if [ -f $(INSTALL_PATH)/vmlinuz ]; then mv $(INSTALL_PATH)/vmlinuz $(INSTALL_PATH)/vmlinuz.old; fi
-	if [ -f $(INSTALL_PATH)/System.map ]; then mv $(INSTALL_PATH)/System.map $(INSTALL_PATH)/System.old; fi
-	cat $(obj)/bzImage > $(INSTALL_PATH)/vmlinuz
-	cp System.map $(INSTALL_PATH)/
-	if [ -x /sbin/lilo ]; then /sbin/lilo; else /etc/lilo/install; fi
-
 install:
-	sh $(srctree)/$(src)/install.sh $(KERNELRELEASE) $(obj)/bzImage \
+	$(CONFIG_SHELL) $(srctree)/$(src)/install.sh \
+		$(KERNELRELEASE) $(obj)/bzImage \
 		System.map "$(INSTALL_PATH)"
diff --git a/arch/x86/boot/genimage.sh b/arch/x86/boot/genimage.sh
index 6a10d52a4145..4b554049ec3c 100644
--- a/arch/x86/boot/genimage.sh
+++ b/arch/x86/boot/genimage.sh
@@ -1,4 +1,4 @@
-#!/bin/sh
+#!/bin/bash
 #
 # This file is subject to the terms and conditions of the GNU General Public
 # License.  See the file "COPYING" in the main directory of this archive
@@ -8,15 +8,24 @@
 #
 # Adapted from code in arch/x86/boot/Makefile by H. Peter Anvin and others
 #
-# "make fdimage/fdimage144/fdimage288/isoimage" script for x86 architecture
+# "make fdimage/fdimage144/fdimage288/hdimage/isoimage"
+# script for x86 architecture
 #
 # Arguments:
-#   $1 - fdimage format
-#   $2 - target image file
-#   $3 - kernel bzImage file
-#   $4 - mtool configuration file
-#   $5 - kernel cmdline
-#   $6 - inird image file
+#   $1  - fdimage format
+#   $2  - target image file
+#   $3  - kernel bzImage file
+#   $4  - mtools configuration file
+#   $5  - kernel cmdline
+#   $6+ - initrd image file(s)
+#
+# This script requires:
+#   bash
+#   syslinux
+#   mtools (for fdimage* and hdimage)
+#   edk2/OVMF (for hdimage)
+#
+# Otherwise try to stick to POSIX shell commands...
 #
 
 # Use "make V=1" to debug this script
@@ -26,105 +35,236 @@ case "${KBUILD_VERBOSE}" in
         ;;
 esac
 
-verify () {
-	if [ ! -f "$1" ]; then
-		echo ""                                                   1>&2
-		echo " *** Missing file: $1"                              1>&2
-		echo ""                                                   1>&2
-		exit 1
+# Exit the top-level shell with an error
+topshell=$$
+trap 'exit 1' USR1
+die() {
+	echo ""        1>&2
+	echo " *** $*" 1>&2
+	echo ""        1>&2
+	kill -USR1 $topshell
+}
+
+# Verify the existence and readability of a file
+verify() {
+	if [ ! -f "$1" -o ! -r "$1" ]; then
+		die "Missing file: $1"
 	fi
 }
 
+diskfmt="$1"
+FIMAGE="$2"
+FBZIMAGE="$3"
+MTOOLSRC="$4"
+KCMDLINE="$5"
+shift 5				# Remaining arguments = initrd files
+
+export MTOOLSRC
 
-export MTOOLSRC=$4
-FIMAGE=$2
-FBZIMAGE=$3
-KCMDLINE=$5
-FDINITRD=$6
+# common options for dd
+dd='dd iflag=fullblock'
 
 # Make sure the files actually exist
 verify "$FBZIMAGE"
 
+declare -a FDINITRDS
+irdpfx=' initrd='
+initrdopts_syslinux=''
+initrdopts_efi=''
+for f in "$@"; do
+	if [ -n "$f" ]; then
+	    FDINITRDS=("${FDINITRDS[@]}" "$f")
+	    fname="$(basename "$f")"
+	    initrdopts_syslinux="${initrdopts_syslinux}${irdpfx}${fname}"
+	    irdpfx=,
+	    initrdopts_efi="${initrdopts_efi} initrd=${fname}"
+	fi
+done
+
+# Read a $3-byte littleendian unsigned value at offset $2 from file $1
+le() {
+	local n=0
+	local m=1
+	for b in $(od -A n -v -j $2 -N $3 -t u1 "$1"); do
+		n=$((n + b*m))
+		m=$((m * 256))
+	done
+	echo $n
+}
+
+# Get the EFI architecture name such that boot{name}.efi is the default
+# boot file name. Returns false with no output if the file is not an
+# EFI image or otherwise unknown.
+efiarch() {
+	[ -f "$1" ] || return
+	[ $(le "$1" 0 2) -eq 23117 ] || return		# MZ magic
+	peoffs=$(le "$1" 60 4)				# PE header offset
+	[ $peoffs -ge 64 ] || return
+	[ $(le "$1" $peoffs 4) -eq 17744 ] || return	# PE magic
+	case $(le "$1" $((peoffs+4+20)) 2) in		# PE type
+		267)	;;				# PE32
+		523)	;;				# PE32+
+		*) return 1 ;;				# Invalid
+	esac
+	[ $(le "$1" $((peoffs+4+20+68)) 2) -eq 10 ] || return # EFI app
+	case $(le "$1" $((peoffs+4)) 2) in		# Machine type
+		 332)	echo i386	;;
+		 450)	echo arm	;;
+		 512)	echo ia64	;;
+		20530)	echo riscv32	;;
+		20580)	echo riscv64	;;
+		20776)	echo riscv128	;;
+		34404)	echo x64	;;
+		43620)	echo aa64	;;
+	esac
+}
+
+# Get the combined sizes in bytes of the files given, counting sparse
+# files as full length, and padding each file to a 4K block size
+filesizes() {
+	local t=0
+	local s
+	for s in $(ls -lnL "$@" 2>/dev/null | awk '/^-/{ print $5; }'); do
+		t=$((t + ((s+4095)/4096)*4096))
+	done
+	echo $t
+}
+
+# Expand directory names which should be in /usr/share into a list
+# of possible alternatives
+sharedirs() {
+	local dir file
+	for dir in /usr/share /usr/lib64 /usr/lib; do
+		for file; do
+			echo "$dir/$file"
+			echo "$dir/${file^^}"
+		done
+	done
+}
+efidirs() {
+	local dir file
+	for dir in /usr/share /boot /usr/lib64 /usr/lib; do
+		for file; do
+			echo "$dir/$file"
+			echo "$dir/${file^^}"
+		done
+	done
+}
+
+findsyslinux() {
+	local f="$(find -L $(sharedirs syslinux isolinux) \
+		    -name "$1" -readable -type f -print -quit 2>/dev/null)"
+	if [ ! -f "$f" ]; then
+		die "Need a $1 file, please install syslinux/isolinux."
+	fi
+	echo "$f"
+	return 0
+}
+
+findovmf() {
+	local arch="$1"
+	shift
+	local -a names=(-false)
+	local name f
+	for name; do
+		names=("${names[@]}" -or -iname "$name")
+	done
+	for f in $(find -L $(efidirs edk2 ovmf) \
+			\( "${names[@]}" \) -readable -type f \
+			-print 2>/dev/null); do
+		if [ "$(efiarch "$f")" = "$arch" ]; then
+			echo "$f"
+			return 0
+		fi
+	done
+	die "Need a $1 file for $arch, please install EDK2/OVMF."
+}
+
+do_mcopy() {
+	if [ ${#FDINITRDS[@]} -gt 0 ]; then
+		mcopy "${FDINITRDS[@]}" "$1"
+	fi
+	if [ -n "$efishell" ]; then
+		mmd "$1"EFI "$1"EFI/Boot
+		mcopy "$efishell" "$1"EFI/Boot/boot${kefiarch}.efi
+	fi
+	if [ -n "$kefiarch" ]; then
+		echo linux "$KCMDLINE$initrdopts_efi" | \
+			mcopy - "$1"startup.nsh
+	fi
+	echo default linux "$KCMDLINE$initrdopts_syslinux" | \
+		mcopy - "$1"syslinux.cfg
+	mcopy "$FBZIMAGE" "$1"linux
+}
+
 genbzdisk() {
 	verify "$MTOOLSRC"
 	mformat a:
-	syslinux $FIMAGE
-	echo "$KCMDLINE" | mcopy - a:syslinux.cfg
-	if [ -f "$FDINITRD" ] ; then
-		mcopy "$FDINITRD" a:initrd.img
-	fi
-	mcopy $FBZIMAGE a:linux
+	syslinux "$FIMAGE"
+	do_mcopy a:
 }
 
 genfdimage144() {
 	verify "$MTOOLSRC"
-	dd if=/dev/zero of=$FIMAGE bs=1024 count=1440 2> /dev/null
+	$dd if=/dev/zero of="$FIMAGE" bs=1024 count=1440 2>/dev/null
 	mformat v:
-	syslinux $FIMAGE
-	echo "$KCMDLINE" | mcopy - v:syslinux.cfg
-	if [ -f "$FDINITRD" ] ; then
-		mcopy "$FDINITRD" v:initrd.img
-	fi
-	mcopy $FBZIMAGE v:linux
+	syslinux "$FIMAGE"
+	do_mcopy v:
 }
 
 genfdimage288() {
 	verify "$MTOOLSRC"
-	dd if=/dev/zero of=$FIMAGE bs=1024 count=2880 2> /dev/null
+	$dd if=/dev/zero of="$FIMAGE" bs=1024 count=2880 2>/dev/null
 	mformat w:
-	syslinux $FIMAGE
-	echo "$KCMDLINE" | mcopy - W:syslinux.cfg
-	if [ -f "$FDINITRD" ] ; then
-		mcopy "$FDINITRD" w:initrd.img
+	syslinux "$FIMAGE"
+	do_mcopy w:
+}
+
+genhdimage() {
+	verify "$MTOOLSRC"
+	mbr="$(findsyslinux mbr.bin)"
+	kefiarch="$(efiarch "$FBZIMAGE")"
+	if [ -n "$kefiarch" ]; then
+		# The efishell provides command line handling
+		efishell="$(findovmf $kefiarch shell.efi shell${kefiarch}.efi)"
+		ptype='-T 0xef'	# EFI system partition, no GPT
 	fi
-	mcopy $FBZIMAGE w:linux
+	sizes=$(filesizes "$FBZIMAGE" "${FDINITRDS[@]}" "$efishell")
+	# Allow 1% + 1 MiB for filesystem and partition table overhead,
+	# syslinux, and config files
+	megs=$(((sizes + sizes/100 + 2*1024*1024 - 1)/(1024*1024)))
+	$dd if=/dev/zero of="$FIMAGE" bs=$((1024*1024)) count=$megs 2>/dev/null
+	mpartition -I -c -s 32 -h 64 -t $megs $ptype -b 512 -a h:
+	$dd if="$mbr" of="$FIMAGE" bs=440 count=1 conv=notrunc 2>/dev/null
+	mformat -s 32 -h 64 -t $megs h:
+	syslinux --offset $((512*512)) "$FIMAGE"
+	do_mcopy h:
 }
 
 geniso() {
-	tmp_dir=`dirname $FIMAGE`/isoimage
-	rm -rf $tmp_dir
-	mkdir $tmp_dir
-	for i in lib lib64 share ; do
-		for j in syslinux ISOLINUX ; do
-			if [ -f /usr/$i/$j/isolinux.bin ] ; then
-				isolinux=/usr/$i/$j/isolinux.bin
-			fi
-		done
-		for j in syslinux syslinux/modules/bios ; do
-			if [ -f /usr/$i/$j/ldlinux.c32 ]; then
-				ldlinux=/usr/$i/$j/ldlinux.c32
-			fi
-		done
-		if [ -n "$isolinux" -a -n "$ldlinux" ] ; then
-			break
-		fi
-	done
-	if [ -z "$isolinux" ] ; then
-		echo 'Need an isolinux.bin file, please install syslinux/isolinux.'
-		exit 1
-	fi
-	if [ -z "$ldlinux" ] ; then
-		echo 'Need an ldlinux.c32 file, please install syslinux/isolinux.'
-		exit 1
-	fi
-	cp $isolinux $tmp_dir
-	cp $ldlinux $tmp_dir
-	cp $FBZIMAGE $tmp_dir/linux
-	echo "$KCMDLINE" > $tmp_dir/isolinux.cfg
-	if [ -f "$FDINITRD" ] ; then
-		cp "$FDINITRD" $tmp_dir/initrd.img
-	fi
-	genisoimage -J -r -input-charset=utf-8 -quiet -o $FIMAGE \
+	tmp_dir="$(dirname "$FIMAGE")/isoimage"
+	rm -rf "$tmp_dir"
+	mkdir "$tmp_dir"
+	isolinux=$(findsyslinux isolinux.bin)
+	ldlinux=$(findsyslinux  ldlinux.c32)
+	cp "$isolinux" "$ldlinux" "$tmp_dir"
+	cp "$FBZIMAGE" "$tmp_dir"/linux
+	echo default linux "$KCMDLINE" > "$tmp_dir"/isolinux.cfg
+	cp "${FDINITRDS[@]}" "$tmp_dir"/
+	genisoimage -J -r -input-charset=utf-8 -quiet -o "$FIMAGE" \
 		-b isolinux.bin -c boot.cat -no-emul-boot -boot-load-size 4 \
-		-boot-info-table $tmp_dir
-	isohybrid $FIMAGE 2>/dev/null || true
-	rm -rf $tmp_dir
+		-boot-info-table "$tmp_dir"
+	isohybrid "$FIMAGE" 2>/dev/null || true
+	rm -rf "$tmp_dir"
 }
 
-case $1 in
+rm -f "$FIMAGE"
+
+case "$diskfmt" in
 	bzdisk)     genbzdisk;;
 	fdimage144) genfdimage144;;
 	fdimage288) genfdimage288;;
+	hdimage)    genhdimage;;
 	isoimage)   geniso;;
-	*)          echo 'Unknown image format'; exit 1;
+	*)          die "Unknown image format: $diskfmt";;
 esac
diff --git a/arch/x86/boot/mtools.conf.in b/arch/x86/boot/mtools.conf.in
index efd6d2490c1d..9e2662d01364 100644
--- a/arch/x86/boot/mtools.conf.in
+++ b/arch/x86/boot/mtools.conf.in
@@ -14,4 +14,7 @@ drive v:
 drive w:
   file="@OBJ@/fdimage" cylinders=80 heads=2 sectors=36 filter
 
+# Hard disk
+drive h:
+  file="@OBJ@/hdimage" partition=1 mformat_only
 
-- 
2.30.2

[PATCH v2 2/3] usr, modules: Add build target for creating a modules initramfs image

[H. Peter Anvin]

From: "H. Peter Anvin (Intel)" <hpa@zytor.com>

Some distributions really cannot be booted without modules
anymore. To allow an externally built kernel to be run, it is handy to
be able to create an initramfs image with all the modules, that can
appended to an existing initramfs image (preferrably without any
modules, but not necessarily, since the /lib/modules path includes the
kernel version.)

This image is generated in usr/modules.img.

Since this image may end up being rather large, change the default for
CONFIG_INITRAMFS_COMPRESSION_*. We already have a special case in the
Makefile to not re-compress an existing initramfs image; add the very
small default builtin initramfs as another case to suppress the
compression. To avoid losing the default compression setting, add a
separate variable for the possibly overridden builtin initramfs
compression method.

Signed-off-by: H. Peter Anvin (Intel) <hpa@zytor.com>
---
 Makefile       | 17 ++++++++++++-----
 usr/.gitignore |  3 +++
 usr/Kconfig    | 31 +++++++++++++++++--------------
 usr/Makefile   | 39 +++++++++++++++++++++++++++++++++++----
 4 files changed, 67 insertions(+), 23 deletions(-)

diff --git a/Makefile b/Makefile
index 4730cf156f6b..89e40760384d 100644
--- a/Makefile
+++ b/Makefile
@@ -622,7 +622,7 @@ KBUILD_MODULES :=
 KBUILD_BUILTIN := 1
 
 # If we have only "make modules", don't compile built-in objects.
-ifeq ($(MAKECMDGOALS),modules)
+ifeq (modules,$(MAKECMDGOALS))
   KBUILD_BUILTIN :=
 endif
 
@@ -630,7 +630,8 @@ endif
 # in addition to whatever we do anyway.
 # Just "make" or "make all" shall build modules as well
 
-ifneq ($(filter all modules nsdeps %compile_commands.json clang-%,$(MAKECMDGOALS)),)
+ifneq ($(filter all modules nsdeps %compile_commands.json clang-% \
+	modules.img usr/modules.img,$(MAKECMDGOALS)),)
   KBUILD_MODULES := 1
 endif
 
@@ -1461,15 +1462,17 @@ modules_prepare: prepare
 PHONY += modules_install
 modules_install: _modinst_ _modinst_post
 
+INSTALL_MOD_LN ?= ln -s
+
 PHONY += _modinst_
 _modinst_:
 	@rm -rf $(MODLIB)/kernel
 	@rm -f $(MODLIB)/source
 	@mkdir -p $(MODLIB)/kernel
-	@ln -s $(abspath $(srctree)) $(MODLIB)/source
+	@$(INSTALL_MOD_LN) $(abspath $(srctree)) $(MODLIB)/source
 	@if [ ! $(objtree) -ef  $(MODLIB)/build ]; then \
-		rm -f $(MODLIB)/build ; \
-		ln -s $(CURDIR) $(MODLIB)/build ; \
+		rm -rf $(MODLIB)/build ; \
+		$(INSTALL_MOD_LN) $(CURDIR) $(MODLIB)/build ; \
 	fi
 	@sed 's:^:kernel/:' modules.order > $(MODLIB)/modules.order
 	@cp -f modules.builtin $(MODLIB)/
@@ -1489,6 +1492,10 @@ modules_sign:
 	$(Q)$(MAKE) -f $(srctree)/scripts/Makefile.modsign
 endif
 
+# initramfs image containing all modules
+modules.img usr/modules.img: modules
+	$(Q)$(MAKE) $(build)=usr usr/modules.img
+
 else # CONFIG_MODULES
 
 # Modules not configured
diff --git a/usr/.gitignore b/usr/.gitignore
index 935442ed1eb2..14e2fae88d8a 100644
--- a/usr/.gitignore
+++ b/usr/.gitignore
@@ -2,3 +2,6 @@
 gen_init_cpio
 initramfs_data.cpio
 /initramfs_inc_data
+/modules
+modules.cpio
+modules.img
diff --git a/usr/Kconfig b/usr/Kconfig
index 8bbcf699fe3b..625372682b5e 100644
--- a/usr/Kconfig
+++ b/usr/Kconfig
@@ -109,18 +109,23 @@ config RD_ZSTD
 	  If unsure, say N.
 
 choice
-	prompt "Built-in initramfs compression mode"
-	depends on INITRAMFS_SOURCE != ""
-	help
-	  This option allows you to decide by which algorithm the builtin
-	  initramfs will be compressed.  Several compression algorithms are
-	  available, which differ in efficiency, compression and
-	  decompression speed.  Compression speed is only relevant
-	  when building a kernel.  Decompression speed is relevant at
-	  each boot. Also the memory usage during decompression may become
-	  relevant on memory constrained systems. This is usually based on the
-	  dictionary size of the algorithm with algorithms like XZ and LZMA
-	  featuring large dictionary sizes.
+	prompt "Preferred initramfs compression mode"
+	default INITRAMFS_COMPRESSION_XZ   if RD_XZ
+	default INITRAMFS_COMPRESSION_GZIP if RD_GZIP
+	default INITRAMFS_COMPRESSION_LZO  if RD_LZO
+	default INITRAMFS_COMPRESSION_LZ4  if RD_LZ4
+	default INITRAMFS_COMPRESSION_NONE
+	help
+	  This option allows you to decide by which algorithm the
+	  builtin initramfs and usr/modules.img will be compressed.
+	  Several compression algorithms are available, which differ
+	  in efficiency, compression and decompression speed.
+	  Compression speed is only relevant when building a kernel.
+	  Decompression speed is relevant at each boot. Also the
+	  memory usage during decompression may become relevant on
+	  memory constrained systems. This is usually based on the
+	  dictionary size of the algorithm with algorithms like XZ and
+	  LZMA featuring large dictionary sizes.
 
 	  High compression options are mostly useful for users who are
 	  low on RAM, since it reduces the memory consumption during
@@ -130,8 +135,6 @@ choice
 	  compression tool to compress the generated initram cpio file for
 	  embedding.
 
-	  If in doubt, select 'None'
-
 config INITRAMFS_COMPRESSION_GZIP
 	bool "Gzip"
 	depends on RD_GZIP
diff --git a/usr/Makefile b/usr/Makefile
index b1a81a40eab1..6257ba8690a3 100644
--- a/usr/Makefile
+++ b/usr/Makefile
@@ -22,12 +22,14 @@ obj-$(CONFIG_BLK_DEV_INITRD) := initramfs_data.o
 $(obj)/initramfs_data.o: $(obj)/initramfs_inc_data
 
 ramfs-input := $(strip $(shell echo $(CONFIG_INITRAMFS_SOURCE)))
+ramfs-compress := $(compress-y)
 cpio-data :=
 
 # If CONFIG_INITRAMFS_SOURCE is empty, generate a small initramfs with the
-# default contents.
+# default contents. No point in compressing it separately from the kernel.
 ifeq ($(ramfs-input),)
 ramfs-input := $(srctree)/$(src)/default_cpio_list
+ramfs-compress := shipped
 endif
 
 ifeq ($(words $(ramfs-input)),1)
@@ -42,7 +44,7 @@ endif
 # .cpio.*, use it directly as an initramfs, and avoid double compression.
 ifeq ($(words $(subst .cpio.,$(space),$(ramfs-input))),2)
 cpio-data := $(ramfs-input)
-compress-y := shipped
+ramfs-compress := shipped
 endif
 
 endif
@@ -76,14 +78,43 @@ quiet_cmd_initfs = GEN     $@
 # 2) There are changes in which files are included (added or deleted)
 # 3) If gen_init_cpio are newer than initramfs_data.cpio
 # 4) Arguments to gen_initramfs.sh changes
-$(obj)/initramfs_data.cpio: $(src)/gen_initramfs.sh $(obj)/gen_init_cpio $(deps_initramfs) FORCE
+$(obj)/initramfs_data.cpio: $(src)/gen_initramfs.sh $(obj)/gen_init_cpio \
+			    $(deps_initramfs) FORCE
 	$(call if_changed,initfs)
 
 endif
 
 $(obj)/initramfs_inc_data: $(cpio-data) FORCE
-	$(call if_changed,$(compress-y))
+	$(call if_changed,$(ramfs-compress))
 
 targets += initramfs_data.cpio initramfs_inc_data
 
 subdir-$(CONFIG_UAPI_HEADER_TEST) += include
+
+# modules.img: an initramfs image containing all the modules
+quiet_cmd_modinst = MODINST $(obj)/modules
+      cmd_modinst = \
+	rm -rf '$(obj)/modules' && \
+	$(MAKE) -f '$(srctree)/Makefile' O='$(objtree)' \
+		INSTALL_MOD_PATH='$(obj)/modules' \
+		INSTALL_MOD_LN=: modules_install
+
+quiet_cmd_modcpio = MODCPIO $@
+      cmd_modcpio = \
+	$(CONFIG_SHELL) $< -o $@ \
+	$(if $(CONFIG_INITRAMFS_ROOT_UID), -u $(CONFIG_INITRAMFS_ROOT_UID)) \
+	$(if $(CONFIG_INITRAMFS_ROOT_GID), -g $(CONFIG_INITRAMFS_ROOT_GID)) \
+	'$(obj)/modules'
+
+modules_order := \
+	$(if $(KBUILD_EXTMOD),$(KBUILD_EXTMOD),$(objtree))/modules.order
+
+$(obj)/modules.cpio: $(src)/gen_initramfs.sh $(obj)/gen_init_cpio \
+	$(modules_order) FORCE
+	$(call if_changed,modinst)
+	$(call if_changed,modcpio)
+
+$(obj)/modules.img: $(obj)/modules.cpio FORCE
+	$(call if_changed,$(compress-y))
+
+targets += modules.cpio modules.img modules
-- 
2.30.2

Re: [PATCH v2 2/3] usr, modules: Add build target for creating a modules initramfs image

[Masahiro Yamada]

On Tue, Apr 20, 2021 at 8:03 AM H. Peter Anvin <hpa@zytor.com> wrote:
>
> From: "H. Peter Anvin (Intel)" <hpa@zytor.com>
>
> Some distributions really cannot be booted without modules
> anymore. To allow an externally built kernel to be run, it is handy to
> be able to create an initramfs image with all the modules, that can
> appended to an existing initramfs image (preferrably without any
> modules, but not necessarily, since the /lib/modules path includes the
> kernel version.)
>
> This image is generated in usr/modules.img.
>
> Since this image may end up being rather large, change the default for
> CONFIG_INITRAMFS_COMPRESSION_*. We already have a special case in the
> Makefile to not re-compress an existing initramfs image; add the very
> small default builtin initramfs as another case to suppress the
> compression. To avoid losing the default compression setting, add a
> separate variable for the possibly overridden builtin initramfs
> compression method.
>
> Signed-off-by: H. Peter Anvin (Intel) <hpa@zytor.com>
> ---
>  Makefile       | 17 ++++++++++++-----
>  usr/.gitignore |  3 +++
>  usr/Kconfig    | 31 +++++++++++++++++--------------
>  usr/Makefile   | 39 +++++++++++++++++++++++++++++++++++----
>  4 files changed, 67 insertions(+), 23 deletions(-)
>
> diff --git a/Makefile b/Makefile
> index 4730cf156f6b..89e40760384d 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -622,7 +622,7 @@ KBUILD_MODULES :=
>  KBUILD_BUILTIN := 1
>
>  # If we have only "make modules", don't compile built-in objects.
> -ifeq ($(MAKECMDGOALS),modules)
> +ifeq (modules,$(MAKECMDGOALS))


Why are you changing this line?


>    KBUILD_BUILTIN :=
>  endif
>
> @@ -630,7 +630,8 @@ endif
>  # in addition to whatever we do anyway.
>  # Just "make" or "make all" shall build modules as well
>
> -ifneq ($(filter all modules nsdeps %compile_commands.json clang-%,$(MAKECMDGOALS)),)
> +ifneq ($(filter all modules nsdeps %compile_commands.json clang-% \
> +       modules.img usr/modules.img,$(MAKECMDGOALS)),)
>    KBUILD_MODULES := 1
>  endif
>
> @@ -1461,15 +1462,17 @@ modules_prepare: prepare
>  PHONY += modules_install
>  modules_install: _modinst_ _modinst_post
>
> +INSTALL_MOD_LN ?= ln -s
> +
>  PHONY += _modinst_
>  _modinst_:
>         @rm -rf $(MODLIB)/kernel
>         @rm -f $(MODLIB)/source
>         @mkdir -p $(MODLIB)/kernel
> -       @ln -s $(abspath $(srctree)) $(MODLIB)/source
> +       @$(INSTALL_MOD_LN) $(abspath $(srctree)) $(MODLIB)/source
>         @if [ ! $(objtree) -ef  $(MODLIB)/build ]; then \
> -               rm -f $(MODLIB)/build ; \
> -               ln -s $(CURDIR) $(MODLIB)/build ; \
> +               rm -rf $(MODLIB)/build ; \
> +               $(INSTALL_MOD_LN) $(CURDIR) $(MODLIB)/build ; \
>         fi
>         @sed 's:^:kernel/:' modules.order > $(MODLIB)/modules.order
>         @cp -f modules.builtin $(MODLIB)/
> @@ -1489,6 +1492,10 @@ modules_sign:
>         $(Q)$(MAKE) -f $(srctree)/scripts/Makefile.modsign
>  endif
>
> +# initramfs image containing all modules
> +modules.img usr/modules.img: modules

What is the difference between 'modules.img' and 'usr/modules.img'?





> +       $(Q)$(MAKE) $(build)=usr usr/modules.img
> +
>  else # CONFIG_MODULES
>
>  # Modules not configured
> diff --git a/usr/.gitignore b/usr/.gitignore
> index 935442ed1eb2..14e2fae88d8a 100644
> --- a/usr/.gitignore
> +++ b/usr/.gitignore
> @@ -2,3 +2,6 @@
>  gen_init_cpio
>  initramfs_data.cpio
>  /initramfs_inc_data
> +/modules
> +modules.cpio
> +modules.img
> diff --git a/usr/Kconfig b/usr/Kconfig
> index 8bbcf699fe3b..625372682b5e 100644
> --- a/usr/Kconfig
> +++ b/usr/Kconfig
> @@ -109,18 +109,23 @@ config RD_ZSTD
>           If unsure, say N.
>
>  choice
> -       prompt "Built-in initramfs compression mode"
> -       depends on INITRAMFS_SOURCE != ""
> -       help
> -         This option allows you to decide by which algorithm the builtin
> -         initramfs will be compressed.  Several compression algorithms are
> -         available, which differ in efficiency, compression and
> -         decompression speed.  Compression speed is only relevant
> -         when building a kernel.  Decompression speed is relevant at
> -         each boot. Also the memory usage during decompression may become
> -         relevant on memory constrained systems. This is usually based on the
> -         dictionary size of the algorithm with algorithms like XZ and LZMA
> -         featuring large dictionary sizes.
> +       prompt "Preferred initramfs compression mode"
> +       default INITRAMFS_COMPRESSION_XZ   if RD_XZ
> +       default INITRAMFS_COMPRESSION_GZIP if RD_GZIP
> +       default INITRAMFS_COMPRESSION_LZO  if RD_LZO
> +       default INITRAMFS_COMPRESSION_LZ4  if RD_LZ4
> +       default INITRAMFS_COMPRESSION_NONE
> +       help
> +         This option allows you to decide by which algorithm the
> +         builtin initramfs and usr/modules.img will be compressed.
> +         Several compression algorithms are available, which differ
> +         in efficiency, compression and decompression speed.
> +         Compression speed is only relevant when building a kernel.
> +         Decompression speed is relevant at each boot. Also the
> +         memory usage during decompression may become relevant on
> +         memory constrained systems. This is usually based on the
> +         dictionary size of the algorithm with algorithms like XZ and
> +         LZMA featuring large dictionary sizes.
>
>           High compression options are mostly useful for users who are
>           low on RAM, since it reduces the memory consumption during
> @@ -130,8 +135,6 @@ choice
>           compression tool to compress the generated initram cpio file for
>           embedding.
>
> -         If in doubt, select 'None'
> -
>  config INITRAMFS_COMPRESSION_GZIP
>         bool "Gzip"
>         depends on RD_GZIP
> diff --git a/usr/Makefile b/usr/Makefile
> index b1a81a40eab1..6257ba8690a3 100644
> --- a/usr/Makefile
> +++ b/usr/Makefile
> @@ -22,12 +22,14 @@ obj-$(CONFIG_BLK_DEV_INITRD) := initramfs_data.o
>  $(obj)/initramfs_data.o: $(obj)/initramfs_inc_data
>
>  ramfs-input := $(strip $(shell echo $(CONFIG_INITRAMFS_SOURCE)))
> +ramfs-compress := $(compress-y)
>  cpio-data :=
>
>  # If CONFIG_INITRAMFS_SOURCE is empty, generate a small initramfs with the
> -# default contents.
> +# default contents. No point in compressing it separately from the kernel.
>  ifeq ($(ramfs-input),)
>  ramfs-input := $(srctree)/$(src)/default_cpio_list
> +ramfs-compress := shipped
>  endif
>
>  ifeq ($(words $(ramfs-input)),1)
> @@ -42,7 +44,7 @@ endif
>  # .cpio.*, use it directly as an initramfs, and avoid double compression.
>  ifeq ($(words $(subst .cpio.,$(space),$(ramfs-input))),2)
>  cpio-data := $(ramfs-input)
> -compress-y := shipped
> +ramfs-compress := shipped
>  endif
>
>  endif
> @@ -76,14 +78,43 @@ quiet_cmd_initfs = GEN     $@
>  # 2) There are changes in which files are included (added or deleted)
>  # 3) If gen_init_cpio are newer than initramfs_data.cpio
>  # 4) Arguments to gen_initramfs.sh changes
> -$(obj)/initramfs_data.cpio: $(src)/gen_initramfs.sh $(obj)/gen_init_cpio $(deps_initramfs) FORCE
> +$(obj)/initramfs_data.cpio: $(src)/gen_initramfs.sh $(obj)/gen_init_cpio \
> +                           $(deps_initramfs) FORCE
>         $(call if_changed,initfs)
>
>  endif
>
>  $(obj)/initramfs_inc_data: $(cpio-data) FORCE
> -       $(call if_changed,$(compress-y))
> +       $(call if_changed,$(ramfs-compress))
>
>  targets += initramfs_data.cpio initramfs_inc_data
>
>  subdir-$(CONFIG_UAPI_HEADER_TEST) += include
> +
> +# modules.img: an initramfs image containing all the modules
> +quiet_cmd_modinst = MODINST $(obj)/modules
> +      cmd_modinst = \
> +       rm -rf '$(obj)/modules' && \
> +       $(MAKE) -f '$(srctree)/Makefile' O='$(objtree)' \
> +               INSTALL_MOD_PATH='$(obj)/modules' \
> +               INSTALL_MOD_LN=: modules_install
> +
> +quiet_cmd_modcpio = MODCPIO $@
> +      cmd_modcpio = \
> +       $(CONFIG_SHELL) $< -o $@ \
> +       $(if $(CONFIG_INITRAMFS_ROOT_UID), -u $(CONFIG_INITRAMFS_ROOT_UID)) \
> +       $(if $(CONFIG_INITRAMFS_ROOT_GID), -g $(CONFIG_INITRAMFS_ROOT_GID)) \
> +       '$(obj)/modules'
> +
> +modules_order := \
> +       $(if $(KBUILD_EXTMOD),$(KBUILD_EXTMOD),$(objtree))/modules.order
> +
> +$(obj)/modules.cpio: $(src)/gen_initramfs.sh $(obj)/gen_init_cpio \
> +       $(modules_order) FORCE
> +       $(call if_changed,modinst)
> +       $(call if_changed,modcpio)


Presumably, this will cause the same bug
as fixed by e9a1379f9219be439f47a0f063431a92dc529eda.


I did not look into the patch any further,
but as you know, it is really ugly,
and I do not like this patch.





> +
> +$(obj)/modules.img: $(obj)/modules.cpio FORCE
> +       $(call if_changed,$(compress-y))
> +
> +targets += modules.cpio modules.img modules
> --
> 2.30.2
>


--
Best Regards
Masahiro Yamada

[PATCH v2 3/3] x86/boot: Add option to add modules.img to {fd,hd,iso}image

[H. Peter Anvin]

From: "H. Peter Anvin (Intel)" <hpa@zytor.com>

Make it easy to generate a disk image which includes the all-modules
initramfs image.

Signed-off-by: H. Peter Anvin (Intel) <hpa@zytor.com>
---
 arch/x86/Makefile      |  3 ++-
 arch/x86/boot/Makefile | 20 ++++++++++++++++----
 2 files changed, 18 insertions(+), 5 deletions(-)

diff --git a/arch/x86/Makefile b/arch/x86/Makefile
index 943f26a32834..74f4e66568d7 100644
--- a/arch/x86/Makefile
+++ b/arch/x86/Makefile
@@ -309,7 +309,8 @@ define archhelp
   echo  '  isoimage		- Create a boot CD-ROM image (arch/x86/boot/image.iso)'
   echo  '			  bzdisk/fdimage*/hdimage/isoimage also accept:'
   echo  '			  FDARGS="..."  arguments for the booted kernel'
-  echo  '                  	  FDINITRD=file initrd for the booted kernel'
+  echo  '			  FDINITRD=file initrd for the booted kernel'
+  echo  '			  FDMODS=1 to include all modules as an initrd'
   echo  ''
   echo  '  kvm_guest.config	- Enable Kconfig items for running this kernel as a KVM guest'
   echo  '  xen.config		- Enable Kconfig items for running this kernel as a Xen guest'
diff --git a/arch/x86/boot/Makefile b/arch/x86/boot/Makefile
index dfbc26a8e924..a4f8c66a63d0 100644
--- a/arch/x86/boot/Makefile
+++ b/arch/x86/boot/Makefile
@@ -116,11 +116,23 @@ $(obj)/compressed/vmlinux: FORCE
 
 # Set this if you want to pass append arguments to the
 # bzdisk/fdimage/hdimage/isoimage kernel
-FDARGS =
+FDARGS   :=
 # Set this if you want one or more initrds included in the image
-FDINITRD =
+FDINITRD :=
+# Set this to 1 if you want usr/modules.img included in the image
+FDMODS   :=
 
-imgdeps = $(obj)/bzImage $(obj)/mtools.conf $(src)/genimage.sh
+imgdeps   := $(obj)/bzImage $(obj)/mtools.conf $(src)/genimage.sh
+fdinitrds := $(FDINITRD)
+
+ifneq ($(FDMODS),)
+imgdeps   += $(objtree)/usr/modules.img
+fdinitrds += $(objtree)/usr/modules.img
+
+$(objtree)/usr/modules.img:
+	$(Q)$(MAKE) -f $(srctree)/Makefile usr/modules.img
+KBUILD_MODULES := 1
+endif
 
 $(obj)/mtools.conf: $(src)/mtools.conf.in
 	sed -e 's|@OBJ@|$(obj)|g' < $< > $@
@@ -131,7 +143,7 @@ targets += mtools.conf
 # external dependencies.
 quiet_cmd_genimage = GENIMAGE $3
 cmd_genimage = $(BASH) $(srctree)/$(src)/genimage.sh $2 $3 $(obj)/bzImage \
-		$(obj)/mtools.conf '$(FDARGS)' $(FDINITRD)
+		$(obj)/mtools.conf '$(FDARGS)' $(fdinitrds)
 
 PHONY += bzdisk fdimage fdimage144 fdimage288 hdimage isoimage install
 
-- 
2.30.2

RE: [PATCH 0/3] x86 disk image and modules initramfs generation

[David Laight]

From: H. Peter Anvin
> Sent: 20 April 2021 00:03
> 
> When compiling on a different machine than the runtime target,
> including but not limited to simulators, it is rather handy to be able
> to produce a bootable image. The scripts for that in x86 are
> relatively old, and assume a BIOS system.

I've given up and copied the kernel tree onto all my test systems.

I needed something like 'make modules_install' and 'make install'
that would generated a directory tree that could be copied (scp -r)
onto the target system.

But the script to run 'update-grub' is all intwined in the commands.

You also don't get a copy of the headers.
Even for the local system (as root) you just get a symlink into
the source tree.
This causes a problem trying to build 'out of tree' modules
after updating the kernel source tree (but not rebulding).

I can (and do) write 'horrid' makefiles (gmake and nmake)
but this seemed to need more refactoring that I wanted to do.

	David

-
Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK
Registration No: 1397386 (Wales)

RE: [PATCH 0/3] x86 disk image and modules initramfs generation

[H. Peter Anvin]

Ok... not really sure how this relates to my patch. You are mentioned three separate things: modules, headers, and enough of the kernel machinery to build out of tree modules. The latter it normally done with a tree that corresponds to the state after build + "make clean"; which I *believe* is the same as after "make prepare". The former two are "make modules_install" and "make headers_install", respectively. Note you can direct them to directory hierarchies other than the local system ones for archiving.

It is simply not possible to provide a *general* solution that fits all distributions × all boot scenarios.

On April 20, 2021 1:30:07 AM PDT, David Laight <David.Laight@ACULAB.COM> wrote:
>From: H. Peter Anvin
>> Sent: 20 April 2021 00:03
>> 
>> When compiling on a different machine than the runtime target,
>> including but not limited to simulators, it is rather handy to be
>able
>> to produce a bootable image. The scripts for that in x86 are
>> relatively old, and assume a BIOS system.
>
>I've given up and copied the kernel tree onto all my test systems.
>
>I needed something like 'make modules_install' and 'make install'
>that would generated a directory tree that could be copied (scp -r)
>onto the target system.
>
>But the script to run 'update-grub' is all intwined in the commands.
>
>You also don't get a copy of the headers.
>Even for the local system (as root) you just get a symlink into
>the source tree.
>This causes a problem trying to build 'out of tree' modules
>after updating the kernel source tree (but not rebulding).
>
>I can (and do) write 'horrid' makefiles (gmake and nmake)
>but this seemed to need more refactoring that I wanted to do.
>
>	David
>
>-
>Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes,
>MK1 1PT, UK
>Registration No: 1397386 (Wales)

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.