Re: [PATCH] x86: Remove a.out support

From: Arnd Bergmann <arnd@arndb.de>
To: James Jones <linux@theinnocuous.com>,
	Borislav Petkov <bp@alien8.de>,
	"the arch/x86 maintainers" <x86@kernel.org>,
	Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
	Kees Cook <keescook@chromium.org>
Subject: Re: [PATCH] x86: Remove a.out support
Date: Wed, 16 Mar 2022 13:38:31 +0100	[thread overview]
Message-ID: <CAK8P3a28dpyEM2+vM+ePZzeFc539b7w_8FDEoRke-j+3AQVZAA@mail.gmail.com> (raw)
In-Reply-To: <bfbd9394-161b-0e70-00c5-79d0dd722e08@theinnocuous.com>

On Wed, Mar 16, 2022 James Jones <linux@theinnocuous.com> wrote:
> On 3/15/22 12:51 PM, Kees Cook wrote:
>
> Probably getting a bit off topic, but I did spend a few hours searching
> around for any existing tools to convert a binary from a.out->ELF, and
> trying to come up with something myself by extracting the sections with
> objdump and re-combining them into an ELF using a linker script placing
> the sections at the same locations. I couldn't get it working in an
> evening or two messing with it so I moved on, but I agree something like
> this seems possible in theory. I got 'mac' converted to an ELF that
> would load up and print its input prompt if run without parameters, but
> only if run as root. As a regular user, it segfaults somewhere before it
> even starts executing in the text section AFAICT. Any actual assembling
> segfaults even when running as root. I never got 'aln' to do anything
> but segfault.
>
> I dug the scripts up in the state they were in when I gave up (September
> 2020 according to mtime), and put them on github in case anyone wants to
> have a go at it:
>
> https://github.com/cubanismo/aout-to-elf/
>
> It was an interesting problem in its own right, and I'd be curious to
> know what I missed.

I noticed that qemu-user has code to run the two other native binary
formats (elf and flat) that the kernel supports, in
https://github.com/qemu/qemu/blob/master/linux-user/linuxload.c

The qemu code is fairly close to the kernel version it is derived
from, and the a.out loader in the kernel is the simplest of those.

If performance is not overly important, I think it should be possible
to turn the Linux binfmt_aout.c code into something that works
in qemu, allowing you to run these executables through CPU
emulation on any architecture. No idea how much work this
is in the end, but it's likely easier than a standalone a.out loader
in user space, or a conversion to ELF format.

       Arnd