* Condensing and re-coding programs to save space
@ 2017-02-27 19:53 Jody Bruchon
2017-02-27 22:02 ` Marc-F. LUCCA-DANIAU
2017-02-28 21:12 ` Alan Cox
0 siblings, 2 replies; 7+ messages in thread
From: Jody Bruchon @ 2017-02-27 19:53 UTC (permalink / raw)
To: ELKS
The recent flare-up of networking development has led to a lot of
additional small programs on the full disk images and that forced me to
eliminate other programs from being installed automatically to avoid
image build failures. I'd like to suggest some things that can be done
to reduce the amount of space these are taking up.
Since all programs are statically linked to C library components, it
would be helpful to condense similar programs into a single binary where
possible, have the Makefile's install section create relative symlinks,
and change the program's behavior based on the name used to invoke it.
The code to do this obviously has its own overhead as well, so it's only
practical if there are enough binaries to justify the trouble. You can
find useful stuff for this in the "busyelks" project I had been trying
out and left in the code.
It may also make sense to condense the new non-standard programs like
the "echo tests" into one binary and choose the specific functionality
with a command parameter instead.
Another thing is the use of write() instead of [vf]printf(). If a
program can completely avoid the use of the printf() family of functions
and use write() instead, it'll make the program significantly smaller on
disk and in memory. If you must use printf() even once then this isn't
going to be helpful. If you can get away with write() everywhere
instead, the program will shrink and we'll be able to fit more on disk.
I'm sure more suggestions could be made and they would be most welcome.
-Jody
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Condensing and re-coding programs to save space
2017-02-27 19:53 Condensing and re-coding programs to save space Jody Bruchon
@ 2017-02-27 22:02 ` Marc-F. LUCCA-DANIAU
2017-02-27 22:11 ` Jody Bruchon
2017-02-28 21:12 ` Alan Cox
1 sibling, 1 reply; 7+ messages in thread
From: Marc-F. LUCCA-DANIAU @ 2017-02-27 22:02 UTC (permalink / raw)
To: ELKS
The unit tests for networking that I grouped in /elkscmd/test should not
be included in the default images (because "test"...), and would be
removed in next commits to make room. But we should also provide a more
convenient way to customize the image, for a developer to be able to
quickly add them back when doing non-regression testing.
Today we have to modify the Makefile to include / exclude a program, 5
different targets for partial / full images, multiplied by the available
image formats. Plus now a build_hd_image script that is not consistent
(and IMHO, poorly designed as it unpacks an existing image to build
another one, in place of building it directly).
Would be far better to follow the Buildroot / Busybox way, and to select
the applets to be installed on the target in a single location. We
already have a quite nice kernel configuration tool, why not using it
for that selection and the choice of the image format ?
MFLD
Le 27/02/2017 à 20:53, Jody Bruchon a écrit :
> The recent flare-up of networking development has led to a lot of
> additional small programs on the full disk images and that forced me
> to eliminate other programs from being installed automatically to
> avoid image build failures. I'd like to suggest some things that can
> be done to reduce the amount of space these are taking up.
>
> Since all programs are statically linked to C library components, it
> would be helpful to condense similar programs into a single binary
> where possible, have the Makefile's install section create relative
> symlinks, and change the program's behavior based on the name used to
> invoke it. The code to do this obviously has its own overhead as well,
> so it's only practical if there are enough binaries to justify the
> trouble. You can find useful stuff for this in the "busyelks" project
> I had been trying out and left in the code.
>
> It may also make sense to condense the new non-standard programs like
> the "echo tests" into one binary and choose the specific functionality
> with a command parameter instead.
>
> Another thing is the use of write() instead of [vf]printf(). If a
> program can completely avoid the use of the printf() family of
> functions and use write() instead, it'll make the program
> significantly smaller on disk and in memory. If you must use printf()
> even once then this isn't going to be helpful. If you can get away
> with write() everywhere instead, the program will shrink and we'll be
> able to fit more on disk.
>
> I'm sure more suggestions could be made and they would be most welcome.
>
> -Jody
> --
> To unsubscribe from this list: send the line "unsubscribe linux-8086" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Condensing and re-coding programs to save space
2017-02-27 22:02 ` Marc-F. LUCCA-DANIAU
@ 2017-02-27 22:11 ` Jody Bruchon
2017-02-28 9:31 ` Marc-François LUCCA-DANIAU
0 siblings, 1 reply; 7+ messages in thread
From: Jody Bruchon @ 2017-02-27 22:11 UTC (permalink / raw)
To: ELKS
On 2017-02-27 17:02, Marc-F. LUCCA-DANIAU wrote:
> The unit tests for networking that I grouped in /elkscmd/test should
> not be included in the default images (because "test"...), and would
> be removed in next commits to make room. But we should also provide a
> more convenient way to customize the image, for a developer to be able
> to quickly add them back when doing non-regression testing.
I think that the current way of using Makefile stuff to decide what goes
in an image does need to be replaced with something more flexible.
> Today we have to modify the Makefile to include / exclude a program, 5
> different targets for partial / full images, multiplied by the
> available image formats. Plus now a build_hd_image script that is not
> consistent (and IMHO, poorly designed as it unpacks an existing image
> to build another one, in place of building it directly).
I accepted it because it does the job of making a bootable hard drive
image; it does need to be changed in the future, but it is a nice
stop-gap measure to make more free space available on a root image while
all this testing work is being done.
> Would be far better to follow the Buildroot / Busybox way, and to
> select the applets to be installed on the target in a single location.
> We already have a quite nice kernel configuration tool, why not using
> it for that selection and the choice of the image format ?
Well, see...that's a problem. The ELKS kernel uses a really old copy of
Kconfig from Linux and when I looked at extracting the one used in a
more modern Linux source code base the task was far from trivial. A lot
of ELKS was cobbled together well over a decade ago. I'm open to moving
to a more all-encompassing build system if someone wants to do it, but
it certainly will not be a simple task.
-Jody
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Condensing and re-coding programs to save space
2017-02-27 22:11 ` Jody Bruchon
@ 2017-02-28 9:31 ` Marc-François LUCCA-DANIAU
2017-02-28 11:54 ` Georg Potthast 2
0 siblings, 1 reply; 7+ messages in thread
From: Marc-François LUCCA-DANIAU @ 2017-02-28 9:31 UTC (permalink / raw)
To: ELKS
> Well, see...that's a problem. The ELKS kernel uses a really old copy of
> Kconfig from Linux and when I looked at extracting the one used in a more
> modern Linux source code base the task was far from trivial. A lot of ELKS
> was cobbled together well over a decade ago. I'm open to moving to a more
> all-encompassing build system if someone wants to do it, but it certainly
> will not be a simple task.
Mmm... let us be pragmatic and stop dreaming with stuff like "changing
the compiler", "moving to another build system", etc. ELKS is
definitively a "hobby" / "educational" project, with a few active
contributors, and is intended to have fun, not for business (except
data erasing / recovery on antic machines ?). So our development
strategy should be consistent with that matter of fact : little steps
forward, and keep consistency and stability. Obsolescence is
absolutely NOT a concern here, as long we find compatible emulators
and embedded systems to play with.
Back to the current Kconfig: it works quite well, fulfills the need,
so why breaking it with an hazardous upgrade ? I only propose to use
it not only for the kernel, but also for the applets and image format
selection. Not to upgrade it.
MFLD
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Condensing and re-coding programs to save space
2017-02-28 9:31 ` Marc-François LUCCA-DANIAU
@ 2017-02-28 11:54 ` Georg Potthast 2
2017-02-28 12:17 ` Jody Bruchon
0 siblings, 1 reply; 7+ messages in thread
From: Georg Potthast 2 @ 2017-02-28 11:54 UTC (permalink / raw)
To: ELKS
I have not tested it, but my understanding is that the Makefile puts different files on the floppy images depending on the type selected. So a 360 Kb image has fewer files than a 1.44 image. As long as the list of files for each format in the Makefile is maintained I think this would be sufficient for now.
With the new ethernet support one could set up a repository ;)
Georg
> Marc-François LUCCA-DANIAU <mfld.fr@gmail.com> hat am 28. Februar 2017 um 10:31 geschrieben:
>
>
> > Well, see...that's a problem. The ELKS kernel uses a really old copy of
> > Kconfig from Linux and when I looked at extracting the one used in a more
> > modern Linux source code base the task was far from trivial. A lot of ELKS
> > was cobbled together well over a decade ago. I'm open to moving to a more
> > all-encompassing build system if someone wants to do it, but it certainly
> > will not be a simple task.
>
> Mmm... let us be pragmatic and stop dreaming with stuff like "changing
> the compiler", "moving to another build system", etc. ELKS is
> definitively a "hobby" / "educational" project, with a few active
> contributors, and is intended to have fun, not for business (except
> data erasing / recovery on antic machines ?). So our development
> strategy should be consistent with that matter of fact : little steps
> forward, and keep consistency and stability. Obsolescence is
> absolutely NOT a concern here, as long we find compatible emulators
> and embedded systems to play with.
>
> Back to the current Kconfig: it works quite well, fulfills the need,
> so why breaking it with an hazardous upgrade ? I only propose to use
> it not only for the kernel, but also for the applets and image format
> selection. Not to upgrade it.
>
> MFLD
> --
> To unsubscribe from this list: send the line "unsubscribe linux-8086" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Condensing and re-coding programs to save space
2017-02-28 11:54 ` Georg Potthast 2
@ 2017-02-28 12:17 ` Jody Bruchon
0 siblings, 0 replies; 7+ messages in thread
From: Jody Bruchon @ 2017-02-28 12:17 UTC (permalink / raw)
To: ELKS
On 2017-02-28 6:54 AM, Georg Potthast 2 wrote:
> I have not tested it, but my understanding is that the Makefile puts different files on the floppy images depending on the type selected. So a 360 Kb image has fewer files than a 1.44 image. As long as the list of files for each format in the Makefile is maintained I think this would be sufficient for now.
The lists are really bad. Each individual Makefile in each program
directory is responsible for deciding what binaries go into what disk
image type. Adding a disk image type requires either making it a clone
of one of the three "standard" types (thus full3 and full5 are both
"full" types) or adding a new image type to every single Makefile in
every program directory. This definitely needs to change.
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Condensing and re-coding programs to save space
2017-02-27 19:53 Condensing and re-coding programs to save space Jody Bruchon
2017-02-27 22:02 ` Marc-F. LUCCA-DANIAU
@ 2017-02-28 21:12 ` Alan Cox
1 sibling, 0 replies; 7+ messages in thread
From: Alan Cox @ 2017-02-28 21:12 UTC (permalink / raw)
To: Jody Bruchon; +Cc: ELKS
You could take a look at the FUZIX userspace. Much of it is far more
compact and a lot of it uses write() etc for size saving. The library
routines are also often a lot more memory efficient (eg the malloc is
much tinier as it was rewritten after I found the ELKS one was being
really buggy).
There shouldn't be too much churn involved as the FUZIX libc is based in
part on the ELKS one with other bits taken from MUSL and the like (and
the FP library largely comes from the Sun contributions)
A second option for the C library is to implement shared libraries not
necessarily by truely sharing it as a library with its own cs: but using
the callers ds:/ss:, as that involves some fun for fixing up callbacks,
but even something as simple as loading it into the process address space
from a different file when execve runs.
And the third of course is to implement support for something like
exomizer, that doesn't even need kernel support except for split I/D
binaries.
The compressor needs a big computer but the decompressor is tiny.
Alan
^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2017-02-28 21:12 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-02-27 19:53 Condensing and re-coding programs to save space Jody Bruchon
2017-02-27 22:02 ` Marc-F. LUCCA-DANIAU
2017-02-27 22:11 ` Jody Bruchon
2017-02-28 9:31 ` Marc-François LUCCA-DANIAU
2017-02-28 11:54 ` Georg Potthast 2
2017-02-28 12:17 ` Jody Bruchon
2017-02-28 21:12 ` Alan Cox
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.