Re: We have a whole new ton of goodies to investigate...

Re: We have a whole new ton of goodies to investigate...

[u-vpoa]

[reading the list archive, that's why this is not a proper thread reply]

Alan Cox wrote 16 Apr 13:49 2015
> > Thanks for the interesting info about various toolchains for Fuzix. I 
> > wonder why you include x86_16 target in the Fuzix challenge, as ELKS is 
> > already playing on that ground, but that is your baby, so...
> 
> I built the core that way mostly to check it compiled. Just portability
> testing.

I wonder, if Fuzix is sufficiently portable to run on 8086, could it
happen to offer a possibly even more suitable foundation for a usable
system, than the linux-derived code in ELKS?

I did not follow ELKS development over the years but it seems to be
a "from scratch" effort.

To compare, I saw once a fully functional BSD 2.9 kernel ported to 8086,
compatible with Venix/86 userspace and toolchain, which might have been
a more efficient (?) starting point too. Too bad, I think the port
is lost.

In other words, starting from a more limited (8-bit) or similar (16-bit)
platform might work better than "scaling down" from a 32-bit system.

The availability of the sources changed radically since the time when
ELKS was started. Now we have got even some pieces of Coherent/86.

> There certainly was
> plenty of stuff in ELKS that was taken from 386 Linux and is perhaps
> somewhat over-engineered for the job (buffer cache is perhaps one bit
> still)

I guess so. Sigh. The original V7's or even V6's quite basic design might
possibly suit 8086 better (I did not look at the ELKS kernel but I saw
V6/7 sources and also pieces of Venix/86 kernel which were apparently
build from literally the same C code, this seemed to work well).

> Going to multiple-segments for code isn't too difficult providing you
> don't follow every detail of pointers to function, but just stick to
> arrays of function pointers or function pointers within segment, but
> always seems to me like admitting defeat 8)

:)
The possiblity to use larger code makes life so much easier, without any
extra complexity in the compiler. Unfortunately there still are limits,
the total number of trampolines which are to fit in every code segment
and the size of a single function. Nevertheless it is very handy!

Rl

Re: We have a whole new ton of goodies to investigate...

[Alan Cox]

> I wonder, if Fuzix is sufficiently portable to run on 8086, could it
> happen to offer a possibly even more suitable foundation for a usable
> system, than the linux-derived code in ELKS?

I'm not sure where the changeover point would be. ELKS supports
asynchronous disk I/O properly, something FUZIX avoids. On an 8bit micro
its pretty much pure overhead. On a PC/XT it's less clear.

> > plenty of stuff in ELKS that was taken from 386 Linux and is perhaps
> > somewhat over-engineered for the job (buffer cache is perhaps one bit
> > still)
> 
> I guess so. Sigh. The original V7's or even V6's quite basic design might
> possibly suit 8086 better (I did not look at the ELKS kernel but I saw
> V6/7 sources and also pieces of Venix/86 kernel which were apparently
> build from literally the same C code, this seemed to work well).

ELKS is not that bloated to be honest. It's got a few areas that would
probably save a chunk of memory if fixed but the basic architecture is
pretty sound including basic 286 mode support.

Some of the utilities are a bit brain-dead or buggy and might benefit
from being pulled from elsewhere instead, and there are things lacking
(like the real bourne shell should fit fine and is nowdays available)

Alan

Re: We have a whole new ton of goodies to investigate...

[u-vpoa]

On Mon, Apr 27, 2015 at 12:31:28AM +0100, Alan Cox wrote:
> > I wonder, if Fuzix is sufficiently portable to run on 8086, could it
> > happen to offer a possibly even more suitable foundation for a usable
> > system, than the linux-derived code in ELKS?
> 
> I'm not sure where the changeover point would be. ELKS supports
> asynchronous disk I/O properly, something FUZIX avoids. On an 8bit micro
> its pretty much pure overhead. On a PC/XT it's less clear.

My experience with again Venix/86: it came with an async XT hard disk
driver. Unfortunately this did not work with differing hd controllers,
not all computers were born equal enough.

Replacing the hd driver with a home made bios based one did not produce
any "noticeable" performance impact (i.e. never measured but never bothered).

There was not that much of parallelizable i/o and the CPU/RAM was often
the bottleneck. This can be different on faster reimplementations of *86
but the disks are now faster as well.

> ELKS is not that bloated to be honest. It's got a few areas that would
> probably save a chunk of memory if fixed but the basic architecture is
> pretty sound including basic 286 mode support.

Nice! I had an impression that it was severely constrained by the
64k limit for itself, which would make it very hard to extend the
functionality. Of course a part of the problem is the compiler
efficiency limitations.

> Some of the utilities are a bit brain-dead or buggy and might benefit
> from being pulled from elsewhere instead, and there are things lacking
> (like the real bourne shell should fit fine and is nowdays available)

Sure. Fortunately the user space is much easier to replace.

Rl

Re: We have a whole new ton of goodies to investigate...

[Alan Cox]

> > ELKS is not that bloated to be honest. It's got a few areas that would
> > probably save a chunk of memory if fixed but the basic architecture is
> > pretty sound including basic 286 mode support.
> 
> Nice! I had an impression that it was severely constrained by the
> 64k limit for itself, which would make it very hard to extend the
> functionality. Of course a part of the problem is the compiler
> efficiency limitations.

Code space there is a lot to save with a better compiler (eg the coherent
one) or bcc improvements and architectural changes here and there. Data
can be drastically improved by getting the wait logic to not put pointer
chains on the stacks. That would mean that you only needed to keep a few
instances of kernel stacks in the kernel map.

In fact some of the Fuzix ports go so far as to only have one kernel and
interrupt stack in kernel data, and to save it on a task switch out and
restore it on a switch in (if the arriving task is not the old one).
Small machines don't task switch a lot and the hard cases can be
restructured so that the wakeup intelligence is in the waker not the
sleeper (which is more efficient anyway)

ie you avoid the plague of what is effectively

		while(!i_am_ready()) {
	                sleep_on(&foo);
		}

type stuff in the big Linux kernel and instead of waking one queue you
end up doing

		foreach(task_waiting_foo) {
			if (task_is_ready(task))
				wakeup(task);
		}

in the context of the waker.

You can't always do that but most cases work.

More seriously though what do you need to find extra space to add ? The
network stack needs work but is mostly out of kernel mapping (and
probably should stay so as with 2.11BSD), the rest of the basic POSIX
infrastructure is pretty complete. The Linux kernel has grown over ten
times bigger since 1.2 days but I don't think it's added that much useful
to the core code.

Alan

Re: We have a whole new ton of goodies to investigate...

[u-vpoa]

On Mon, Apr 27, 2015 at 11:53:27AM +0100, Alan Cox wrote:
> restructured so that the wakeup intelligence is in the waker not the
> sleeper (which is more efficient anyway)
> 
> ie you avoid the plague of what is effectively
> 
> 		while(!i_am_ready()) {
> 	                sleep_on(&foo);
> 		}
> 
> type stuff in the big Linux kernel and instead of waking one queue you
> end up doing
> 
> 		foreach(task_waiting_foo) {
> 			if (task_is_ready(task))
> 				wakeup(task);
> 		}
> 
> in the context of the waker.

I see.

> More seriously though what do you need to find extra space to add ? The
> network stack needs work but is mostly out of kernel mapping (and
> probably should stay so as with 2.11BSD), the rest of the basic POSIX

Sure.

> infrastructure is pretty complete. The Linux kernel has grown over ten
> times bigger since 1.2 days but I don't think it's added that much useful
> to the core code.

One extra thing I would appreciate in the kernel is of course the support
for code segment switching, making several-times-64k code sizes practical.

(Also there is always a desire for support of additional hardware,
this can easily push the limits both for the code and the data, if not
putting the drivers into different address spaces or strictly limiting
the number of simultaneously usable drivers)

This aside, do you say that the ELKS kernel is pretty much complete
nowadays, and that it is only user space which needs fixing to make the
system usable?

Rl

Re: We have a whole new ton of goodies to investigate...

[Alan Cox]

> > infrastructure is pretty complete. The Linux kernel has grown over ten
> > times bigger since 1.2 days but I don't think it's added that much useful
> > to the core code.
> 
> One extra thing I would appreciate in the kernel is of course the support
> for code segment switching, making several-times-64k code sizes practical.

If you are willing to accept violating the standard when comparing
function pointers, and you are willing to pull a couple of other stunts
then that is actually trivial. You generate far calls for all inter
function calls, and also tell the compiler to assume the arguments are 2
bytes further up the stack.

For function pointers (eg in arrays and structs) the code generates a
little stub function which saves the bank calls the function and returns,
so that a near call to the stub function goes to the correct far function
and back. Again on 8086 that's much easier to generate as it's basically

	func_stub:
		call far func
		retf

and all the calls to a function pointer then effectively end up as

	call far STUBSEG:ax

or similar depending how the compiler chooses to construct them.

Strictly speaking you can do maths on function pointers and cast them
about in odd ways so a function pointer ought to be 4 bytes, so a void *
needs to be 4 bytes and it all goes to hell, but in the real world the
above works even if it offends the more anal members of standards
committees.

I think the Coherent compiler can also do it "properly". It can for 286
anyway, I've not been through the 8086 version in enough detail to figure
it out.

> (Also there is always a desire for support of additional hardware,
> this can easily push the limits both for the code and the data, if not
> putting the drivers into different address spaces or strictly limiting
> the number of simultaneously usable drivers)

On a box with very little memory it makes little sense to include drivers
you don't need!

> This aside, do you say that the ELKS kernel is pretty much complete
> nowadays, and that it is only user space which needs fixing to make the
> system usable?

If you compare it with 4BSD or System 5 on the syscall level I'd say yes.
It may well need bug fixing and structural work for size but it's only
really lacking the relatively modern syscalls, most of which are
inappropriate and the others could I guess be added but none of the small
userspaces use them - openat etc.

The networking side lacks IPv6 and a lot of other bits, but that's
outside the core, and could probably be reworked to keep the current
kernel bits pretty much as is and adopt lwip for the stack.

Alan

Re: We have a whole new ton of goodies to investigate...

[u-vpoa]

On Mon, Apr 27, 2015 at 02:35:54PM +0100, Alan Cox wrote:
> > One extra thing I would appreciate in the kernel is of course the support
> > for code segment switching, making several-times-64k code sizes practical.
> 
> If you are willing to accept violating the standard when comparing
> function pointers, and you are willing to pull a couple of other stunts
> then that is actually trivial. You generate far calls for all inter
> function calls, and also tell the compiler to assume the arguments are 2
> bytes further up the stack.

This implies that the compiler must be "more-than-16-bits-aware" and
produce a larger code.

I like the trampolines approach, when you have all pointers, including
the function ones, as 16 bit, and you can use any compiler.
The address of a function is the same in all segments, which is
the address of its linker-generated trampoline. There is some size and
run time overhead but only at inter-segment calls and at function pointer
dereferencing. This needs only support in the linker and in the kernel
(like "ld -m" on Venix/86).

> I think the Coherent compiler can also do it "properly". It can for 286
> anyway, I've not been through the 8086 version in enough detail to figure
> it out.

Wonder how hard it were to port it to run under ELKS.

> > this can easily push the limits both for the code and the data, if not
> > putting the drivers into different address spaces or strictly limiting
> > the number of simultaneously usable drivers)
> 
> On a box with very little memory it makes little sense to include drivers
> you don't need!

Oh yes, but you can happen to have lots of various connected hardware _and_
many times 64K memory which would make this argument void.

> If you compare it with 4BSD or System 5 on the syscall level I'd say yes.

Nice!

> It may well need bug fixing and structural work for size but it's only
> really lacking the relatively modern syscalls, most of which are
> inappropriate and the others could I guess be added but none of the small
> userspaces use them - openat etc.

Yes.

> The networking side lacks IPv6 and a lot of other bits, but that's
> outside the core, and could probably be reworked to keep the current
> kernel bits pretty much as is and adopt lwip for the stack.

+1

Rl

Re: We have a whole new ton of goodies to investigate...

[LM]

Has anyone looked into using OpenWatcom with Elks?

http://www.openwatcom.org/index.php/Main_Page

http://sourceforge.net/projects/openwatcom/

I've used Watcom before it was Open Source as my compiler of choice
and I still find occasion to work with it even now.  It supports C,
C++ and Fortran on a variety of platforms.  It's the only decent 16
bit C++ compiler I've been able to find.  The Open Source version
added a program/front end so one could use a command line similar to
the gnu compiler and more gnu autotools/make scripts would work with
it without needing major modifications.

Re: We have a whole new ton of goodies to investigate...

[Alan Cox]

On Thu, 16 Apr 2015 07:14:03 -0400
LM <lmemsm@gmail.com> wrote:

> Has anyone looked into using OpenWatcom with Elks?
> 
> http://www.openwatcom.org/index.php/Main_Page

I never managed to get it to build on Linux, nor the fork which seems to
be active. I'd be very interested to know if anyone has.

Alan

We have a whole new ton of goodies to investigate...

[Alan Cox]

http://www.nesssoftware.com/home/mwc/source.php

And in the Coherent tree there isn't just the 386 release but the 286
release and all the compiler and tool chain pieces plus all the
utilities, manuals, etc. The dump seems to include 8086, 286 and 386
compilers and assemblers but not an 8086/286 linker.

Alan

Re: We have a whole new ton of goodies to investigate...

[MFLD]

Hello,

Started to have a look to that huge stuff (not really sorted...), but 
could you please explain the idea you have in mind ? Is this a suggest 
to replace the DEV86 toolchain by something else ?

By the way, for the background, could you also tell what is the 
toolchain you target for Fuzix on 8080/Z80 ?

Regards,

MFLD

Le 03/04/2015 22:40, Alan Cox a écrit :
> http://www.nesssoftware.com/home/mwc/source.php
>
> And in the Coherent tree there isn't just the 386 release but the 286
> release and all the compiler and tool chain pieces plus all the
> utilities, manuals, etc. The dump seems to include 8086, 286 and 386
> compilers and assemblers but not an 8086/286 linker.
>
--
To unsubscribe from this list: send the line "unsubscribe linux-8086" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: We have a whole new ton of goodies to investigate...

[Alan Cox]

On Wed, 15 Apr 2015 19:10:05 +0200
MFLD <mfld.fr@gmail.com> wrote:

> Hello,
> 
> Started to have a look to that huge stuff (not really sorted...), but 
> could you please explain the idea you have in mind ? Is this a suggest 
> to replace the DEV86 toolchain by something else ?

At least the compiler. The Coherent compiler is an ANSI C compiler with a
reasonably good optimiser on it. The tree seems to have versions for
8086, 80286 (protected mode included) and 80386 (which is probably not
useful)

> By the way, for the background, could you also tell what is the 
> toolchain you target for Fuzix on 8080/Z80 ?

Z80 uses SDCC. I kep a couple of patches on top, one for compactness
which sdcc has now partially adopted, the other of which adds support for
banked binaries.

6502 uses cc65

6809, 68000 use gcc

8086 I did a test build of the core with bcc, but would probably use the
coherent compiler eventually if it did 8086. Not sure there is that much
point doing 8086, fixing the dumber bits ELKS inherited from 32bit Linux
would probably shrink it down to the point the size change wasn't that
big a deal.

Alan

Re: We have a whole new ton of goodies to investigate...

[MFLD]

Hello,

Thanks for the interesting info about various toolchains for Fuzix. I 
wonder why you include x86_16 target in the Fuzix challenge, as ELKS is 
already playing on that ground, but that is your baby, so...

Back to ELKS : from what I see in the Coherent tree ('duff' was my 
friend to have a more clear view), CC and AS386 output object files in 
COFF, not A.OUT as BCC and AS86, so we would have to migrate to the 
associated LD386, and update other tools as DIS88 (and the upcoming 
MON86 and EMU86, scoop!). And despite that move, staying out of the 
mainsteam on an unmaintained toolchain...

I am afraid it would be a too heavy workload just to have fun with old 
hardware or for learning purpose. I would instead spend my spare time in 
splitting the recursive parser / code generator in BCC to keep it 
simple, easy understandable and more maintainable (BCC = Basic C 
Compiler for Beginner ?), following the PCC design without portability 
in mind, and to be able to improve the register allocator without 
dealing with any generic register representation as in GCC.

The latest because I see Juan or Jody saving few bytes with big effort, 
while many bytes could be saved at the root by optimizing register usage 
in function bodies (starting with the SI / DI push / pop for nothing); 
and because I am already breaking the 64K segment limit with a first 
draft of an Ethernet driver for my board. ANSI non-compliance is not 
really a concern at that stage. But I could understand my rational is 
only valid for my specific use case...

Regards,

MFLD

Le 15/04/2015 23:41, Alan Cox a écrit :
> Started to have a look to that huge stuff (not really sorted...), but
> could you please explain the idea you have in mind ? Is this a suggest
> to replace the DEV86 toolchain by something else ?
> >At least the compiler. The Coherent compiler is an ANSI C compiler with a
> reasonably good optimiser on it. The tree seems to have versions for
> 8086, 80286 (protected mode included) and 80386 (which is probably not
> useful)
>
> >8086 I did a test build of the core with bcc, but would probably use 
> the coherent compiler eventually if it did 8086. Not sure there is 
> that much point doing 8086, fixing the dumber bits ELKS inherited from 
> 32bit Linux would probably shrink it down to the point the size change 
> wasn't that big a deal. Alan 

--
To unsubscribe from this list: send the line "unsubscribe linux-8086" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: We have a whole new ton of goodies to investigate...

[Alan Cox]

On Thu, 16 Apr 2015 09:35:34 +0200
MFLD <mfld.fr@gmail.com> wrote:

> Hello,
> 
> Thanks for the interesting info about various toolchains for Fuzix. I 
> wonder why you include x86_16 target in the Fuzix challenge, as ELKS is 
> already playing on that ground, but that is your baby, so...

I built the core that way mostly to check it compiled. Just portability
testing.

> 
> Back to ELKS : from what I see in the Coherent tree ('duff' was my 
> friend to have a more clear view), CC and AS386 output object files in 
> COFF, not A.OUT as BCC and AS86, so we would have to migrate to the 

as386 outputs coff for 386 code. The 286/8086 ones appear to produce
*.out (a.out/l.ou/x.out) type formats.
 
> splitting the recursive parser / code generator in BCC to keep it 
> simple, easy understandable and more maintainable (BCC = Basic C 
> Compiler for Beginner ?), following the PCC design without portability 

Bruce's C compiler after the man who wrote it, initially for 6809 and
later for 8086.

> in mind, and to be able to improve the register allocator without 
> dealing with any generic register representation as in GCC.

That and an ANSI parser would make a huge difference I agree.
 
> The latest because I see Juan or Jody saving few bytes with big effort, 
> while many bytes could be saved at the root by optimizing register usage 
> in function bodies (starting with the SI / DI push / pop for nothing); 
> and because I am already breaking the 64K segment limit with a first 
> draft of an Ethernet driver for my board. ANSI non-compliance is not 
> really a concern at that stage. But I could understand my rational is 
> only valid for my specific use case...

ANSI is valuable for debug.

For size I'd also look at things like wait queues and any repetetive long
winded pieces of code (eg Linux style wait queues). There certainly was
plenty of stuff in ELKS that was taken from 386 Linux and is perhaps
somewhat over-engineered for the job (buffer cache is perhaps one bit
still)

Going to multiple-segments for code isn't too difficult providing you
don't follow every detail of pointers to function, but just stick to
arrays of function pointers or function pointers within segment, but
always seems to me like admitting defeat 8)

Alan