Memory issues and USB support?

Memory issues and USB support?

[Derek Johansen]

Just out of curiosity:

1) Why the restriction to ~640K of memory for running ELKS.  Lots of
implementations banging up against this lately.  Can ELKS be modified
to use all available memory (e,g, my virtual system has ~1 GB
available; could ELKS be modified to use all of that)?

2)  Does ELKS support USB devices?  Will it ever?  Is it even possible?

Thanks,
Derek

Re: Memory issues and USB support?

[Jody Bruchon]

On 2017-03-19 2:02 AM, Derek Johansen wrote:
> 1) Why the restriction to ~640K of memory for running ELKS
The Intel 8086 can address 1024K of memory. On all IBM PCs and 
compatibles, the upper 384K is reserved for I/O and ROM. ELKS primarily 
targets 8086-based IBM-compatible PCs. Yes, it is possible to use more 
than 640K of RAM, but that also requires the use of protected mode and 
BIOS support for hardware isn't available in protected mode. We would 
have to have device drivers that can access the hardware directly 
instead. This would also require that we have kernel support for both 
real and protected mode. I am aware of "unreal" mode but it also has its 
fair share of quirks and requires at least a 386.
> 2)  Does ELKS support USB devices?  Will it ever?  Is it even possible?
ELKS does not target platforms with USB. Technically it is possible to 
add USB support, but a lot more would be needed than just support for 
USB itself.

At some point "can we add support for this 386+ feature?" becomes "can 
we just morph ELKS into something like an older Linux distribution?"

Re: Memory issues and USB support?

[Alan Cox]

On Sun, 19 Mar 2017 00:02:20 -0600
Derek Johansen <djohanse678@gmail.com> wrote:

> Just out of curiosity:
> 
> 1) Why the restriction to ~640K of memory for running ELKS.  Lots of
> implementations banging up against this lately.  Can ELKS be modified
> to use all available memory (e,g, my virtual system has ~1 GB
> available; could ELKS be modified to use all of that)?

Actually applications are hitting the 64K code / 64K data limit. The
total system limit is 1MB for an 8086 but on most 8086 platforms a chunk
of that is video, MMIO and BIOS so you get 640K. Beyond that 640K you can
swap, so for a normal sane system from that era running appropriate apps
640K should be plenty of memory.

On a 286 in protected mode (which ELKS kind of supports) you should be
able to go to 16MB (again you don't get 384K of it on a PC). There is
some minimal 286 protected mode support in ELKS but only for 64K/64K
binaries.

Getting bigger user applications means teaching the compiler to generate
protected mode safe larger mode binaries. For code that's actually not
too horrible providing you understand the LDT/GDT have to be OS managed
to allow for swap and memory compaction. For data it's a lot more
exciting if you want to support objects bigger than 64Kbyte in size, and
still quite 'interesting' from a compiler perspective anyway.

On an 8086 the other option is to support EMM boards. That allows you up
to 4MB or so of which 64K at a time is visible in a fixed memory window -
that's actually ideal for ELKS but fiddly for memory management as any
application that's got split code/data and can be over 64K will need to
be arranged so only half of it lives in EMM space.

> 2)  Does ELKS support USB devices?  Will it ever?  Is it even possible?

No and USB is actually very hard for three reasons
1. There is almost no ISA USB hardware so you'll need to design your own
2. USB adds device hotplug to your system, something ELKS and older
OSs - older Linux included simply have no concept of. That's a major
rework, although you can dodge some of it with restricted functionality
3. The USB stack itself is a complicated beast needing a lot of memory

And if you are trying to do a small OS on an old 386 or later x86 platform
I'd just grab NetBSD/OpenBSD. They've not inflated in the same way Linux
(and in particular the Linux user space) has.

Alan

Re: Memory issues and USB support?

[Alan Cox]

> Apologies if I'm getting things extremely muddled up here, but I take
> it that since you're not in protected mode, you don't get to have page
> faults which give the kernel the opportunity to make the code/data the
> application wants available, and instead the application is going to
> have to be aware of the fact that some of it is not necessarily
> directly addressable without asking the kernel first?

The 8086 has no concept of page faults at all. The bigger limit is that
for small mode you've only got 64K offsets from each segment register.
The moment you try and deal with more than 64K data it gets ugly. More
than 64K code is ugly but slightly less so.

In the Linux 8086 case the big problem is the kernel stack for each
process comes from that 64K. Original Unix on the PDP-11 instead was able
to bank in and out a chunk of memory and put all the kernel stack and
per-process data that no other process needed in that chunk ('the uarea').

8086 doesn't have the MMU facilities for that but it does have the option
of using rep movs to block move such an area in and out on a task switch.

> I always knew about EMS having a 64KB page window from the DOS days,
> but https://en.wikipedia.org/wiki/Expanded_memory says that in LIM EMS
> 4.0 "any 16 KB region in lower RAM [could] be mapped to expanded
> memory", which makes it sound to me like the ELKS kernel could at
> least map in a process's memory when it's about to run it and then
> unmap it when it switches to another process?  As it is, with LIM EMS
> 3.2, even if you can only map in a single 64KB page at a time, you
> could at least do that for some of their memory.  I guess you also get
> some memory protection from this, right?

Not a lot no. Most of this comes in with later processors.

The 286 has the 16bit segment limits but supports a protected mode (and
ELKS can use this in its most basic form), but any large applications
would need a rather different compiler and a load of kernel pieces to
support local descriptor table modification (as well as being really slow
when handling large stuff).

The 386 added 32bit mode and page basd protection.

Alan

Re: Memory issues and USB support?

[David O'Shea]

Hi all,

I'm new here, and I'm just about to send a bunch of emails about
various things, but I will probably quieten down fairly quickly, so
don't worry :)

> From: Alan Cox <alan@xxxxxxxxxxxxxxxxxxx>
> Date: Sun, 19 Mar 2017 14:27:45 +0000
>
> ...
>
> On an 8086 the other option is to support EMM boards. That allows you up
> to 4MB or so of which 64K at a time is visible in a fixed memory window -
> that's actually ideal for ELKS but fiddly for memory management as any
> application that's got split code/data and can be over 64K will need to
> be arranged so only half of it lives in EMM space.

Apologies if I'm getting things extremely muddled up here, but I take
it that since you're not in protected mode, you don't get to have page
faults which give the kernel the opportunity to make the code/data the
application wants available, and instead the application is going to
have to be aware of the fact that some of it is not necessarily
directly addressable without asking the kernel first?

I always knew about EMS having a 64KB page window from the DOS days,
but https://en.wikipedia.org/wiki/Expanded_memory says that in LIM EMS
4.0 "any 16 KB region in lower RAM [could] be mapped to expanded
memory", which makes it sound to me like the ELKS kernel could at
least map in a process's memory when it's about to run it and then
unmap it when it switches to another process?  As it is, with LIM EMS
3.2, even if you can only map in a single 64KB page at a time, you
could at least do that for some of their memory.  I guess you also get
some memory protection from this, right?

Just curious - not planning to try my hand at implementing any of that, sorry!

Thanks in advance,
David