[OT] Documentation for PC Architecture

[OT] Documentation for PC Architecture

[Paolo Ornati]

Hi,
(sorry for the OT but I think here someone will relpy :-)

Problem: I'm interested on PC architecture and looking for documentation I've 
found al lot of good doc for the CPU (Intel manuals...) but about the basic 
hardware that surround it I've found only a lot of "pieces of a puzzle"!

IOW: there is an (un)official document spoking about general PC architecture?

In this document I want to find some basic things about PC hardware and 
especially about these themes (from the logical point of view):

1) communication between CPU, memory, I/O devices:
Example:
If I write byte 'A' to physical address 0xb8000 how does this go to video 
memory instead of RAM?
I know about memory mapped I/O but I don't know who "decides" WHAT must go 
WHERE.
Since the CPU can't know if it's writing to memory or some memory mapped I/O 
device I suppose that is the "BUS CONTROLLER" (PCI BRIDGE in case of a PCI 
BUS) to write the byte 'A' to the right device.
But how does the "BUS CONTROLLER" do this association?
For now I think in this simple way:
- all the devices are on the BUS and anyone has his own memory mapped I/O 
addresses;
- if no devices own the address that the CPU ask for the request go to real 
memory.

Curiosity: since the memory addresses from 640KB to 1MB are reserved for 
memory mapped I/O (video memory) and BIOS ROM... the corrispondent range in 
the REAL MEMORY isn't usable and so we lost 384KB of memory. Is this right?

And what about the I/O address space (IN, OUT x86 instructions)?
I know this is a separate address space but...
- how are the addresses assigned? In a static way?
- for these instructions the CPU uses the same address (and data) lines that 
when it try to acces to memory, how does this work?

2) Compatibilty issues:
for example something about A20 line... is it possible that I found 
information about this only in unofficial docs?
I didn't know about it before. A day I've found on a web page something like 
"enabling A20 line" and I've said: "A20 line? What is it?" ;-)

...

Must I search info for any "piece of the puzzle"? This is OK for me but I want 
to know AT LEAST which are the pieces!


Bye,
	Paolo

Re: [OT] Documentation for PC Architecture

[Nagendra Singh Tomar]

Jamie,
	What is this 2nd address translation you are mentioning. 
I always thought that for the sake of cleanliness we just forget about the 
384K of memory starting from 640K. RAM anyway is cheap.
Pl correct me if I'm missing something.

Thanx
tomar

On Tue, 19 Aug 2003, Jamie Lokier wrote:

> Herbert Pötzl wrote:
> > On Mon, Aug 18, 2003 at 11:11:53PM +0200, Paolo Ornati wrote:
> > > >
> > > > > Curiosity: since the memory addresses from 640KB to 1MB are
> reserved for
> > > > > memory mapped I/O (video memory) and BIOS ROM... the
> corrispondent range
> > > > > in
> > > >
> > > > uh oh ...
> > > >
> > > > > the REAL MEMORY isn't usable and so we lost 384KB of memory. Is
> this
> > > > > right?
> > > >
> > > > for DOS, withouth upper memory manager yes ;)
> > > 
> > > I'm talking about an OS in protected mode... in any case how 
> > > can I access to this memory region if it's mapped for other things?
> > 
> > this is usually done via the MMU which can do miracolous
> > things to memory and addresses ...
> > 
> > > I've done some tests with a simple kernel which I wrote: all that
> region 
> > > (except video memory at 0xb8000) results "read only"...
> > 
> > because it is usually designated as rom area, which naturally
> > is read only ... 
> > 
> > > So I THINK YOU mean: "you can use more than 640KB in real mode using
> 
> > > a memory manager that "remap" 0xC0000 (for example) to 0x100000 or 
> > > something like it"
> > 
> > basically a memory manager (depending on the processors 
> > capabilities) does either use memory mapping or simple
> > memory exchanging between designated regions and above
> > 1M+64k memory ...
> 
> The MMU _is_ used to remap memory addresses.  It is part of the CPU
> itself.
> But it translates what's called "virtual address" space to "physical
> address space".  Physical addresses seemingly map directly to RAM and
> memory-mapped I/O.
> 
> Paolo's question is, what happens to the 384k of _physical_ addresses
> starting at 0xa0000, which should correspond with 384k of actual
> physical RAM?
> 
> If you use the MMU to map a virtual address to the physical addresses
> from
> 0xa0000..0xfffff, then whichever virtual addresses you chose will map to
> video memory, ROM, BIOS etc.
> 
> The answer is that after the MMU has translated, a _second_ address
> translation takes place, outside the CPU, which maps the physical
> addresses
> so that a hole is created in the RAM without any RAM going missing.
> This
> second translation is done by the motherboard chipset.
> 
> Enjyo,
> -- Jamie
> 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel"
> in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
> 

Re: [OT] Documentation for PC Architecture

[Herbert Pötzl]

On Mon, Aug 18, 2003 at 11:27:43AM +0200, Paolo Ornati wrote:
> Hi,
> (sorry for the OT but I think here someone will relpy :-)
> 
> Problem: I'm interested on PC architecture and looking for documentation I've 
> found al lot of good doc for the CPU (Intel manuals...) but about the basic 
> hardware that surround it I've found only a lot of "pieces of a puzzle"!
> 
> IOW: there is an (un)official document spoking about general PC architecture?
> 
> In this document I want to find some basic things about PC hardware and 
> especially about these themes (from the logical point of view):
> 
> 1) communication between CPU, memory, I/O devices:
> Example:
> If I write byte 'A' to physical address 0xb8000 how does this go to video 
> memory instead of RAM?
> I know about memory mapped I/O but I don't know who "decides" WHAT must go 
> WHERE.
> Since the CPU can't know if it's writing to memory or some memory mapped I/O 
> device I suppose that is the "BUS CONTROLLER" (PCI BRIDGE in case of a PCI 
> BUS) to write the byte 'A' to the right device.
> But how does the "BUS CONTROLLER" do this association?
> For now I think in this simple way:
> - all the devices are on the BUS and anyone has his own memory mapped I/O 
> addresses;
> - if no devices own the address that the CPU ask for the request go to real 
> memory.

google for 'IBM' 'PC' 'architecture' ...
and variants of 'Upper Memory' 'Shadow' 'VGA' ...

(for example ...)
http://www.pcguide.com/ref/ram/logicOverview-c.html
http://www.infokomp.no/techinfo/doc/DosMemory.htm
http://world.std.com/~swmcd/steven/rants/pc.html

> Curiosity: since the memory addresses from 640KB to 1MB are reserved for 
> memory mapped I/O (video memory) and BIOS ROM... the corrispondent range in 

uh oh ...

> the REAL MEMORY isn't usable and so we lost 384KB of memory. Is this right?

for DOS, withouth upper memory manager yes ;)

> And what about the I/O address space (IN, OUT x86 instructions)?
> I know this is a separate address space but...
> - how are the addresses assigned? In a static way?
> - for these instructions the CPU uses the same address (and data) lines that 
> when it try to acces to memory, how does this work?

there is a line (signal) telling if it is memory or I/O ...

> 2) Compatibilty issues:
> for example something about A20 line... is it possible that I found 
> information about this only in unofficial docs?
> I didn't know about it before. A day I've found on a web page something like 
> "enabling A20 line" and I've said: "A20 line? What is it?" ;-)
> 
> ...
> 
> Must I search info for any "piece of the puzzle"? This is OK for me but I want 
> to know AT LEAST which are the pieces!

HTH,
Herbert

> 
> 
> Bye,
> 	Paolo
> 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/

Re: [OT] Documentation for PC Architecture

[Paolo Ornati]

>
> > Curiosity: since the memory addresses from 640KB to 1MB are reserved for
> > memory mapped I/O (video memory) and BIOS ROM... the corrispondent range
> > in
>
> uh oh ...
>
> > the REAL MEMORY isn't usable and so we lost 384KB of memory. Is this
> > right?
>
> for DOS, withouth upper memory manager yes ;)

I'm talking about an OS in protected mode... in any case how can I access to 
this memory region if it's mapped for other things?

I've done some tests with a simple kernel which I wrote: all that region 
(except video memory at 0xb8000) results "read only"...

So I THINK YOU mean: "you can use more than 640KB in real mode using a memory 
manager that "remap" 0xC0000 (for example) to 0x100000 or something like it"

Right?

bye,
	Paolo

Re: [OT] Documentation for PC Architecture

[Herbert Pötzl]

On Mon, Aug 18, 2003 at 11:11:53PM +0200, Paolo Ornati wrote:
> >
> > > Curiosity: since the memory addresses from 640KB to 1MB are reserved for
> > > memory mapped I/O (video memory) and BIOS ROM... the corrispondent range
> > > in
> >
> > uh oh ...
> >
> > > the REAL MEMORY isn't usable and so we lost 384KB of memory. Is this
> > > right?
> >
> > for DOS, withouth upper memory manager yes ;)
> 
> I'm talking about an OS in protected mode... in any case how 
> can I access to this memory region if it's mapped for other things?

this is usually done via the MMU which can do miracolous
things to memory and addresses ...

> I've done some tests with a simple kernel which I wrote: all that region 
> (except video memory at 0xb8000) results "read only"...

because it is usually designated as rom area, which naturally
is read only ... 

> So I THINK YOU mean: "you can use more than 640KB in real mode using 
> a memory manager that "remap" 0xC0000 (for example) to 0x100000 or 
> something like it"

basically a memory manager (depending on the processors 
capabilities) does either use memory mapping or simple
memory exchanging between designated regions and above
1M+64k memory ...

given an apropriate address mapping (via mmu) you should
be able to access the entire memory (not just the first
megabyte) without disturbing any memory mapped hardware ...

how could this be possible? simple there are 32 address
lines on a full 386 and above, which gives you 4GB of
physical memory addresses, now the memory mapped hardware
(pci,agp, but also isa) is only activated if the address
lies within a defined range. memory can be designated to
a different range, like (ep)roms ...

for example the Intel 386EX allows to define signals for
each configured memory range (chip selects) to enable or
disable the devices (memory) on demand, other systems
use some chipset/onboard logic to do this ...

> Right?
> 
> bye,
> 	Paolo
> 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/

Re: [OT] Documentation for PC Architecture

[Jamie Lokier]

Paolo Ornati wrote:
> So I THINK YOU mean: "you can use more than 640KB in real mode using a memory 
> manager that "remap" 0xC0000 (for example) to 0x100000 or something like it"

As with all things PC, you cannot easily tell what RAM address it is
mapped to, but yes, it is remapped somewhere by the memory controller
so that it isn't wasted.

Perhaps on machines with lots of RAM this isn't done any more.

Also, these days there are other regions of RAM which you cannot see,
because they are only available to the BIOS, when the CPU is in SMM
mode :)

Your larger question, about how addresses are matched by the hardware,
is a complex one and it is different on different PC systems.  That is
some 20 years of computing history; it has changed a lot even though
many of the old programming methods continue to be supported.

Your best bet is to learn about PCI, PCI configuration space, PCI
transactions, PCI bridges and additive/subtractive decoding,
northbridge and southbridge chipsets (just what they are), PCI to ISA
bridge, legacy I/O controller (what provides most of the ISA space I/O
ports), AGP and GART, MTRR registers, and get some idea of how the
L1/L2/L3 cache hierarchy works.

And of course read all the big x86 manuals from developer.intel.com :-)

Your other large question, How does a PC work?, is huge because there
is much variety.  For example, on every other architecture in the
world, interrupts are quite simple, easy to program and easy to
understand.  On a modern PC they are extraordinarily complicated, to
the point where even the top kernel developers are still changing the
code, year on year, to workaround new surprises in different PCs.  As
you see, there is no readily available answer to your question
that is definitive for all the PCs out there.  Unfortunately.

-- Jamie

Re: [OT] Documentation for PC Architecture

[Jamie Lokier]

Herbert Pötzl wrote:
> On Mon, Aug 18, 2003 at 11:11:53PM +0200, Paolo Ornati wrote:
> > >
> > > > Curiosity: since the memory addresses from 640KB to 1MB are reserved for
> > > > memory mapped I/O (video memory) and BIOS ROM... the corrispondent range
> > > > in
> > >
> > > uh oh ...
> > >
> > > > the REAL MEMORY isn't usable and so we lost 384KB of memory. Is this
> > > > right?
> > >
> > > for DOS, withouth upper memory manager yes ;)
> > 
> > I'm talking about an OS in protected mode... in any case how 
> > can I access to this memory region if it's mapped for other things?
> 
> this is usually done via the MMU which can do miracolous
> things to memory and addresses ...
> 
> > I've done some tests with a simple kernel which I wrote: all that region 
> > (except video memory at 0xb8000) results "read only"...
> 
> because it is usually designated as rom area, which naturally
> is read only ... 
> 
> > So I THINK YOU mean: "you can use more than 640KB in real mode using 
> > a memory manager that "remap" 0xC0000 (for example) to 0x100000 or 
> > something like it"
> 
> basically a memory manager (depending on the processors 
> capabilities) does either use memory mapping or simple
> memory exchanging between designated regions and above
> 1M+64k memory ...

The MMU _is_ used to remap memory addresses.  It is part of the CPU itself.
But it translates what's called "virtual address" space to "physical
address space".  Physical addresses seemingly map directly to RAM and
memory-mapped I/O.

Paolo's question is, what happens to the 384k of _physical_ addresses
starting at 0xa0000, which should correspond with 384k of actual
physical RAM?

If you use the MMU to map a virtual address to the physical addresses from
0xa0000..0xfffff, then whichever virtual addresses you chose will map to
video memory, ROM, BIOS etc.

The answer is that after the MMU has translated, a _second_ address
translation takes place, outside the CPU, which maps the physical addresses
so that a hole is created in the RAM without any RAM going missing.  This
second translation is done by the motherboard chipset.

Enjyo,
-- Jamie

Re: [OT] Documentation for PC Architecture

[Paolo Ornati]

On Tuesday 19 August 2003 03:02, Jamie Lokier wrote:
>
> The MMU _is_ used to remap memory addresses.  It is part of the CPU itself.
> But it translates what's called "virtual address" space to "physical
> address space".  Physical addresses seemingly map directly to RAM and
> memory-mapped I/O.

I know...

>
> Paolo's question is, what happens to the 384k of _physical_ addresses
> starting at 0xa0000, which should correspond with 384k of actual
> physical RAM?

It seems that only you have understand my question! :-)

>
> If you use the MMU to map a virtual address to the physical addresses from
> 0xa0000..0xfffff, then whichever virtual addresses you chose will map to
> video memory, ROM, BIOS etc.
>
> The answer is that after the MMU has translated, a _second_ address
> translation takes place, outside the CPU, which maps the physical addresses
> so that a hole is created in the RAM without any RAM going missing.  This
> second translation is done by the motherboard chipset.

OK.

>
> Enjyo,
> -- Jamie


VERY [OT]:

Why do I do all these questions?
At present I'm working on a very small kernel (PabloX :):
- it's very simple: it only uses segmentation and has drivers only for stupid 
things like AT-PS/2 keyboard (do you have USB keyboard? I'm sorry!)
- some (a lot of) code is taken from linux 0.01 / 1.0 / ... 

If anyone want to see this stupid thing:
http://members.xoom.virgilio.it/javaman/

NOTE: don't read the comments in source code! They are a mix of pseudo-English 
and Italian!

To try it:
tar xzf pablox.tar.gx
cd pablox/
make

now you should have a floppy image in ./IMAGE, I suggest insert a floppy and 
do "make disk", than reboot and see keyboard leds! (if you have an AT or PS/2 
keyboard)

press (1,2,3,4) to switch to console (1,2,3,4): the consoles are stupid! All 
the programs write to the current console...
press "ESC" to reboot!


Bye,
	Paolo

Re: [OT] Documentation for PC Architecture

[Krzysztof Halasa]

Jamie Lokier <jamie@shareable.org> writes:

> Paolo's question is, what happens to the 384k of _physical_ addresses
> starting at 0xa0000, which should correspond with 384k of actual
> physical RAM?

Part of it is used for BIOS (shadow of EPROM/flash EPROM - usually
written by BIOS executing from EPROM and then made read-only by
programming chipset registers). This involves main motherboard BIOS
(usually 64 KB at 0xF000-0xFFFF) and any extension BIOSes (VGA, LAN
etc) in 0xC000-0xEFFF (or 0xC000-0xDFFF) range.

The remaining RAM from 0xA000-0xFFFF is unusable. One could program
chipset registers to enable parts of this RAM. It would require
knowledge of particular chipset registers, and there might be only
one read-only/read-write bit for all shadows (so, unless you want the
shadow BIOS to be R/W, the additional RAM would have to be R/O).
RAM is quite cheap and these things are very non-standard so I think
nobody bothers.

And yes, older machines (older chipsets) used to remap fragments of
this memory to the top of RAM. It was common on 80286 machines and
I probably haven't seen a 386DX or faster board doing that.
-- 
Krzysztof Halasa
Network Administrator

Re: [OT] Documentation for PC Architecture

[Jamie Lokier]

Nagendra Singh Tomar wrote:
> 	What is this 2nd address translation you are mentioning. 
> I always thought that for the sake of cleanliness we just forget about the 
> 384K of memory starting from 640K. RAM anyway is cheap.
> Pl correct me if I'm missing something.

That is probably true of modern machines.  It is hard to tell from the
e820 map on two machines I just looked at, because other bits of RAM
are missing, more than 384k.

The original question asked about "the PC architecture", and I can say
for sure that a PC with 1MB does not just forget about 384k of it :)

-- Jamie

Re: [OT] Documentation for PC Architecture

[Maciej W. Rozycki]

On Tue, 19 Aug 2003, John Bradford wrote:

> On some boards I've seen, there is 384K onboard for ROM shadowing
> purposes, and when only 128K is actually used, (as it is in a lot of
> configurations), the other 256K is available as system memory.
> 
> However, this on-board 256K is only remapped when you have 8 MB RAM or
> less on the board.  So with 8 MB the board reports 8448K of RAM, but
> with 16 MB, it only reports 16384K.  In that case 256K of real RAM is,
> indeed, lost.

 Since the system firmware of a typical PC these days is compressed in the
ROM, part of the RAM in the upper 384kB of the first MB of physical
address space is used to hold an uncompressed image of the firmware --
typically 128kB.  The preceding 128kB is available for shadowing firmware
of option boards and the remaining 128kB is used for SMM code or not at
all.  Remapping of these 384kB of RAM in the chipset for general use is
possible in some systems, but I haven't seen this option in any system
newer than ones from the 486 era. 

-- 
+  Maciej W. Rozycki, Technical University of Gdansk, Poland   +
+--------------------------------------------------------------+
+        e-mail: macro@ds2.pg.gda.pl, PGP key available        +

Re: [OT] Documentation for PC Architecture

[John Bradford]

> > I've done some tests with a simple kernel which I wrote: all that region 
> > (except video memory at 0xb8000) results "read only"...
>
> because it is usually designated as rom area, which naturally
> is read only ... 

On some boards I've seen, there is 384K onboard for ROM shadowing
purposes, and when only 128K is actually used, (as it is in a lot of
configurations), the other 256K is available as system memory.

However, this on-board 256K is only remapped when you have 8 MB RAM or
less on the board.  So with 8 MB the board reports 8448K of RAM, but
with 16 MB, it only reports 16384K.  In that case 256K of real RAM is,
indeed, lost.

John.

Re: [OT] Documentation for PC Architecture

[Paolo Ornati]

On Monday 18 August 2003 12:14, John Bradford wrote:
> > 2) Compatibilty issues:
> > for example something about A20 line... is it possible that I found
> > information about this only in unofficial docs?
> > I didn't know about it before. A day I've found on a web page something
> > like "enabling A20 line" and I've said: "A20 line? What is it?" ;-)
>
> http://www.win.tue.nl/~aeb/linux/kbd/A20.html
>
> John

thank you... but I've writed:
(and I've said: "A20 line? What is it?")
        ^^^^^
So I already know about it!

In any case it's only a "piece of the puzzle".

Bye,
	Paolo

Re: [OT] Documentation for PC Architecture

[John Bradford]

> 2) Compatibilty issues:
> for example something about A20 line... is it possible that I found 
> information about this only in unofficial docs?
> I didn't know about it before. A day I've found on a web page something like 
> "enabling A20 line" and I've said: "A20 line? What is it?" ;-)

http://www.win.tue.nl/~aeb/linux/kbd/A20.html

John