[Q] Question about memory access

[Q] Question about memory access

[Cho, joon-woo]

Hello.

I am highly curious about memory access problem.

Maybe this is not exactly 'kernel' development problem, but I think this is
highly

related to 'kernel'.


Anyway my question is below.

If someone want to transfer large data from some device to memory, he may
use DMA method.

At this point, i am confused.

I think that only one process can access physical memory(RAM) at a time.

During DMA data transferring(This may need  long time), how can other
physical memory access occurs?

I think that this is solved by time-sharing, process switching, or etc. Is
it true?

Please say abstract mechanism or kernel code location about this problem.


My english is poor, so question meaning may be confused. I am sorry about
this.

Please answer my question. Thanks!

Re: [Q] Question about memory access

[Gianni Tedesco]

On Mon, 2003-08-04 at 08:14, Cho, joon-woo wrote:
> If someone want to transfer large data from some device to memory, he may
> use DMA method.
> 
> At this point, i am confused.
> 
> I think that only one process can access physical memory(RAM) at a time.

The DMA controller is a dedicated piece of hardware that copies the data
from devices to RAM. This means that other processes can use the CPU
while the DMA is in progress. That is the whole point of DMA.

-- 
// Gianni Tedesco (gianni at scaramanga dot co dot uk)
lynx --source www.scaramanga.co.uk/gianni-at-ecsc.asc | gpg --import
8646BE7D: 6D9F 2287 870E A2C9 8F60 3A3C 91B5 7669 8646 BE7D

Re: [Q] Question about memory access

[Antonio Vargas]

On Mon, Aug 04, 2003 at 10:35:25AM +0100, Gianni Tedesco wrote:
> On Mon, 2003-08-04 at 08:14, Cho, joon-woo wrote:
> > If someone want to transfer large data from some device to memory, he may
> > use DMA method.
> > 
> > At this point, i am confused.
> > 
> > I think that only one process can access physical memory(RAM) at a time.
> 
> The DMA controller is a dedicated piece of hardware that copies the data
> from devices to RAM. This means that other processes can use the CPU
> while the DMA is in progress. That is the whole point of DMA.

Yes, this is called having 2 bus masters, which are the chips that
can use the bus to read and write to memory. What can be done is to
timeshare the bus, for example the cpu accesses memory on odd cycles
and the dma chip does on even cycles.

A more complex design would allow the cpu to access memory on all
cycles, but give the dma chip more priority. This would mean that
a dma transfer would take priority over the cpu. Think about
a sound card reading the sound data to pump it to the speakers,
you would prefer not to have it skip.

Greets, Antonio.

-- 

1. Dado un programa, siempre tiene al menos un fallo.
2. Dadas varias lineas de codigo, siempre se pueden acortar a menos lineas.
3. Por induccion, todos los programas se pueden
   reducir a una linea que no funciona.

Re: [Q] Question about memory access

[jw schultz]

On Mon, Aug 04, 2003 at 06:18:46PM +0200, Antonio Vargas wrote:
> On Mon, Aug 04, 2003 at 10:35:25AM +0100, Gianni Tedesco wrote:
> > On Mon, 2003-08-04 at 08:14, Cho, joon-woo wrote:
> > > If someone want to transfer large data from some device to memory, he may
> > > use DMA method.
> > > 
> > > At this point, i am confused.
> > > 
> > > I think that only one process can access physical memory(RAM) at a time.
> > 
> > The DMA controller is a dedicated piece of hardware that copies the data
> > from devices to RAM. This means that other processes can use the CPU
> > while the DMA is in progress. That is the whole point of DMA.
> 
> Yes, this is called having 2 bus masters, which are the chips that
> can use the bus to read and write to memory. What can be done is to
> timeshare the bus, for example the cpu accesses memory on odd cycles
> and the dma chip does on even cycles.
> 
> A more complex design would allow the cpu to access memory on all
> cycles, but give the dma chip more priority. This would mean that
> a dma transfer would take priority over the cpu. Think about
> a sound card reading the sound data to pump it to the speakers,
> you would prefer not to have it skip.

A few of things to ponder:

	Multiple CPUs may share the same memory.

	CPUs mostly access memory in burst mode to fill
	cache lines (usually 32 bytes) so once the
	memory begins transfer it can pump at buss speed.

	CPU-to-memory buss is clocked at speeds of
	100MHz and above.  Most newer Intel boxes have the
	memory bus clocked at 133MHz and may use DDR which
	allows transfers at double the clock rate.  That is
	~1064BG/s (less access initialisation time)

	Devices on the PCI buss are limited to the PCI bus
	speed which is typically 33 or 66 MHz.  This limits
	DMA transfers to about 120 or 240 MB/s.  So Even if
	you saturate the PCI buss there are still a lot of
	memory buss cycles left.

	Few devices are capable of saturating the PCI buss.
	In theory a medium to large RAID array could do it.
	Combining multiple Gbit NICs and RAID arrays would
	do it in practice under the right kind of loads.

	Access to the memory buss is arbitrated by a bridge
	chipset (sometimes called "North Bridge") between
	CPU, memory, AGP and other bridges.  The PCI buss
	bridge may packetise (for lack of a better word) a
	DMA transfer into multiple short bursts which
	will be fed to the North Bridge at memory buss
	speeds.

The days of interleaving DMA with CPU at the cycle level or
stalling the CPU or other device are long gone.  It is
pretty much all done in bursts mediated by a bridge
functioning as a memory controller.


-- 
________________________________________________________________
	J.W. Schultz            Pegasystems Technologies
	email address:		jw@pegasys.ws

		Remember Cernan and Schmitt

Re: [Q] Question about memory access

[Antonio Vargas]

On Mon, Aug 04, 2003 at 05:13:04PM -0700, jw schultz wrote:
> On Mon, Aug 04, 2003 at 06:18:46PM +0200, Antonio Vargas wrote:
> > On Mon, Aug 04, 2003 at 10:35:25AM +0100, Gianni Tedesco wrote:
> > > On Mon, 2003-08-04 at 08:14, Cho, joon-woo wrote:
> > > > If someone want to transfer large data from some device to memory, he may
> > > > use DMA method.
> > > > 
> > > > At this point, i am confused.
> > > > 
> > > > I think that only one process can access physical memory(RAM) at a time.
> > > 
> > > The DMA controller is a dedicated piece of hardware that copies the data
> > > from devices to RAM. This means that other processes can use the CPU
> > > while the DMA is in progress. That is the whole point of DMA.
> > 
> > Yes, this is called having 2 bus masters, which are the chips that
> > can use the bus to read and write to memory. What can be done is to
> > timeshare the bus, for example the cpu accesses memory on odd cycles
> > and the dma chip does on even cycles.
> > 
> > A more complex design would allow the cpu to access memory on all
> > cycles, but give the dma chip more priority. This would mean that
> > a dma transfer would take priority over the cpu. Think about
> > a sound card reading the sound data to pump it to the speakers,
> > you would prefer not to have it skip.
> 
> A few of things to ponder:
> 
> 	Multiple CPUs may share the same memory.
> 
> 	CPUs mostly access memory in burst mode to fill
> 	cache lines (usually 32 bytes) so once the
> 	memory begins transfer it can pump at buss speed.
> 
> 	CPU-to-memory buss is clocked at speeds of
> 	100MHz and above.  Most newer Intel boxes have the
> 	memory bus clocked at 133MHz and may use DDR which
> 	allows transfers at double the clock rate.  That is
> 	~1064BG/s (less access initialisation time)
> 
> 	Devices on the PCI buss are limited to the PCI bus
> 	speed which is typically 33 or 66 MHz.  This limits
> 	DMA transfers to about 120 or 240 MB/s.  So Even if
> 	you saturate the PCI buss there are still a lot of
> 	memory buss cycles left.
> 
> 	Few devices are capable of saturating the PCI buss.
> 	In theory a medium to large RAID array could do it.
> 	Combining multiple Gbit NICs and RAID arrays would
> 	do it in practice under the right kind of loads.
> 
> 	Access to the memory buss is arbitrated by a bridge
> 	chipset (sometimes called "North Bridge") between
> 	CPU, memory, AGP and other bridges.  The PCI buss
> 	bridge may packetise (for lack of a better word) a
> 	DMA transfer into multiple short bursts which
> 	will be fed to the North Bridge at memory buss
> 	speeds.
> 
> The days of interleaving DMA with CPU at the cycle level or
> stalling the CPU or other device are long gone.  It is
> pretty much all done in bursts mediated by a bridge
> functioning as a memory controller.

Now I understand the fuss about northbridge speeds
on motherboard reviews :)

Anyways, if the memory is not connected directly to the cpu but
to an intermediate controller, this controller is in fact stalling
the cpu memory access when he prefers to give this memory cycle
to the pci devices. The fact that the communication between, say,
cpu and memory controller can ask for burst read/write doesn't
mean that in a heavy loaded system such as an SMP, some accesses
will be stalled because another device needs it earlier.

My background on DMA programming come from Amiga and Gameboy systems,
this may explain this "long gone"-style example.

Greets, Antonio.

-- 

1. Dado un programa, siempre tiene al menos un fallo.
2. Dadas varias lineas de codigo, siempre se pueden acortar a menos lineas.
3. Por induccion, todos los programas se pueden
   reducir a una linea que no funciona.