Re: Prefered method to map PCI memory into userspace.

Re: Prefered method to map PCI memory into userspace.

[Andi Kleen]

Jim Deas <jdeas@jadsystems.com> writes:

> I am looking for the most current (blessed) structure
> for mapping PCI memory to a user process. One that allows
> both PIO and busmastering to work on a common block of
> PCI RAM. I am not concerned with backporting to older
> kernels but it would be nice if the solution wasn't ibm specific.

mmap on /dev/mem

PIO cannot be done on memory, for that you have to use iopl() 
or ioperm() to get access to the port and then issue the PIO 
instructions yourself

The only trap with the mapping is getting uncached mappings
(most PCI hardware prefers uncached accesses). When the mapping
is beyond the end of memory the kernel will automatically 
map it uncached. PCI mappings are normally in the PCI hole
at the end of the 4GB area. When you machine has more than
4GB of ram this heuristic does not work and you have 
set MTRRs by hand using /proc/mtrr or use O_SYNC. The later is 
cleanest, but only works in newer kernels.

[BTW I consider this a kernel bug - it should always map mappings
to non memory uncached] 


-Andi

Re: Prefered method to map PCI memory into userspace.

[David S. Miller]

On 27 Sep 2003 00:03:42 +0200
Andi Kleen <ak@suse.de> wrote:

> mmap on /dev/mem
> 
> PIO cannot be done on memory, for that you have to use iopl() 
> or ioperm() to get access to the port and then issue the PIO 
> instructions yourself

Platform dependant, mmap on /dev/mem doesn't work on many systems.

Re: Prefered method to map PCI memory into userspace.

[Andi Kleen]

"David S. Miller" <davem@redhat.com> writes:

> If someone is looking for something to do, it would be incredibly
> value to make it so that all arches support this, and in particular
> get right the case of mmap()'ing the PCI host bridge.
> 
> If this were done, the PCI domain code in xfree86 could be enabled
> for all Linux platforms, not just the ones that have this implemented
> properly.

What semantics would it have? Can the "seek offset" just be the 
bus address?

If yes then it would be trivial to implement for x86/x86-64

-andi

Re: Prefered method to map PCI memory into userspace.

[David S. Miller]

On 27 Sep 2003 07:09:01 +0200
Andi Kleen <ak@suse.de> wrote:

> "David S. Miller" <davem@redhat.com> writes:
> 
> > If someone is looking for something to do, it would be incredibly
> > value to make it so that all arches support this, and in particular
> > get right the case of mmap()'ing the PCI host bridge.
> > 
> > If this were done, the PCI domain code in xfree86 could be enabled
> > for all Linux platforms, not just the ones that have this implemented
> > properly.
> 
> What semantics would it have? Can the "seek offset" just be the 
> bus address?
> 
> If yes then it would be trivial to implement for x86/x86-64

I did everything for x86 already, egrep for "PCI_MMAP" under
arch/i386/.

The uncompleted part is the PCI host bridge mmap() support, I just
never got around to it.  Someone can easily complete it.

Re: Prefered method to map PCI memory into userspace.

[Andi Kleen]

"David S. Miller" <davem@redhat.com> writes:

> On 27 Sep 2003 00:03:42 +0200
> Andi Kleen <ak@suse.de> wrote:
>
>> mmap on /dev/mem
>> 
>> PIO cannot be done on memory, for that you have to use iopl() 
>> or ioperm() to get access to the port and then issue the PIO 
>> instructions yourself
>
> Platform dependant, mmap on /dev/mem doesn't work on many systems.

Just curious - what does the X server use on these many systems then ?

-Andi

Re: Prefered method to map PCI memory into userspace.

[Anton Blanchard]

> Just curious - what does the X server use on these many systems then ?

FYI ppc64 and some ppc32 systems fall into the cant use /dev/mem
category. The answer is to use pci domains (ie using /proc/bus/pci/...
to be able to mmap PCI memory and IO regions)

Anton

Re: Prefered method to map PCI memory into userspace.

[David S. Miller]

On Fri, 26 Sep 2003 23:59:08 +1000
Anton Blanchard <anton@samba.org> wrote:

> 
> > Just curious - what does the X server use on these many systems then ?
> 
> FYI ppc64 and some ppc32 systems fall into the cant use /dev/mem
> category. The answer is to use pci domains (ie using /proc/bus/pci/...
> to be able to mmap PCI memory and IO regions)

Exactly.

XFREE86-4.3.0 and later has full domain infrastructure, it just isn't
enabled on anything other than ppc and sparc because the other platforms
haven't made their /proc/bus/pci/* mmap() arch support routines fully
functional yet.

Prefered method to map PCI memory into userspace.

[Jim Deas]

I am looking for the most current (blessed) structure
for mapping PCI memory to a user process. One that allows
both PIO and busmastering to work on a common block of
PCI RAM. I am not concerned with backporting to older
kernels but it would be nice if the solution wasn't ibm specific.

My problem is a 64M window into a frame buffer that I would
like to map into user space. I am more than willing to put
forth the effort, I just want to make sure I'm heading in
the right direction.

Is there a better forum for posting this? Regards,
J. Deas

RH9.0 2.4.20-6smp kernel and above.


-- 
Jim Deas <jdeas@jadsystems.com>
JAD Systems

Re: Prefered method to map PCI memory into userspace.

[Joe Korty]

On Fri, Sep 26, 2003 at 01:53:43PM -0700, Jim Deas wrote:
> I am looking for the most current (blessed) structure
> for mapping PCI memory to a user process. One that allows
> both PIO and busmastering to work on a common block of
> PCI RAM. I am not concerned with backporting to older
> kernels but it would be nice if the solution wasn't ibm specific.
> 
> My problem is a 64M window into a frame buffer that I would
> like to map into user space. I am more than willing to put
> forth the effort, I just want to make sure I'm heading in
> the right direction.
> 
> Is there a better forum for posting this? Regards,
> J. Deas
> 
> RH9.0 2.4.20-6smp kernel and above.

Albert Cahalan wrote a patch, for 2.6, that makes mmappable all PCI device
memory regions.  They show up as files in the appropriate subdirectories
under /proc/bus/pci.  See http://lkml.org/lkml/2003/7/13/258 for the
patch and details.

Joe

Re: Prefered method to map PCI memory into userspace.

[Benjamin Herrenschmidt]

> 
> Albert Cahalan wrote a patch, for 2.6, that makes mmappable all PCI device
> memory regions.  They show up as files in the appropriate subdirectories
> under /proc/bus/pci.  See http://lkml.org/lkml/2003/7/13/258 for the
> patch and details.

mmap of /proc/bus/pci is a standard thing in 2.4 already and so in 2.6
as well, though not all archs may implement it..

Regarding bus mastering, there's no way to do that without a kernel
driver.

Ben.

Re: Prefered method to map PCI memory into userspace.

[David S. Miller]

On Sat, 27 Sep 2003 02:40:10 +0200
Benjamin Herrenschmidt <benh@kernel.crashing.org> wrote:

> mmap of /proc/bus/pci is a standard thing in 2.4 already and so in 2.6
> as well, though not all archs may implement it..

If someone is looking for something to do, it would be incredibly
value to make it so that all arches support this, and in particular
get right the case of mmap()'ing the PCI host bridge.

If this were done, the PCI domain code in xfree86 could be enabled
for all Linux platforms, not just the ones that have this implemented
properly.