Re: Graphics pass-through

[Jan Kiszka <jan.kiszka@siemens.com>]

On 2011-05-09 16:55, Prasad Joshi wrote:
> On Mon, May 9, 2011 at 12:14 PM, Jan Kiszka <jan.kiszka@siemens.com> wrote:
>> On 2011-05-05 17:17, Alex Williamson wrote:
>>>> And what about the host? When does Linux release the legacy range?
>>>> Always or only when a specific (!=vga/vesa) framebuffer driver is loaded?
>>>
>>> Well, that's where it'd be nice if the vga arbiter was actually in more
>>> widespread use.  It currently seems to be nothing more than a shared
>>> mutex, but it would actually be useful if it included backends to do the
>>> chipset vga routing changes.  I think when I was testing this, I was
>>> externally poking PCI bridge chipset to toggle the VGA_EN bit.
>>
>> Right, we had to drop the approach to pass through the secondary card
>> for now, the arbiter was not switching properly. Haven't checked yet if
>> VGA_EN was properly set, though the kernel code looks like it should
>> take care of this.
>>
>> Even with handing out the primary adapter, we had only mixed success so
>> far. The onboard adapter worked well (in VESA mode), but the NVIDIA is
>> not displaying early boot messages at all. Maybe a vgabios issue.
>> Windows was booting nevertheless - until we installed the NVIDIA
>> drivers. Than it ran into a blue screen.
>>
>> BTW, what ATI adapter did you use precisely, and what did work, what not?
> 
> Not hijacking the mail thread. Just wanted to provide some inputs.

Much appreciated in fact!

> 
> Few days back I had tried passing through the secondary graphics card.
> I could pass-through two graphics cards to virtual machine.
> 
> 02:00.0 VGA compatible controller: ATI Technologies Inc Redwood
> [Radeon HD 5670] (prog-if 00 [VGA controller])
> 	Subsystem: PC Partner Limited Device e151
> 	Flags: bus master, fast devsel, latency 0, IRQ 87
> 	Memory at d0000000 (64-bit, prefetchable) [size=256M]
> 	Memory at fe6e0000 (64-bit, non-prefetchable) [size=128K]
> 	I/O ports at b000 [size=256]
> 	Expansion ROM at fe6c0000 [disabled] [size=128K]
> 	Capabilities: <access denied>
> 	Kernel driver in use: radeon
> 	Kernel modules: radeon
> 
> 07:00.0 VGA compatible controller: nVidia Corporation G86 [Quadro NVS
> 290] (rev a1) (prog-if 00 [VGA controller])
>        Subsystem: nVidia Corporation Device 0492
>        Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop-
> ParErr-Stepping- SERR+ FastB2B- DisINTx-
>        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast
>> TAbort-<TAbort- <MAbort- >SERR- <PERR- INTx-
>        Latency: 0, Cache Line Size: 64 bytes
>        Interrupt: pin A routed to IRQ 24
>        Region 0: Memory at fd000000 (32-bit, non-prefetchable) [size=16M]
>        Region 1: Memory at d0000000 (64-bit, prefetchable) [size=256M]
>        Region 3: Memory at fa000000 (64-bit, non-prefetchable) [size=32M]
>        Region 5: I/O ports at ec00 [size=128]
>        Expansion ROM at fe9e0000 [disabled] [size=128K]
>        Capabilities: <access denied>
>        Kernel driver in use: nouveau
>        Kernel modules: nouveau, nvidiafb
> 
> Both of them are PCIe cards. I have one more ATI card and another
> NVIDIA card which does not work.

Interesting. That may rule out missing PCIe capabilities as source for
the NVIDIA driver indisposition.

Did you passed those cards each as primary to the guest, or was the
guest seeing multiple adapters? I presume you only got output after
early boot was completed, right?

To avoid having to deal with legacy I/O forwarding, we started with a
dual adapter setup in the hope to leave the primary guest adapter at
know-to-work cirrus-vga. But already in a native setup with on-board
primary + NVIDIA secondary, the NVIDIA Windows drivers refused to talk
to its hardware in this constellation.

> 
> One of the reason the pass-through did not work is because of the
> limit on amount of pci configuration memory by SeaBIOS. SeaBIOS places
> a hard limit of 256MB or so on the amount of PCI memory space. Thus,
> for some of the VGA device that need more memory never worked for me.
> 
> SeaBIOS allows this memory region to be extended to some value near
> 512MB, but even then the range is not enough.
> 
> Another problem with SeaBIOS which limits the amount of memory space
> is: SeaBIOS allocates the BAR regions as they are encountered. As far
> as I know, the BAR regions should be naturally aligned. Thus the
> simple strategy of the SeaBIOS results in large fragmentation.
> Therefore, even after increasing the PCI memory space to 512MB the BAR
> regions were unallocated.

That's an interesting trace! We'll check this here, but I bet it
contributes to the problems. Our FX 3800 has 1G memory...

> 
> I will confirm you the details of other graphics cards which do not work.

TiA,
Jan

-- 
Siemens AG, Corporate Technology, CT T DE IT 1
Corporate Competence Center Embedded Linux