* Device pass-through
@ 2012-01-02 19:41 Gordon Messmer
2012-01-03 21:34 ` Alex Williamson
0 siblings, 1 reply; 11+ messages in thread
From: Gordon Messmer @ 2012-01-02 19:41 UTC (permalink / raw)
To: kvm
I'm going to summarize my questions first and write the long explanation
afterward...
1: What happens when Linux rejects the aperture allocated by the BIOS
for an AMD IOMMU?
2: What is unsafe about the allow_unsafe_assigned_interrupts option?
What are the potential consequences of its use?
3: How can I get the ROM from a graphics card?
4: What do these errors mean, and is there any way I can correct the
problem?
> create_userspace_phys_mem: File exists
> assigned_dev_iomem_map: Error: create new mapping failed
My goal is to replace two machines that I previously used with one
machine and a KVM guest. I previously had one system that was always on
and ran a stable OS, and a second system for my desktop. I could afford
to experiment or format the desktop system without interrupting access
to my data, all of which is on the stable system.
I'd like to use KVM to accomplish something more or less equivalent by
creating a guest and passing the keyboard, mouse, audio device, and a
video card to it. To this end, I have a new AMD system with a Gigabyte
GA-970A-D3 motherboard and two Radeon video cards. Based on my reading,
I should probably be able to pass one of those to the guest as a PCI
device. It isn't in use by the host, and I'm not trying to make it the
guest's primary display, so I don't have to worry about early boot
output. The guest will still have a VNC console as its primary display.
I'm mostly there now. I can pass through the host's keyboard, mouse,
and audio device.
Naturally, there are a couple of challenges. First, the IOMMU on this
board is a little wonky. The kernel currently reports:
> kernel: Checking aperture...
> kernel: No AGP bridge found
> kernel: Node 0: aperture @ 20000000 size 32 MB
> kernel: Aperture pointing to e820 RAM. Ignoring.
> kernel: Your BIOS doesn't leave a aperture memory hole
> kernel: Please enable the IOMMU option in the BIOS setup
> kernel: This costs you 64 MB of RAM
> kernel: Mapping aperture over 65536 KB of RAM @ 20000000
> kernel: PM: Registered nosave memory: 0000000020000000 -
> 0000000024000000
> kernel: PCI-DMA: Using software bounce buffering for IO (SWIOTLB)
> kernel: Placing 64MB software IO TLB between ffff880037ff0000 -
> ffff88003bff0000
> kernel: software IO TLB at phys 0x37ff0000 - 0x3bff0000
additionally:
> kernel: BIOS-provided physical RAM map:
> kernel: BIOS-e820: 0000000000000000 - 0000000000098800 (usable)
> kernel: BIOS-e820: 000000000009f800 - 00000000000a0000 (reserved)
> kernel: BIOS-e820: 00000000000f0000 - 0000000000100000 (reserved)
> kernel: BIOS-e820: 0000000000100000 - 00000000bfda0000 (usable)
> kernel: BIOS-e820: 00000000bfda0000 - 00000000bfdd1000 (ACPI NVS)
> kernel: BIOS-e820: 00000000bfdd1000 - 00000000bfe00000 (ACPI data)
> kernel: BIOS-e820: 00000000bfe00000 - 00000000bff00000 (reserved)
> kernel: BIOS-e820: 00000000e0000000 - 00000000f0000000 (reserved)
> kernel: BIOS-e820: 00000000fec00000 - 0000000100000000 (reserved)
> kernel: BIOS-e820: 0000000100000000 - 0000000440000000 (usable)
> ...
> kernel: ACPI: IVRS 00000000bfddff00 000D0
> (v01 AMD RD890S 00202031 AMD 00000000)
Initially the kernel reported "AMD-Vi disabled by default: pass
amd_iommu=on to enable" After booting with that option, I seem to be
able to boot guests with devices passed through. That leads me to the
first question above. How bad is the state of the IOMMU on this board?
Where should the IVRS be mapped instead?
In addition to the amd_iommu boot option, I had to add:
options kvm allow_unsafe_assigned_interrupts=1
in order to pass through the audio device. That leads me to the second
question. What's unsafe about this?
When I try to boot the guest and pass in the video card, I get a warning
about the ROM:
> pci-assign: Cannot read from host /sys/bus/pci/devices/0000:05:00.0/rom
> Device option ROM contents are probably invalid (check dmesg).
> Skip option ROM probe with rombar=0, or load from file with
> romfile=
> create_userspace_phys_mem: File exists
> assigned_dev_iomem_map: Error: create new mapping failed
Third question, when I try to read that path, I get an IO error and the
kernel says "pci 0000:05:00.0: Invalid ROM contents". Is there any
other way to get the ROM from the card?
I've tried the other option. I added <rom bar='off'/> to the libvirt
definition for this guest. When I try to boot the guest with that
option, the kernel doesn't add anything to dmesg that looks relevant,
but the libvirt log says:
> LC_ALL=C PATH=/sbin:/usr/sbin:/bin:/usr/bin QEMU_AUDIO_DRV=none /usr/libexec/qemu-kvm -S -M rhel6.2.0 -enable-kvm -m 4096 -smp 4,sockets=4,cores=1,threads=1 -name herald -uuid 1929f6a4-b2dd-a4fd-fe07-747b7fdf9fae -nodefconfig -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/herald.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -drive file=/dev/mapper/VolGroup-lv_vm_herald,if=none,id=drive-virtio-disk0,format=raw,cache=none,aio=native -device virtio-blk-pci,bus=pci.0,addr=0x4,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -netdev tap,fd=25,id=hostnet0,vhost=on,vhostfd=26 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:86:e0:ea,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=
charserial0,id=serial0 -usb -device usb-tablet,id=input0 -vnc 127.0.0.1:0 -vga cirrus -device pci-assign,host=00:14.2,id=hostdev0,configfd=27,bus=pci.0,addr=0x5 -device pci-assign,host=05:00
.0,id=hostdev1,configfd=28,bus=pci.0,addr=0x7,rombar=0 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x6
> char device redirected to /dev/pts/1
> create_userspace_phys_mem: File exists
> assigned_dev_iomem_map: Error: create new mapping failed
So, my last question is what do the last two lines mean?
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Device pass-through
2012-01-02 19:41 Device pass-through Gordon Messmer
@ 2012-01-03 21:34 ` Alex Williamson
2012-01-04 0:29 ` Gordon Messmer
0 siblings, 1 reply; 11+ messages in thread
From: Alex Williamson @ 2012-01-03 21:34 UTC (permalink / raw)
To: Gordon Messmer; +Cc: kvm
On Mon, 2012-01-02 at 11:41 -0800, Gordon Messmer wrote:
> I'm going to summarize my questions first and write the long explanation
> afterward...
>
> 1: What happens when Linux rejects the aperture allocated by the BIOS
> for an AMD IOMMU?
This is actually the GART aperture, which Linux will try to use as an
IOMMU. You can pretty much ignore this, but it may be wasting memory
for no good reason if it's really hiding usable memory and allocating a
bounce buffer area. The important lines for device assignment are
these:
AMD-Vi: Enabling IOMMU at 0000:00:00.2 cap 0x40
AMD-Vi: Initialized for Passthrough Mode
AMD-Vi: Enabling IOMMU at 0000:00:00.2 cap 0x40
> 2: What is unsafe about the allow_unsafe_assigned_interrupts option?
> What are the potential consequences of its use?
The host is potentially exposed to a malicious guest trying to trigger
MSI interrupts for denial of service or exploit probing. If you trust
your guest, it's safe to allow this.
> 3: How can I get the ROM from a graphics card?
Not sure, if the cards are identical, try using the one from the primary
adapter. ISTR the Xen folks had some interesting methods for getting
these, google for graphics pass through on xen. Assignment of graphics
cards isn't supported. Your use as a secondary head on the VM may be
possible, but I wouldn't be surprised if it doesn't work.
> 4: What do these errors mean, and is there any way I can correct the
> problem?
> > create_userspace_phys_mem: File exists
> > assigned_dev_iomem_map: Error: create new mapping failed
Might mean the resources for the graphics card are getting mapped to
overlap with guest memory. What does lspci -v for the graphics card
show? More recent changes upstream might make better use of the space
we have for devices, might be worth a test.
Alex
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Device pass-through
2012-01-03 21:34 ` Alex Williamson
@ 2012-01-04 0:29 ` Gordon Messmer
2012-01-04 3:44 ` Alex Williamson
0 siblings, 1 reply; 11+ messages in thread
From: Gordon Messmer @ 2012-01-04 0:29 UTC (permalink / raw)
To: Alex Williamson; +Cc: kvm
Thanks, Alex. I really appreciate the reply.
On 01/03/2012 01:34 PM, Alex Williamson wrote:
> This is actually the GART aperture, which Linux will try to use as an
> IOMMU. You can pretty much ignore this, but it may be wasting memory
> for no good reason if it's really hiding usable memory and allocating a
> bounce buffer area.
It does look that way, but I doubt I'll miss it. Would you suggest that
I boot with swiotlb=0 ?
> The host is potentially exposed to a malicious guest trying to trigger
> MSI interrupts for denial of service or exploit probing. If you trust
> your guest, it's safe to allow this.
I do. It's a Fedora guest that I'm personally using. Suppose I didn't.
My kernel was built with CONFIG_INTR_REMAP=y but the kernel said
"kvm_iommu_map_guest: No interrupt remapping support, disallowing device
assignment." Does that mean that the motherboard doesn't support
interrupt remapping?
>> 4: What do these errors mean, and is there any way I can correct the
>> problem?
>>> create_userspace_phys_mem: File exists
>>> assigned_dev_iomem_map: Error: create new mapping failed
>
> Might mean the resources for the graphics card are getting mapped to
> overlap with guest memory. What does lspci -v for the graphics card
> show? More recent changes upstream might make better use of the space
> we have for devices, might be worth a test.
05:00.0 VGA compatible controller: ATI Technologies Inc RV620 PRO
[Radeon HD 3470] (prog-if 00 [VGA controller])
Subsystem: Dell Device 3243
Flags: fast devsel, IRQ 17
Memory at c0000000 (64-bit, prefetchable) [disabled] [size=256M]
Memory at fd9f0000 (64-bit, non-prefetchable)
[disabled] [size=64K]
I/O ports at ae00 [disabled] [size=256]
Expansion ROM at fd900000 [disabled by cmd] [size=128K]
Capabilities: [50] Power Management version 3
Capabilities: [58] Express Legacy Endpoint, MSI 00
Capabilities: [a0] MSI: Enable- Count=1/1 Maskable- 64bit+
Capabilities: [100] Vendor Specific Information <?>
Kernel modules: radeon
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Device pass-through
2012-01-04 0:29 ` Gordon Messmer
@ 2012-01-04 3:44 ` Alex Williamson
2012-01-05 19:07 ` Gordon Messmer
0 siblings, 1 reply; 11+ messages in thread
From: Alex Williamson @ 2012-01-04 3:44 UTC (permalink / raw)
To: Gordon Messmer; +Cc: kvm
On Tue, 2012-01-03 at 16:29 -0800, Gordon Messmer wrote:
> Thanks, Alex. I really appreciate the reply.
>
> On 01/03/2012 01:34 PM, Alex Williamson wrote:
> > This is actually the GART aperture, which Linux will try to use as an
> > IOMMU. You can pretty much ignore this, but it may be wasting memory
> > for no good reason if it's really hiding usable memory and allocating a
> > bounce buffer area.
>
> It does look that way, but I doubt I'll miss it. Would you suggest that
> I boot with swiotlb=0 ?
It looks like the AMD path initializes the GART as a fallback in case
AMD IOMMU initialization fails and allocates the swiotlb if there are
devices that are not handled by the IOMMU, so probably best just to
forget about that memory.
> > The host is potentially exposed to a malicious guest trying to trigger
> > MSI interrupts for denial of service or exploit probing. If you trust
> > your guest, it's safe to allow this.
>
> I do. It's a Fedora guest that I'm personally using. Suppose I didn't.
> My kernel was built with CONFIG_INTR_REMAP=y but the kernel said
> "kvm_iommu_map_guest: No interrupt remapping support, disallowing device
> assignment." Does that mean that the motherboard doesn't support
> interrupt remapping?
All AMD IOMMU (AMD-Vi) hardware supports interrupt remapping, but
there's no Linux software support for it, so that config option really
only enables the Intel code.
> >> 4: What do these errors mean, and is there any way I can correct the
> >> problem?
> >>> create_userspace_phys_mem: File exists
> >>> assigned_dev_iomem_map: Error: create new mapping failed
> >
> > Might mean the resources for the graphics card are getting mapped to
> > overlap with guest memory. What does lspci -v for the graphics card
> > show? More recent changes upstream might make better use of the space
> > we have for devices, might be worth a test.
>
> 05:00.0 VGA compatible controller: ATI Technologies Inc RV620 PRO
> [Radeon HD 3470] (prog-if 00 [VGA controller])
> Subsystem: Dell Device 3243
> Flags: fast devsel, IRQ 17
> Memory at c0000000 (64-bit, prefetchable) [disabled] [size=256M]
> Memory at fd9f0000 (64-bit, non-prefetchable)
> [disabled] [size=64K]
> I/O ports at ae00 [disabled] [size=256]
> Expansion ROM at fd900000 [disabled by cmd] [size=128K]
> Capabilities: [50] Power Management version 3
> Capabilities: [58] Express Legacy Endpoint, MSI 00
> Capabilities: [a0] MSI: Enable- Count=1/1 Maskable- 64bit+
> Capabilities: [100] Vendor Specific Information <?>
> Kernel modules: radeon
Yep, that's what I would have guessed, there's a 256MB resource. I'm
not sure if the seabios you're using is mapping the MMIO hole
efficiently enough to handle that. Can you test on new upstream
qemu-kvm? Thanks,
Alex
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Device pass-through
2012-01-04 3:44 ` Alex Williamson
@ 2012-01-05 19:07 ` Gordon Messmer
2012-01-06 7:25 ` Gordon Messmer
0 siblings, 1 reply; 11+ messages in thread
From: Gordon Messmer @ 2012-01-05 19:07 UTC (permalink / raw)
To: Alex Williamson; +Cc: kvm
On 01/03/2012 07:44 PM, Alex Williamson wrote:
> Yep, that's what I would have guessed, there's a 256MB resource. I'm
> not sure if the seabios you're using is mapping the MMIO hole
> efficiently enough to handle that. Can you test on new upstream
> qemu-kvm? Thanks,
I'm not done poking the system, but I thought I'd send an update.
I started with an update to seabios, from the bundled version
0.6.1.2-8.el6 to a rebuilt package from F16, 0.6.2-3.el6. That's enough
to get the guest to boot with the pass-through video card. It doesn't
work, currently, and I'm pretty sure that's because I can't read the ROM
from the card. I'm going to look around for a solution to that and will
update again.
lspci in the guest shows the device:
00:07.0 VGA compatible controller: ATI Technologies Inc Mobility Radeon
HD 3470 (prog-if 00 [VGA controller])
Subsystem: Dell Device 3243
Physical Slot: 7
Flags: fast devsel, IRQ 11
Memory at e0000000 (32-bit, prefetchable) [size=256M]
Memory at f2050000 (32-bit, non-prefetchable) [size=64K]
I/O ports at c100 [size=256]
Expansion ROM at <unassigned> [disabled]
Capabilities: [58] Express Legacy Endpoint, MSI 00
Capabilities: [50] Power Management version 3
Capabilities: [a0] MSI: Enable- Count=1/1 Maskable- 64bit-
Kernel modules: radeon
dmesg reports some errors:
[ 1.048791] [drm] Initialized drm 1.1.0 20060810
[ 1.066131] [drm] radeon defaulting to kernel modesetting.
[ 1.066133] [drm] radeon kernel modesetting enabled.
[ 1.066220] radeon 0000:00:07.0: PCI INT A -> Link[LNKC] ->
GSI 11 (level, high) -> IRQ 11
[ 1.066267] radeon 0000:00:07.0: setting latency timer to 64
[ 1.066611] [drm] initializing kernel modesetting
(RV620 0x1002:0x95C0 0x1028:0x3243).
[ 1.066774] [drm] register mmio base: 0xF2050000
[ 1.066776] [drm] register mmio size: 65536
[ 1.066884] radeon 0000:00:07.0: Expecting atombios for R600 GPU
[ 1.066886] radeon 0000:00:07.0: Fatal error during GPU init
[ 1.066888] [drm] radeon: finishing device.
[ 1.066890] [TTM] Memory type 2 has not been initialized.
[ 1.067924] vga_switcheroo: disabled
[ 1.068070] radeon 0000:00:07.0: PCI INT A disabled
[ 1.068075] radeon: probe of 0000:00:07.0 failed with error -22
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Device pass-through
2012-01-05 19:07 ` Gordon Messmer
@ 2012-01-06 7:25 ` Gordon Messmer
2012-01-06 9:31 ` André Weidemann
0 siblings, 1 reply; 11+ messages in thread
From: Gordon Messmer @ 2012-01-06 7:25 UTC (permalink / raw)
To: Alex Williamson; +Cc: kvm
On 01/05/2012 11:07 AM, Gordon Messmer wrote:
> I started with an update to seabios, from the bundled version
> 0.6.1.2-8.el6 to a rebuilt package from F16, 0.6.2-3.el6. That's enough
> to get the guest to boot with the pass-through video card. It doesn't
> work, currently, and I'm pretty sure that's because I can't read the ROM
> from the card. I'm going to look around for a solution to that and will
> update again.
Well, I finally figured out that I have to enable the reading of roms
from the device by writing "1" to the "rom" node in /sys/. Now the
problem is that the rom is 64k, and only 32k are making it into the
guest. I saw a reference to this problem here:
http://www.spinics.net/lists/kvm/msg49946.html
I've updated seabios to 0.6.3, and I've rebuilt the qemu-kvm packages
from F16 on CentOS 6.2 to try those. It took a while, but didn't change
the results. I'm still only able to read 32k of rom from the node in
/sys/ in the guest.
Perhaps I'd have better luck with a different card.
In the meantime, thanks for your advice. I think I'm close, and I'll
keep working on the project. If I get it working, I'll send along
further details with the steps I had to take.
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Device pass-through
2012-01-06 7:25 ` Gordon Messmer
@ 2012-01-06 9:31 ` André Weidemann
2012-01-07 2:07 ` Gordon Messmer
0 siblings, 1 reply; 11+ messages in thread
From: André Weidemann @ 2012-01-06 9:31 UTC (permalink / raw)
To: Gordon Messmer; +Cc: Alex Williamson, kvm
On 06.01.2012 08:25, Gordon Messmer wrote:
> On 01/05/2012 11:07 AM, Gordon Messmer wrote:
>> I started with an update to seabios, from the bundled version
>> 0.6.1.2-8.el6 to a rebuilt package from F16, 0.6.2-3.el6. That's enough
>> to get the guest to boot with the pass-through video card. It doesn't
>> work, currently, and I'm pretty sure that's because I can't read the ROM
>> from the card. I'm going to look around for a solution to that and will
>> update again.
>
> Well, I finally figured out that I have to enable the reading of roms
> from the device by writing "1" to the "rom" node in /sys/. Now the
> problem is that the rom is 64k, and only 32k are making it into the
> guest. I saw a reference to this problem here:
> http://www.spinics.net/lists/kvm/msg49946.html
>
> I've updated seabios to 0.6.3, and I've rebuilt the qemu-kvm packages
> from F16 on CentOS 6.2 to try those. It took a while, but didn't change
> the results. I'm still only able to read 32k of rom from the node in
> /sys/ in the guest.
>
> Perhaps I'd have better luck with a different card.
>
> In the meantime, thanks for your advice. I think I'm close, and I'll
> keep working on the project. If I get it working, I'll send along
> further details with the steps I had to take.
Do you have the ROM as a file on your HDD perhaps? If so, you can try
the following: -device pci-assign,host=05:00.0,romfile=${ROMFILE}.
This used to work fine for me even with ROM file of around 130k.
Regards,
André
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Device pass-through
2012-01-06 9:31 ` André Weidemann
@ 2012-01-07 2:07 ` Gordon Messmer
2012-01-07 22:21 ` Gordon Messmer
0 siblings, 1 reply; 11+ messages in thread
From: Gordon Messmer @ 2012-01-07 2:07 UTC (permalink / raw)
To: André Weidemann; +Cc: Alex Williamson, kvm
On 01/06/2012 01:31 AM, André Weidemann wrote:
> On 06.01.2012 08:25, Gordon Messmer wrote:
>> Well, I finally figured out that I have to enable the reading of roms
>> from the device by writing "1" to the "rom" node in /sys/. Now the
>> problem is that the rom is 64k, and only 32k are making it into the
>> guest. I saw a reference to this problem here:
>
> Do you have the ROM as a file on your HDD perhaps? If so, you can try
> the following: -device pci-assign,host=05:00.0,romfile=${ROMFILE}.
> This used to work fine for me even with ROM file of around 130k.
I do, and I'm actually using that. I'm using a shell script wrapper to
"fix" the argument for the PCI video card, since libvirt XML doesn't
seem to provide an option for rom files. The command that gets run is
included below.
I've learned that I was actually incorrect about the problem. The card
in the guest *does* have a 32K rom, but it's not a truncated version of
the ATI rom. Instead, it's the "Plex86/Bochs VGABios (PCI)." I've
tried starting the guest with no graphics or video hardware specified in
the libvirt XML, but qemu-kvm just eats a bunch of CPU time. I don't
see any serial console output (I do see grub and kernel output on the
serial console when I have virtual video hardware specified), so I'm
pretty sure the guest isn't actually starting.
/usr/libexec/qemu-kvm -S -M rhel6.2.0 -enable-kvm -m 4096 -smp
4,sockets=4,cores=1,threads=1 -name herald -uuid
556638be-ee50-e2f6-1d22-37b98b63b8d1 -nodefconfig -nodefaults -chardev
socket,id=charmonitor,path=/var/lib/libvirt/qemu/herald.monitor,server,nowait
-mon chardev=charmonitor,id=monitor,mode=readline -rtc base=utc -drive
file=/dev/mapper/VolGroup-lv_vm_herald,if=none,id=drive-virtio-disk0,format=raw,cache=none,aio=native
-device
virtio-blk-pci,bus=pci.0,addr=0x4,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1
-device
virtio-net-pci,vlan=0,id=net0,mac=52:54:00:21:e7:9e,bus=pci.0,addr=0x3
-net tap,fd=25,vlan=0,name=hostnet0 -chardev pty,id=charserial0 -device
isa-serial,chardev=charserial0,id=serial0 -usb -device
usb-tablet,id=input0 -vnc 127.0.0.1:0 -vga cirrus -device
pci-assign,host=00:14.2,id=hostdev0,configfd=26,bus=pci.0,addr=0x5
-device
pci-assign,host=05:00.0,id=hostdev1,configfd=27,bus=pci.0,addr=0x7,romfile=/var/lib/libvirt/images/Radeon3470.rom
-device usb-host,hostbus=5,hostaddr=3,id=hostdev2 -device
usb-host,hostbus=7,hostaddr=3,id=hostdev3 -device
virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x6
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Device pass-through
2012-01-07 2:07 ` Gordon Messmer
@ 2012-01-07 22:21 ` Gordon Messmer
2012-01-09 6:22 ` Gordon Messmer
0 siblings, 1 reply; 11+ messages in thread
From: Gordon Messmer @ 2012-01-07 22:21 UTC (permalink / raw)
Cc: kvm
On 01/06/2012 06:07 PM, Gordon Messmer wrote:
>
> /usr/libexec/qemu-kvm\
> -vnc 127.0.0.1:0 -vga cirrus \
> -device
> pci-assign,host=05:00.0,id=hostdev1,configfd=27,bus=pci.0,addr=0x7,romfile=/var/lib/libvirt/images/Radeon3470.rom
Snipped a bunch of other args... I'm certain that qemu-kvm loads both
/usr/share/qemu-kvm/vgabios-cirrus.bin and
/var/lib/libvirt/images/Radeon3470.rom, which seems correct. However,
in the guest, both the virtual VGA card and the real PCI one have the
same ROM, from vgabios-cirrus.bin.
I've looked through the qemu-kvm source and don't see anything that
would obviously cause that sort of problem. Should I be looking into
seabios?
00:02.0 0300: 1013:00b8 (prog-if 00 [VGA controller])
Subsystem: 1af4:1100
Physical Slot: 2
Flags: fast devsel
Memory at f0000000 (32-bit, prefetchable) [size=32M]
Memory at f2000000 (32-bit, non-prefetchable) [size=4K]
Expansion ROM at f2010000 [disabled] [size=64K]
Kernel modules: cirrusfb
00:07.0 0300: 1002:95c0 (prog-if 00 [VGA controller])
Subsystem: 1028:3243
Physical Slot: 7
Flags: fast devsel, IRQ 11
Memory at e0000000 (32-bit, prefetchable) [size=256M]
Memory at f2050000 (32-bit, non-prefetchable) [size=64K]
I/O ports at c100 [size=256]
Expansion ROM at f2060000 [disabled] [size=64K]
Capabilities: [58] Express Legacy Endpoint, MSI 00
Capabilities: [50] Power Management version 3
Capabilities: [a0] MSI: Enable- Count=1/1 Maskable- 64bit-
Kernel modules: radeon
# strings /sys/bus/pci/devices/0000\:00\:02.0/rom | head -2
Plex86/Bochs VGABios (PCI)
current-cvs 19 Jul 2011
# strings /sys/bus/pci/devices/0000\:00\:07.0/rom | head -2
Plex86/Bochs VGABios (PCI)
current-cvs 19 Jul 2011
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Device pass-through
2012-01-07 22:21 ` Gordon Messmer
@ 2012-01-09 6:22 ` Gordon Messmer
2012-01-16 6:52 ` Gordon Messmer
0 siblings, 1 reply; 11+ messages in thread
From: Gordon Messmer @ 2012-01-09 6:22 UTC (permalink / raw)
To: kvm
On 01/07/2012 02:21 PM, Gordon Messmer wrote:
> I'm certain that qemu-kvm loads both
> /usr/share/qemu-kvm/vgabios-cirrus.bin and
> /var/lib/libvirt/images/Radeon3470.rom, which seems correct. However,
> in the guest, both the virtual VGA card and the real PCI one have the
> same ROM, from vgabios-cirrus.bin.
I've learned slightly more. First, I built a version of seabios that
never calls vga_setup.
Once I disabled vga_setup, I was able to boot my guest with no emulated
VGA device. Now, the PCI device passed through no longer has the Cirrus
ROM. Instead it has the etherboot rom attached to the virtio ethernet
device. When I remove that device and boot the guest, the VGA card has
no ROM at all.
I get the same results on qemu-kvm-0.12.1.2 from CentOS 6.2 and with
qemu-0.15.0 from Fedora 16, rebuilt for CentOS.
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Device pass-through
2012-01-09 6:22 ` Gordon Messmer
@ 2012-01-16 6:52 ` Gordon Messmer
0 siblings, 0 replies; 11+ messages in thread
From: Gordon Messmer @ 2012-01-16 6:52 UTC (permalink / raw)
To: kvm
I still poke at this when I get time.
I notice that pci_update_mappings calls pci_bar_address, and the latter
will always return PCI_BAR_UNMAPPED for this device. For the emulated
devices, pci_bar_address will get an address with PCI_ROM_ADDRESS_ENABLE
at some point, and pci_map_option_rom will be called. With the card
that I'm passing in, that never happens. Could anyone point me in the
direction I should go from here? What sets that bit for the emulated
devices that might not happen for the PCI card?
if (type & PCI_BASE_ADDRESS_MEM_TYPE_64) {
new_addr = pci_get_quad(d->config + bar);
} else {
new_addr = pci_get_long(d->config + bar);
}
/* the ROM slot has a specific enable bit */
if (reg == PCI_ROM_SLOT && !(new_addr & PCI_ROM_ADDRESS_ENABLE)) {
return PCI_BAR_UNMAPPED;
}
^ permalink raw reply [flat|nested] 11+ messages in thread
end of thread, other threads:[~2012-01-16 6:46 UTC | newest]
Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2012-01-02 19:41 Device pass-through Gordon Messmer
2012-01-03 21:34 ` Alex Williamson
2012-01-04 0:29 ` Gordon Messmer
2012-01-04 3:44 ` Alex Williamson
2012-01-05 19:07 ` Gordon Messmer
2012-01-06 7:25 ` Gordon Messmer
2012-01-06 9:31 ` André Weidemann
2012-01-07 2:07 ` Gordon Messmer
2012-01-07 22:21 ` Gordon Messmer
2012-01-09 6:22 ` Gordon Messmer
2012-01-16 6:52 ` Gordon Messmer
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.