[GIT PULL] pv/pcifront-2.6.32

[GIT PULL] pv/pcifront-2.6.32

[Konrad Rzeszutek Wilk]

Hey Jeremy,

Please pull from
git://git.kernel.org/pub/scm/linux/kernel/git/konrad/xen.git pv/pcifront-2.6.32

It is a back-port of the Xen PCI front driver (not the SR-IOV one, just
old plain one). It has been tested with a bare 2.6.32 tree
(pv/master.2.6.32) and with xen/next

The merge of this is a bit hairy, so I did my own which is available in
the pv/xen.next.merge (which is basically xen/next + pv/pcifront-2.6.32
+ pv/fbfbront + xen: Allow unprivileged Xen domains to create iomap
pages).

To make the PCI front driver work, another patch has to be put added in
the xen/next (or better yet in the swiotlb - but it really does not seem
to fit there), which is the "xen: Allow unprivileged Xen domains to
create iomap pages" (5a7357bdb10b40414d97d7582f5467e4a709bd07)


Alex Nixon (3):
      xen: Don't disable the I/O space
      x86/PCI: Clean up pci_cache_line_size
      x86/PCI: Enable scanning of all pci functions

Ian Campbell (1):
      xen: fix off-by-one error in find_unbound_irq

Jeremy Fitzhardinge (8):
      x86/pci: make sure _PAGE_IOMAP it set on pci mappings
      xen: define BIOVEC_PHYS_MERGEABLE()
      xen: implement pirq type event channels
      x86/io_apic: add get_nr_irqs_gsi()
      xen/apic: identity map gsi->irqs
      xen: dynamically allocate irq & event structures
      xen: statically initialize cpu_evtchn_mask_p
      xen/pci: clean up Kconfig a bit

Konrad Rzeszutek Wilk (50):
      xen: set pirq name to something useful.
      Xen/x86/PCI: Add support for the Xen PCI subsystem
      pci/xen: For non-privileged domains, implement a pcibios_enable_irq (xen_pcifront_enable_irq) function.
      pci: Export pci_walk_bus function.
      xen: Find an unbound irq number in reverse order (high to low).
      xen: Provide a variant of xen_poll_irq with timeout.
      xen: Export xen_clear_irq_pending and xen_poll_irq_timeout as the PCI front module utilizes them.
      xen/apic: fix shared irq device passthrough
      xen-pci: Take advantage of the shared IRQ flags.
      pci/xen: Provide a registration mechanism for the PCI frontend module functions.
      x86: Copy-n-paste arch_teardown_msi_irqs from msi.c to io_apic.c.
      pci-xen: Add two new functions to deal with MSI/MSI-X type interrupts.
      x86: In PV guest mode, initialize MSI/MSI-X via the pci-front driver.
      xen-pci: Add support for MSI destroying of IRQ.
      xenbus: Xen paravirtualised PCI hotplug.
      xenbus: Add new states to xenbus_strstate()
      xenbus: prevent warnings on unhandled enumeration values
      xen-pcifront: Initial copy from linux-2.6.18.hg of the pcifront driver.
      xen-pcifront: Including the pciif.h header file.
      xen-pcifront: Fix include header name change.
      xen-pcifront: Fix compile warning: ignoring return value of 'pci_bus_add_device', declared with attribute warn_unused_result
      xen-pcifront: Fix compile warning: passing argument 2 of 'pci_walk_bus' from incompatible pointer type
      xen-pcifront: Fix compile error. The bind_to_irq_handler has different arguments.
      xen-pcifront: Fix compile error: implicit declaration of function 'virt_to_mfn'
      xen-pcifront: Fix compile error: implicit declaration of function 'clear_evtchn'
      xen-pcifront: Fix compile error: implicit declaration of function 'gnttab_end_foreign_access'
      xen-pcifront: Fix compile error: too few arguments to function 'gnttab_end_foreign_access'
      xen-pcifront: Remove function declerations (CONFIG_PCI_DOMAIN) that exist in recent kernels.
      xen-pcifront: Fix uage of INIT_WORK.
      xen-pcifront: Add proper check to see if running under Xen.
      xen-pcifront: Improper assumption that event channel == IRQ number.
      xen-pcifront: Replace HYPERVISOR_poll with 'xen_poll_irq_timout' function.
      xen-pcifront: Coalesce pci.c functions in xenbus.c.
      xen-pcifront: Coalesce xen/pcifront.h in drivers/xen/pcifront/pcifront.h
      xen-pcifront: Remove ia64 from pcifront.c support.
      xen-pcifront: Remove unused pci_bus_sem extern, as we don't use it.
      xen-pcifront: Coalesce pcifront.h in xenbus.c.
      xen-pcifront: Coalesce pci_op.c in xenbus.c.
      xen-pcifront: Remove unnecessary function declerations.
      xen-pcifront: Rename the drivers/xen/pcifront/* driver to drivers/pci/xen-pcifront.c.
      xen-pcifront: Change the boot-order of initialising the PCI frontend.
      xen-pcifront: Fix warnings/errors reported by checkpatch.pl
      xen-pcifront: In xen-pcifront.c updated printk(KERN_ERR ... to dev_err macro.
      xen-pcifront: Have pci_frontend_enable_[msi|msix] save PIRQ values in a int array.
      xen-pcifront: In pci_frontend_disable_msi do not alter the dev->irq.
      xen-pcifront: Check pci_claim_resource return value.
      xen-pcifront: remove driver_data direct access of struct device
      xen-pcifront: Register pci_frontend_[enable|disable]_[msi|msix] functions.
      xen-pcifront: Enable Xen PCI front to be compiled as module and also on x86.
      xen-pcifront: Make the driver build only under X86

-- diffstat:
 arch/x86/Kconfig                   |    4 +
 arch/x86/include/asm/io.h          |   13 +
 arch/x86/include/asm/io_apic.h     |    1 +
 arch/x86/include/asm/pci.h         |    8 +
 arch/x86/include/asm/pci_x86.h     |    2 +
 arch/x86/include/asm/xen/pci.h     |   55 ++
 arch/x86/kernel/apic/io_apic.c     |   34 +-
 arch/x86/pci/Makefile              |    1 +
 arch/x86/pci/common.c              |   18 +-
 arch/x86/pci/i386.c                |    3 +
 arch/x86/pci/init.c                |    6 +
 arch/x86/pci/xen.c                 |  147 +++++
 arch/x86/xen/Kconfig               |    8 +
 arch/x86/xen/setup.c               |    2 -
 drivers/block/xen-blkfront.c       |    2 +
 drivers/input/xen-kbdfront.c       |    2 +
 drivers/net/xen-netfront.c         |    2 +
 drivers/pci/Kconfig                |   10 +
 drivers/pci/Makefile               |    2 +
 drivers/pci/bus.c                  |    1 +
 drivers/pci/xen-pcifront.c         | 1156 ++++++++++++++++++++++++++++++++++++
 drivers/video/xen-fbfront.c        |    2 +
 drivers/xen/Kconfig                |    3 +-
 drivers/xen/Makefile               |    2 +-
 drivers/xen/biomerge.c             |   14 +
 drivers/xen/events.c               |  336 ++++++++++-
 drivers/xen/xenbus/xenbus_client.c |    2 +
 include/asm-generic/pci.h          |    2 +
 include/xen/events.h               |   18 +
 include/xen/interface/io/pciif.h   |  124 ++++
 include/xen/interface/io/xenbus.h  |    8 +-
 31 files changed, 1961 insertions(+), 27 deletions(-)

Re: [GIT PULL] pv/pcifront-2.6.32

[Ian Campbell]

On Tue, 2010-03-02 at 03:12 +0000, Konrad Rzeszutek Wilk wrote:
> 
> Ian Campbell (1):
>       xen: fix off-by-one error in find_unbound_irq 

You need to add this commit if you include the above:

commit 716645983e03118d11924cc245cd63fd67c6bfa8
Author: Ian Campbell <ian.campbell@citrix.com>
Date:   Mon Mar 1 12:06:15 2010 +0000

    xen: fix error handling in in find_unbound_irq
    
    68458a36 "fix off-by-one error in find_unbound_irq" introduced an issue with
    the error handling in this function. It incorrectly assumed that exiting the
    searhc loop with irq == nr_irqs - 1 was an error case when in fact it is
    prefectly possible for irq == nr_irqs - 1 to be an available IRQ.
    
    The actual error condition which 68458a36 tried to fix is when start ==
    nr_irqs, IOW when there is literaly no interrupts which aren't already h/w
    interrupts.

Re: [GIT PULL] pv/pcifront-2.6.32

[Konrad Rzeszutek Wilk]

On Tue, Mar 02, 2010 at 09:21:55AM +0000, Ian Campbell wrote:
> On Tue, 2010-03-02 at 03:12 +0000, Konrad Rzeszutek Wilk wrote:
> > 
> > Ian Campbell (1):
> >       xen: fix off-by-one error in find_unbound_irq 
> 
> You need to add this commit if you include the above:

I merged both patches and joined the  description.
This is what I have (I hope this is OK?):


commit bfbc226dc3473b49d6a8721ca8f6457e6fa1c690
Author: Ian Campbell <Ian.Campbell@citrix.com>
Date:   Mon Mar 1 13:07:18 2010 +0000

    xen: fix off-by-one error in find_unbound_irq
    
    68458a36 "fix off-by-one error in find_unbound_irq" introduced an issue with
    the error handling in this function. It incorrectly assumed that exiting the
    searhc loop with irq == nr_irqs - 1 was an error case when in fact it is
    prefectly possible for irq == nr_irqs - 1 to be an available IRQ.
    
    The actual error condition which 68458a36 tried to fix is when start ==
    nr_irqs, IOW when there is literaly no interrupts which aren't already h/w
    interrupts.
    
    Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
     drivers/xen/events.c |   10 ++++++++--
     1 files changed, 8 insertions(+), 2 deletions(-)

diff --git a/drivers/xen/events.c b/drivers/xen/events.c
index c75614b..be9ad7f 100644
--- a/drivers/xen/events.c
+++ b/drivers/xen/events.c
@@ -363,13 +363,16 @@ static int find_unbound_irq(void)
 	struct irq_desc *desc;
 	int start = get_nr_hw_irqs();
 
+	if (start == nr_irqs)
+		goto no_irqs;
+
 	/* nr_irqs is a magic value. Must not use it.*/
 	for (irq = nr_irqs-1; irq > start; irq--)
 		if (irq_info[irq].type == IRQT_UNBOUND)
 			break;
 
-	if (irq == start || irq == nr_irqs)
-		panic("No available IRQ to bind to: increase nr_irqs!\n");
+	if (irq == start)
+		goto no_irqs;
 
 	desc = irq_to_desc_alloc_node(irq, 0);
 	if (WARN_ON(desc == NULL))
@@ -378,6 +381,9 @@ static int find_unbound_irq(void)
 	dynamic_irq_init(irq);
 
 	return irq;
+
+no_irqs:
+	panic("No available IRQ to bind to: increase nr_irqs!\n");
 }
 
 static bool identity_mapped_irq(unsigned irq)
> 
> commit 716645983e03118d11924cc245cd63fd67c6bfa8
> Author: Ian Campbell <ian.campbell@citrix.com>
> Date:   Mon Mar 1 12:06:15 2010 +0000
> 
>     xen: fix error handling in in find_unbound_irq
>     
>     68458a36 "fix off-by-one error in find_unbound_irq" introduced an issue with
>     the error handling in this function. It incorrectly assumed that exiting the
>     searhc loop with irq == nr_irqs - 1 was an error case when in fact it is
>     prefectly possible for irq == nr_irqs - 1 to be an available IRQ.
>     
>     The actual error condition which 68458a36 tried to fix is when start ==
>     nr_irqs, IOW when there is literaly no interrupts which aren't already h/w
>     interrupts.
> 
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel

Re: [GIT PULL] pv/pcifront-2.6.32

[Konrad Rzeszutek Wilk]

On Mon, Mar 01, 2010 at 10:12:17PM -0500, Konrad Rzeszutek Wilk wrote:
> Hey Jeremy,
> 
> Please pull from
> git://git.kernel.org/pub/scm/linux/kernel/git/konrad/xen.git pv/pcifront-2.6.32
> 
> It is a back-port of the Xen PCI front driver (not the SR-IOV one, just
> old plain one). It has been tested with a bare 2.6.32 tree
> (pv/master.2.6.32) and with xen/next

Thought I must say I haven't done Dom0 booting of the kernel, so it
might not work (hopefully it will work just peachy). But for PV and HVM
guests it works splendidly.

Re: [GIT PULL] pv/pcifront-2.6.32

[Ian Campbell]

On Tue, 2010-03-02 at 13:55 +0000, Konrad Rzeszutek Wilk wrote:
> On Tue, Mar 02, 2010 at 09:21:55AM +0000, Ian Campbell wrote:
> > On Tue, 2010-03-02 at 03:12 +0000, Konrad Rzeszutek Wilk wrote:
> > > 
> > > Ian Campbell (1):
> > >       xen: fix off-by-one error in find_unbound_irq 
> > 
> > You need to add this commit if you include the above:
> 
> I merged both patches and joined the  description.
> This is what I have (I hope this is OK?):

Fine with me, thanks.

Ian.

> 
> 
> commit bfbc226dc3473b49d6a8721ca8f6457e6fa1c690
> Author: Ian Campbell <Ian.Campbell@citrix.com>
> Date:   Mon Mar 1 13:07:18 2010 +0000
> 
>     xen: fix off-by-one error in find_unbound_irq
>     
>     68458a36 "fix off-by-one error in find_unbound_irq" introduced an issue with
>     the error handling in this function. It incorrectly assumed that exiting the
>     searhc loop with irq == nr_irqs - 1 was an error case when in fact it is
>     prefectly possible for irq == nr_irqs - 1 to be an available IRQ.
>     
>     The actual error condition which 68458a36 tried to fix is when start ==
>     nr_irqs, IOW when there is literaly no interrupts which aren't already h/w
>     interrupts.
>     
>     Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
>      drivers/xen/events.c |   10 ++++++++--
>      1 files changed, 8 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/xen/events.c b/drivers/xen/events.c
> index c75614b..be9ad7f 100644
> --- a/drivers/xen/events.c
> +++ b/drivers/xen/events.c
> @@ -363,13 +363,16 @@ static int find_unbound_irq(void)
>  	struct irq_desc *desc;
>  	int start = get_nr_hw_irqs();
>  
> +	if (start == nr_irqs)
> +		goto no_irqs;
> +
>  	/* nr_irqs is a magic value. Must not use it.*/
>  	for (irq = nr_irqs-1; irq > start; irq--)
>  		if (irq_info[irq].type == IRQT_UNBOUND)
>  			break;
>  
> -	if (irq == start || irq == nr_irqs)
> -		panic("No available IRQ to bind to: increase nr_irqs!\n");
> +	if (irq == start)
> +		goto no_irqs;
>  
>  	desc = irq_to_desc_alloc_node(irq, 0);
>  	if (WARN_ON(desc == NULL))
> @@ -378,6 +381,9 @@ static int find_unbound_irq(void)
>  	dynamic_irq_init(irq);
>  
>  	return irq;
> +
> +no_irqs:
> +	panic("No available IRQ to bind to: increase nr_irqs!\n");
>  }
>  
>  static bool identity_mapped_irq(unsigned irq)
> > 
> > commit 716645983e03118d11924cc245cd63fd67c6bfa8
> > Author: Ian Campbell <ian.campbell@citrix.com>
> > Date:   Mon Mar 1 12:06:15 2010 +0000
> > 
> >     xen: fix error handling in in find_unbound_irq
> >     
> >     68458a36 "fix off-by-one error in find_unbound_irq" introduced an issue with
> >     the error handling in this function. It incorrectly assumed that exiting the
> >     searhc loop with irq == nr_irqs - 1 was an error case when in fact it is
> >     prefectly possible for irq == nr_irqs - 1 to be an available IRQ.
> >     
> >     The actual error condition which 68458a36 tried to fix is when start ==
> >     nr_irqs, IOW when there is literaly no interrupts which aren't already h/w
> >     interrupts.
> > 
> > 
> > 
> > _______________________________________________
> > Xen-devel mailing list
> > Xen-devel@lists.xensource.com
> > http://lists.xensource.com/xen-devel

Re: [GIT PULL] pv/pcifront-2.6.32

[Jeremy Fitzhardinge]

On 03/01/2010 07:12 PM, Konrad Rzeszutek Wilk wrote:
> Hey Jeremy,
>
> Please pull from
> git://git.kernel.org/pub/scm/linux/kernel/git/konrad/xen.git pv/pcifront-2.6.32
>
> It is a back-port of the Xen PCI front driver (not the SR-IOV one, just
> old plain one). It has been tested with a bare 2.6.32 tree
> (pv/master.2.6.32) and with xen/next
>
> The merge of this is a bit hairy, so I did my own which is available in
> the pv/xen.next.merge (which is basically xen/next + pv/pcifront-2.6.32
> + pv/fbfbront + xen: Allow unprivileged Xen domains to create iomap
> pages).
>    

I did a separate merge of the fbfront branch.  Can you do another merge 
with just the pcifront stuff?

(BTW, do you use git rerere?  It remembers the results of previous merge 
resolutions and will apply them again if you re-merge.)

> To make the PCI front driver work, another patch has to be put added in
> the xen/next (or better yet in the swiotlb - but it really does not seem
> to fit there), which is the "xen: Allow unprivileged Xen domains to
> create iomap pages" (5a7357bdb10b40414d97d7582f5467e4a709bd07)
>    

Do we really need that patch any more?  The comment doesn't seem to bear 
any relationship to its contents any more; it talks about disallowing 
ISA mappings, but all it does it mask write-through/uncachable from PTEs 
for non-privileged domains, and I'm not sure that's even desirable.  I 
think Xen will mask those flags from the PTE if the domain isn't allowed 
to touch the device, and we want to be able to let passthrough mappings 
to be uncached or writethrough.

Do things break without it?

(BTW, it looks like its change 979e121cb348add17ed8171bf447b27a3a9d1be3 
in your tree; I don't have 5a7357bdb10b40414d97d7582f5467e4a709bd07 in 
my repo at all).

Thanks,
     J

Re: Re: [GIT PULL] pv/pcifront-2.6.32

[Konrad Rzeszutek Wilk]

On Tue, Mar 02, 2010 at 12:12:23PM -0800, Jeremy Fitzhardinge wrote:
> On 03/01/2010 07:12 PM, Konrad Rzeszutek Wilk wrote:
>> Hey Jeremy,
>>
>> Please pull from
>> git://git.kernel.org/pub/scm/linux/kernel/git/konrad/xen.git pv/pcifront-2.6.32
>>
>> It is a back-port of the Xen PCI front driver (not the SR-IOV one, just
>> old plain one). It has been tested with a bare 2.6.32 tree
>> (pv/master.2.6.32) and with xen/next
>>
>> The merge of this is a bit hairy, so I did my own which is available in
>> the pv/xen.next.merge (which is basically xen/next + pv/pcifront-2.6.32
>> + pv/fbfbront + xen: Allow unprivileged Xen domains to create iomap
>> pages).
>>    
>
> I did a separate merge of the fbfront branch.  Can you do another merge  
> with just the pcifront stuff?

Done: pv/merge.xen.next
>
> (BTW, do you use git rerere?  It remembers the results of previous merge  
> resolutions and will apply them again if you re-merge.)

Oooh goodies. Will start using it.
>
>> To make the PCI front driver work, another patch has to be put added in
>> the xen/next (or better yet in the swiotlb - but it really does not seem
>> to fit there), which is the "xen: Allow unprivileged Xen domains to
>> create iomap pages" (5a7357bdb10b40414d97d7582f5467e4a709bd07)
>>    
>
> Do we really need that patch any more?  The comment doesn't seem to bear  
> any relationship to its contents any more; it talks about disallowing  
> ISA mappings, but all it does it mask write-through/uncachable from PTEs  
> for non-privileged domains, and I'm not sure that's even desirable.  I  
> think Xen will mask those flags from the PTE if the domain isn't allowed  
> to touch the device, and we want to be able to let passthrough mappings  
> to be uncached or writethrough.
>
> Do things break without it?

I needed it for v2.6.32 + jeremy/xen/dom0/konrad-swiotlb-2.6.32 +
konrad/pv/pcifront-2.6.32. Otherwise I would get this:

[    0.580008]   alloc kstat_irqs on node 0
[    0.586116] BUG: unable to handle kernel paging request at ffff88001fc9f7b0
[    0.586133] IP: [<ffffffff8100cb39>] xen_set_pte+0x42/0x4d
[    0.586149] PGD 1002067 PUD 1006067 PMD 1d4067 PTE 801000001fc9f065
[    0.586171] Oops: 0003 [#1] SMP 
[    0.586183] last sysfs file: 
[    0.586191] CPU 0 
[    0.586199] Modules linked in:
[    0.586211] Pid: 1, comm: swapper Not tainted 2.6.32NEB-00120-gd21d0fb #71 
[    0.586220] RIP: e030:[<ffffffff8100cb39>]  [<ffffffff8100cb39>] xen_set_pte+0x42/0x4d
[    0.586236] RSP: e02b:ffff88001fcffaf0  EFLAGS: 00010296
[    0.586245] RAX: 0000000000000000 RBX: ffff88001fc9f7b0 RCX: fffff7fffffff463
[    0.586255] RDX: 0000000000000000 RSI: fffff7fffffff463 RDI: ffff88001fc9f7b0
[    0.586265] RBP: ffff88001fcffb10 R08: ffffffff81547490 R09: 00000000000000d0
[    0.586275] R10: ffffffff81547160 R11: ffff88001bd3e480 R12: fffff7fffffff463
[    0.586285] R13: 0000000000000000 R14: ffffffff815560f0 R15: ffff88001fd00000
[    0.586299] FS:  0000000000000000(0000) GS:ffff880004ab8000(0000) knlGS:0000000000000000
[    0.586311] CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
[    0.586320] CR2: ffff88001fc9f7b0 CR3: 0000000001001000 CR4: 0000000000002660
[    0.586331] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[    0.586341] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[    0.586352] Process swapper (pid: 1, threadinfo ffff88001fcfe000, task ffff88001fd00000)
[    0.586385] Stack:
[    0.586401]  ffff88001fcffb08 ffff88001fc9f7b0 fffff7fffffff463 ffffc900000f6000
[    0.586423] <0> ffff88001fcffb70 ffffffff8100cd16 ffffffff8100c67b ffff88001fcffc20
[    0.586459] <0> ffffffff8100c241 ffff88001bd3e480 ffffffff81547160 ffffc900000f7000
[    0.586499] Call Trace:
[    0.586515]  [<ffffffff8100cd16>] xen_set_pte_at+0xdb/0xea
[    0.586528]  [<ffffffff8100c67b>] ? xen_make_pte+0x9/0xb
[    0.586540]  [<ffffffff8100c241>] ? __raw_callee_save_xen_make_pte+0x11/0x1e
[    0.586555]  [<ffffffff811b9bc2>] ioremap_page_range+0x236/0x298
[    0.586569]  [<ffffffff81033e69>] __ioremap_caller+0x291/0x2f7
[    0.586581]  [<ffffffff81033f9d>] ioremap_nocache+0x12/0x14
[    0.586595]  [<ffffffff8127d25c>] usb_hcd_pci_probe+0x155/0x305
[    0.586608]  [<ffffffff8100f28d>] ? xen_spin_unlock+0x11/0x30
[    0.586622]  [<ffffffff811d0433>] local_pci_probe+0x12/0x16
[    0.586634]  [<ffffffff811d1138>] pci_device_probe+0xc2/0xf2
[    0.586648]  [<ffffffff8124c535>] ? driver_sysfs_add+0x47/0x6c
[    0.586660]  [<ffffffff8124c66d>] driver_probe_device+0x9a/0x11f
[    0.586673]  [<ffffffff8124c74a>] __driver_attach+0x58/0x7c
[    0.586685]  [<ffffffff8124c6f2>] ? __driver_attach+0x0/0x7c
[    0.586697]  [<ffffffff8124bce4>] bus_for_each_dev+0x4e/0x83
[    0.586710]  [<ffffffff8124c4ec>] driver_attach+0x19/0x1b
[    0.586721]  [<ffffffff8124c15d>] bus_add_driver+0xb1/0x200
[    0.586733]  [<ffffffff8124ca0c>] driver_register+0x98/0x109
[    0.586746]  [<ffffffff811d1381>] __pci_register_driver+0x53/0xc3
[    0.586759]  [<ffffffff815f5514>] ehci_hcd_init+0xbc/0xe7
[    0.586770]  [<ffffffff815f5458>] ? ehci_hcd_init+0x0/0xe7
[    0.586782]  [<ffffffff8100a05f>] do_one_initcall+0x59/0x154
[    0.586795]  [<ffffffff815c86bc>] kernel_init+0x165/0x1bb
[    0.586807]  [<ffffffff81012a6a>] child_rip+0xa/0x20
[    0.586819]  [<ffffffff81011c21>] ? int_ret_from_sys_call+0x7/0x1b
[    0.586831]  [<ffffffff810123dd>] ? retint_restore_args+0x5/0x6
[    0.586843]  [<ffffffff81012a60>] ? child_rip+0x0/0x20
[    0.586851] Code: ff ff 05 df 05 66 00 e8 63 fb ff ff 44 8b 2d db 05 66 00 e8 fa ff 01 00 ff c8 0f 94 c0 0f b6 c0 46 8d 2c 28 44 89 2d c3 05 66 00 <4c> 89 23 5b 5b 41 5c 41 5d c9 c3 55 48 89 e5 53 89 fb 48 83 ec 
[    0.587057] RIP  [<ffffffff8100cb39>] xen_set_pte+0x42/0x4d
[    0.587060]  RSP <ffff88001fcffaf0>
[    0.587060] CR2: ffff88001fc9f7b0
[    0.587060] ---[ end trace c6d643cd9bbe496e ]---


_But_, with xen/next + pv/pcifront there was no need for it.

Re: Re: [GIT PULL] pv/pcifront-2.6.32

[Konrad Rzeszutek Wilk]

On Tue, Mar 02, 2010 at 04:54:32PM -0500, Konrad Rzeszutek Wilk wrote:
> On Tue, Mar 02, 2010 at 12:12:23PM -0800, Jeremy Fitzhardinge wrote:
> > On 03/01/2010 07:12 PM, Konrad Rzeszutek Wilk wrote:
> >> Hey Jeremy,
> >>
> >> Please pull from
> >> git://git.kernel.org/pub/scm/linux/kernel/git/konrad/xen.git pv/pcifront-2.6.32
> >>
> >> It is a back-port of the Xen PCI front driver (not the SR-IOV one, just
> >> old plain one). It has been tested with a bare 2.6.32 tree
> >> (pv/master.2.6.32) and with xen/next
> >>
> >> The merge of this is a bit hairy, so I did my own which is available in
> >> the pv/xen.next.merge (which is basically xen/next + pv/pcifront-2.6.32
> >> + pv/fbfbront + xen: Allow unprivileged Xen domains to create iomap
> >> pages).
> >>    
> >
> > I did a separate merge of the fbfront branch.  Can you do another merge  
> > with just the pcifront stuff?
> 
> Done: pv/merge.xen.next

And in case you interested in a xen/stable merge: pv/merge.xen.stable

> >
> > (BTW, do you use git rerere?  It remembers the results of previous merge  
> > resolutions and will apply them again if you re-merge.)
> 
> Oooh goodies. Will start using it.
> >
> >> To make the PCI front driver work, another patch has to be put added in
> >> the xen/next (or better yet in the swiotlb - but it really does not seem
> >> to fit there), which is the "xen: Allow unprivileged Xen domains to
> >> create iomap pages" (5a7357bdb10b40414d97d7582f5467e4a709bd07)
> >>    
> >
> > Do we really need that patch any more?  The comment doesn't seem to bear  

A bit of testing shows that for xen/next +pv/pcifront-2.6.32 and
xen/stable +  pv/pcifront-2.6.32 that patch is not needed.

Re: Re: [GIT PULL] pv/pcifront-2.6.32

[Jeremy Fitzhardinge]

On 03/02/2010 01:54 PM, Konrad Rzeszutek Wilk wrote:
> On Tue, Mar 02, 2010 at 12:12:23PM -0800, Jeremy Fitzhardinge wrote:
>    
>> On 03/01/2010 07:12 PM, Konrad Rzeszutek Wilk wrote:
>>      
>>> Hey Jeremy,
>>>
>>> Please pull from
>>> git://git.kernel.org/pub/scm/linux/kernel/git/konrad/xen.git pv/pcifront-2.6.32
>>>
>>> It is a back-port of the Xen PCI front driver (not the SR-IOV one, just
>>> old plain one). It has been tested with a bare 2.6.32 tree
>>> (pv/master.2.6.32) and with xen/next
>>>
>>> The merge of this is a bit hairy, so I did my own which is available in
>>> the pv/xen.next.merge (which is basically xen/next + pv/pcifront-2.6.32
>>> + pv/fbfbront + xen: Allow unprivileged Xen domains to create iomap
>>> pages).
>>>
>>>        
>> I did a separate merge of the fbfront branch.  Can you do another merge
>> with just the pcifront stuff?
>>      
> Done: pv/merge.xen.next
>    
>> (BTW, do you use git rerere?  It remembers the results of previous merge
>> resolutions and will apply them again if you re-merge.)
>>      
> Oooh goodies. Will start using it.
>    
>>      
>>> To make the PCI front driver work, another patch has to be put added in
>>> the xen/next (or better yet in the swiotlb - but it really does not seem
>>> to fit there), which is the "xen: Allow unprivileged Xen domains to
>>> create iomap pages" (5a7357bdb10b40414d97d7582f5467e4a709bd07)
>>>
>>>        
>> Do we really need that patch any more?  The comment doesn't seem to bear
>> any relationship to its contents any more; it talks about disallowing
>> ISA mappings, but all it does it mask write-through/uncachable from PTEs
>> for non-privileged domains, and I'm not sure that's even desirable.  I
>> think Xen will mask those flags from the PTE if the domain isn't allowed
>> to touch the device, and we want to be able to let passthrough mappings
>> to be uncached or writethrough.
>>
>> Do things break without it?
>>      
> I needed it for v2.6.32 + jeremy/xen/dom0/konrad-swiotlb-2.6.32 +
> konrad/pv/pcifront-2.6.32. Otherwise I would get this:
>    

OK, so I guess Xen does complain if you try to set UC/WT ptes but are 
non-privileged.  But still, with that patch in place, it means that even 
a passthrough hardware mapping may lack UC/WT.  Perhaps that doesn't 
matter?  I bet it will when we get around to fixing up PAT.

> _But_, with xen/next + pv/pcifront there was no need for it.
>    

That change is already present.

     J

Re: Re: [GIT PULL] pv/pcifront-2.6.32

[Jeremy Fitzhardinge]

On 03/02/2010 02:20 PM, Konrad Rzeszutek Wilk wrote:
> And in case you interested in a xen/stable merge: pv/merge.xen.stable
>    

Thanks, but I just merge xen/next into stable rather than re-merging any 
of the topic branches.

     J

Re: Re: [GIT PULL] pv/pcifront-2.6.32

[Konrad Rzeszutek Wilk]

>> I needed it for v2.6.32 + jeremy/xen/dom0/konrad-swiotlb-2.6.32 +
>> konrad/pv/pcifront-2.6.32. Otherwise I would get this:
>>    
>
> OK, so I guess Xen does complain if you try to set UC/WT ptes but are  
> non-privileged.  But still, with that patch in place, it means that even  
> a passthrough hardware mapping may lack UC/WT.  Perhaps that doesn't  
> matter?  I bet it will when we get around to fixing up PAT.

It is more than just the UC/WT PTEs. Here is the commit:

>
commit f6a09fcc104ca416e98feb1cecc3ee16509dc376
Author: Alex Nixon <alex.nixon@citrix.com>
Date:   Mon Feb 9 15:05:46 2009 -0500

    xen: Allow unprivileged Xen domains to create iomap pages
    
    PV DomU domains are allowed to map hardware MFNs for PCI passthrough,
    but are not generally allowed to map raw machine pages.  In particular,
    various pieces of code try to map DMI and ACPI tables in the ISA ROM
    range.  We disallow _PAGE_IOMAP for those mappings, so that they are
    redirected to a set of local zeroed pages we reserve for that purpose.
    
    [ Impact: prevent passthrough of ISA space, as we only allow PCI ]
    
    Signed-off-by: Alex Nixon <alex.nixon@citrix.com>
    Signed-off-by: Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com>
    
    Conflicts:
    
    	arch/x86/xen/enlighten.c

diff --git a/arch/x86/xen/enlighten.c b/arch/x86/xen/enlighten.c
index dfbf70e..cc2829a 100644
--- a/arch/x86/xen/enlighten.c
+++ b/arch/x86/xen/enlighten.c
@@ -1138,6 +1138,10 @@ asmlinkage void __init xen_start_kernel(void)
 
 	pgd = (pgd_t *)xen_start_info->pt_base;
 
+	if (!xen_initial_domain())
+		__supported_pte_mask &= ~(_PAGE_PWT | _PAGE_PCD);
+
+	__supported_pte_mask |= _PAGE_IOMAP;
 	/* Don't do the full vcpu_info placement stuff until we have a
 	   possible map and a non-dummy shared_info. */
 	per_cpu(xen_vcpu, 0) = &HYPERVISOR_shared_info->vcpu_info[0];
diff --git a/arch/x86/xen/mmu.c b/arch/x86/xen/mmu.c
index 2630313..0bc8fa9 100644
--- a/arch/x86/xen/mmu.c
+++ b/arch/x86/xen/mmu.c
@@ -50,6 +50,7 @@
 #include <asm/mmu_context.h>
 #include <asm/setup.h>
 #include <asm/paravirt.h>
+#include <asm/e820.h>
 #include <asm/linkage.h>
 #include <asm/page.h>
 
@@ -387,7 +388,7 @@ static bool xen_page_pinned(void *ptr)
 
 static bool xen_iomap_pte(pte_t pte)
 {
-	return xen_initial_domain() && (pte_flags(pte) & _PAGE_IOMAP);
+	return pte_flags(pte) & _PAGE_IOMAP;
 }
 
 static void xen_set_iomap_pte(pte_t *ptep, pte_t pteval)
@@ -589,10 +590,21 @@ PV_CALLEE_SAVE_REGS_THUNK(xen_pgd_val);
 
 pte_t xen_make_pte(pteval_t pte)
 {
-	if (unlikely(xen_initial_domain() && (pte & _PAGE_IOMAP)))
+	phys_addr_t addr = (pte & PTE_PFN_MASK);
+
+	/*
+	 * Unprivileged domains are allowed to do IOMAPpings for
+	 * PCI passthrough, but not map ISA space.  The ISA
+	 * mappings are just dummy local mappings to keep other
+	 * parts of the kernel happy.
+	 */
+	if (unlikely(pte & _PAGE_IOMAP) &&
+	    (xen_initial_domain() || addr >= ISA_END_ADDRESS)) {
 		pte = iomap_pte(pte);
-	else
+	} else {
+		pte &= ~_PAGE_IOMAP;
 		pte = pte_pfn_to_mfn(pte);
+	}
 
 	return native_make_pte(pte);
 }