[PATCH] apic: fix kvm build on UP without IOAPIC

[PATCH] apic: fix kvm build on UP without IOAPIC

[Michael S. Tsirkin]

On UP i386, when APIC is disabled
# CONFIG_X86_UP_APIC is not set
# CONFIG_PCI_IOAPIC is not set

code looking at apicdrivers never has any effect but it
still gets compiled in. In particular, this causes
build failures with kvm, but it generally bloats the kernel
unnecessarily.

Fix by defining both __apicdrivers and __apicdrivers_end
to be NULL when CONFIG_X86_LOCAL_APIC is unset: I verified
that as the result any loop scanning __apicdrivers gets optimized out by
the compiler.

Warning: a .config with apic disabled doesn't seem to boot
for me (even without this patch). Still verifying why,
meanwhile this patch is compile-tested only.

Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---

Note: if this patch makes sense, can x86 maintainers
please ACK applying it through the kvm tree, since that is
where we see the issue that it addresses?
Avi, Marcelo, maybe you can carry this in kvm/linux-next as a temporary
measure so that linux-next builds?

 arch/x86/include/asm/apic.h | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/arch/x86/include/asm/apic.h b/arch/x86/include/asm/apic.h
index eaff479..aa5b2ee 100644
--- a/arch/x86/include/asm/apic.h
+++ b/arch/x86/include/asm/apic.h
@@ -417,7 +417,12 @@ extern struct apic *apic;
 	__aligned(sizeof(struct apic *))				\
 	__section(.apicdrivers) = { &sym1, &sym2 }
 
+#ifdef CONFIG_X86_LOCAL_APIC
 extern struct apic *__apicdrivers[], *__apicdrivers_end[];
+#else
+#define __apicdrivers ((struct apic **)NULL)
+#define __apicdrivers_end ((struct apic **)NULL)
+#endif
 
 /*
  * APIC functionality to boot other CPUs - only used on SMP:
-- 
MST

Re: [PATCH] apic: fix kvm build on UP without IOAPIC

[Randy Dunlap]

On 07/01/2012 08:05 AM, Michael S. Tsirkin wrote:

> On UP i386, when APIC is disabled
> # CONFIG_X86_UP_APIC is not set
> # CONFIG_PCI_IOAPIC is not set
> 
> code looking at apicdrivers never has any effect but it
> still gets compiled in. In particular, this causes
> build failures with kvm, but it generally bloats the kernel
> unnecessarily.
> 
> Fix by defining both __apicdrivers and __apicdrivers_end
> to be NULL when CONFIG_X86_LOCAL_APIC is unset: I verified
> that as the result any loop scanning __apicdrivers gets optimized out by
> the compiler.
> 
> Warning: a .config with apic disabled doesn't seem to boot
> for me (even without this patch). Still verifying why,
> meanwhile this patch is compile-tested only.
> 
> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>


Reported-by: Randy Dunlap <rdunlap@xenotime.net>
Acked-by: Randy Dunlap <rdunlap@xenotime.net>

Thanks.

> ---
> 
> Note: if this patch makes sense, can x86 maintainers
> please ACK applying it through the kvm tree, since that is
> where we see the issue that it addresses?
> Avi, Marcelo, maybe you can carry this in kvm/linux-next as a temporary
> measure so that linux-next builds?
> 
>  arch/x86/include/asm/apic.h | 5 +++++
>  1 file changed, 5 insertions(+)
> 
> diff --git a/arch/x86/include/asm/apic.h b/arch/x86/include/asm/apic.h
> index eaff479..aa5b2ee 100644
> --- a/arch/x86/include/asm/apic.h
> +++ b/arch/x86/include/asm/apic.h
> @@ -417,7 +417,12 @@ extern struct apic *apic;
>  	__aligned(sizeof(struct apic *))				\
>  	__section(.apicdrivers) = { &sym1, &sym2 }
>  
> +#ifdef CONFIG_X86_LOCAL_APIC
>  extern struct apic *__apicdrivers[], *__apicdrivers_end[];
> +#else
> +#define __apicdrivers ((struct apic **)NULL)
> +#define __apicdrivers_end ((struct apic **)NULL)
> +#endif
>  
>  /*
>   * APIC functionality to boot other CPUs - only used on SMP:



-- 
~Randy

Re: [PATCH] apic: fix kvm build on UP without IOAPIC

[H. Peter Anvin]

On 07/01/2012 08:05 AM, Michael S. Tsirkin wrote:
> On UP i386, when APIC is disabled
> # CONFIG_X86_UP_APIC is not set
> # CONFIG_PCI_IOAPIC is not set
>
> code looking at apicdrivers never has any effect but it
> still gets compiled in. In particular, this causes
> build failures with kvm, but it generally bloats the kernel
> unnecessarily.
>
> Fix by defining both __apicdrivers and __apicdrivers_end
> to be NULL when CONFIG_X86_LOCAL_APIC is unset: I verified
> that as the result any loop scanning __apicdrivers gets optimized out by
> the compiler.
>
> Warning: a .config with apic disabled doesn't seem to boot
> for me (even without this patch). Still verifying why,
> meanwhile this patch is compile-tested only.
>
> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
> ---
>
> Note: if this patch makes sense, can x86 maintainers
> please ACK applying it through the kvm tree, since that is
> where we see the issue that it addresses?
> Avi, Marcelo, maybe you can carry this in kvm/linux-next as a temporary
> measure so that linux-next builds?
>

I'm not happy about that as a workflow, but since you guys have an 
immediate problem I guess we can do that.


Acked-by: H. Peter Anvin <hpa@linux.intel.com>

	-hpa

-- 
H. Peter Anvin, Intel Open Source Technology Center
I work for Intel.  I don't speak on their behalf.

Re: [PATCH] apic: fix kvm build on UP without IOAPIC

[Marcelo Tosatti]

On Sun, Jul 01, 2012 at 06:05:06PM +0300, Michael S. Tsirkin wrote:
> On UP i386, when APIC is disabled
> # CONFIG_X86_UP_APIC is not set
> # CONFIG_PCI_IOAPIC is not set
> 
> code looking at apicdrivers never has any effect but it
> still gets compiled in. In particular, this causes
> build failures with kvm, but it generally bloats the kernel
> unnecessarily.
> 
> Fix by defining both __apicdrivers and __apicdrivers_end
> to be NULL when CONFIG_X86_LOCAL_APIC is unset: I verified
> that as the result any loop scanning __apicdrivers gets optimized out by
> the compiler.
> 
> Warning: a .config with apic disabled doesn't seem to boot
> for me (even without this patch). Still verifying why,
> meanwhile this patch is compile-tested only.
> 
> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
> ---
> 
> Note: if this patch makes sense, can x86 maintainers
> please ACK applying it through the kvm tree, since that is
> where we see the issue that it addresses?
> Avi, Marcelo, maybe you can carry this in kvm/linux-next as a temporary
> measure so that linux-next builds?

Applied, thanks.

Re: [PATCH] apic: fix kvm build on UP without IOAPIC

[Ingo Molnar]

* H. Peter Anvin <hpa@zytor.com> wrote:

> On 07/01/2012 08:05 AM, Michael S. Tsirkin wrote:
> >On UP i386, when APIC is disabled
> ># CONFIG_X86_UP_APIC is not set
> ># CONFIG_PCI_IOAPIC is not set
> >
> >code looking at apicdrivers never has any effect but it
> >still gets compiled in. In particular, this causes
> >build failures with kvm, but it generally bloats the kernel
> >unnecessarily.
> >
> >Fix by defining both __apicdrivers and __apicdrivers_end
> >to be NULL when CONFIG_X86_LOCAL_APIC is unset: I verified
> >that as the result any loop scanning __apicdrivers gets optimized out by
> >the compiler.
> >
> >Warning: a .config with apic disabled doesn't seem to boot
> >for me (even without this patch). Still verifying why,
> >meanwhile this patch is compile-tested only.
> >
> >Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
> >---
> >
> >Note: if this patch makes sense, can x86 maintainers
> >please ACK applying it through the kvm tree, since that is
> >where we see the issue that it addresses?
> >Avi, Marcelo, maybe you can carry this in kvm/linux-next as a temporary
> >measure so that linux-next builds?
> >
> 
> I'm not happy about that as a workflow, but since you guys have an
> immediate problem I guess we can do that.

I'm rather unhappy about this workflow - we've got quite a few 
apic bits in the x86 tree this cycle as well and need extra 
external interaction, not.

Which KVM tree commit caused this, could someone please give a 
lkml link or quote it here? It's not referenced in the fix patch 
either.

Thanks,

	Ingo

Re: [PATCH] apic: fix kvm build on UP without IOAPIC

[Marcelo Tosatti]

On Fri, Jul 06, 2012 at 01:13:14PM +0200, Ingo Molnar wrote:
> 
> * H. Peter Anvin <hpa@zytor.com> wrote:
> 
> > On 07/01/2012 08:05 AM, Michael S. Tsirkin wrote:
> > >On UP i386, when APIC is disabled
> > ># CONFIG_X86_UP_APIC is not set
> > ># CONFIG_PCI_IOAPIC is not set
> > >
> > >code looking at apicdrivers never has any effect but it
> > >still gets compiled in. In particular, this causes
> > >build failures with kvm, but it generally bloats the kernel
> > >unnecessarily.
> > >
> > >Fix by defining both __apicdrivers and __apicdrivers_end
> > >to be NULL when CONFIG_X86_LOCAL_APIC is unset: I verified
> > >that as the result any loop scanning __apicdrivers gets optimized out by
> > >the compiler.
> > >
> > >Warning: a .config with apic disabled doesn't seem to boot
> > >for me (even without this patch). Still verifying why,
> > >meanwhile this patch is compile-tested only.
> > >
> > >Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
> > >---
> > >
> > >Note: if this patch makes sense, can x86 maintainers
> > >please ACK applying it through the kvm tree, since that is
> > >where we see the issue that it addresses?
> > >Avi, Marcelo, maybe you can carry this in kvm/linux-next as a temporary
> > >measure so that linux-next builds?
> > >
> > 
> > I'm not happy about that as a workflow, but since you guys have an
> > immediate problem I guess we can do that.
> 
> I'm rather unhappy about this workflow - we've got quite a few 
> apic bits in the x86 tree this cycle as well and need extra 
> external interaction, not.
> 
> Which KVM tree commit caused this, could someone please give a 
> lkml link or quote it here? It's not referenced in the fix patch 
> either.
> 
> Thanks,
> 
> 	Ingo

This tree (kvm.git next):

http://git.kernel.org/?p=virt/kvm/kvm.git;a=shortlog;h=refs/heads/next

Introduced by this commit:

http://git.kernel.org/?p=virt/kvm/kvm.git;a=commit;h=ab9cf4996bb989983e73da894b8dd0239aa2c3c2

Re: [PATCH] apic: fix kvm build on UP without IOAPIC

[Ingo Molnar]

* Marcelo Tosatti <mtosatti@redhat.com> wrote:

> On Fri, Jul 06, 2012 at 01:13:14PM +0200, Ingo Molnar wrote:
> > 
> > * H. Peter Anvin <hpa@zytor.com> wrote:
> > 
> > > On 07/01/2012 08:05 AM, Michael S. Tsirkin wrote:
> > > >On UP i386, when APIC is disabled
> > > ># CONFIG_X86_UP_APIC is not set
> > > ># CONFIG_PCI_IOAPIC is not set
> > > >
> > > >code looking at apicdrivers never has any effect but it
> > > >still gets compiled in. In particular, this causes
> > > >build failures with kvm, but it generally bloats the kernel
> > > >unnecessarily.
> > > >
> > > >Fix by defining both __apicdrivers and __apicdrivers_end
> > > >to be NULL when CONFIG_X86_LOCAL_APIC is unset: I verified
> > > >that as the result any loop scanning __apicdrivers gets optimized out by
> > > >the compiler.
> > > >
> > > >Warning: a .config with apic disabled doesn't seem to boot
> > > >for me (even without this patch). Still verifying why,
> > > >meanwhile this patch is compile-tested only.
> > > >
> > > >Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
> > > >---
> > > >
> > > >Note: if this patch makes sense, can x86 maintainers
> > > >please ACK applying it through the kvm tree, since that is
> > > >where we see the issue that it addresses?
> > > >Avi, Marcelo, maybe you can carry this in kvm/linux-next as a temporary
> > > >measure so that linux-next builds?
> > > >
> > > 
> > > I'm not happy about that as a workflow, but since you guys have an
> > > immediate problem I guess we can do that.
> > 
> > I'm rather unhappy about this workflow - we've got quite a few 
> > apic bits in the x86 tree this cycle as well and need extra 
> > external interaction, not.
> > 
> > Which KVM tree commit caused this, could someone please give a 
> > lkml link or quote it here? It's not referenced in the fix patch 
> > either.
> > 
> > Thanks,
> > 
> > 	Ingo
> 
> This tree (kvm.git next):
> 
> http://git.kernel.org/?p=virt/kvm/kvm.git;a=shortlog;h=refs/heads/next
> 
> Introduced by this commit:
> 
> http://git.kernel.org/?p=virt/kvm/kvm.git;a=commit;h=ab9cf4996bb989983e73da894b8dd0239aa2c3c2

This bit:

> +	if (kvm_para_has_feature(KVM_FEATURE_PV_EOI)) {
> +		struct apic **drv;
> +
> +		for (drv = __apicdrivers; drv < __apicdrivers_end; drv++) {
> +			/* Should happen once for each apic */
> +			WARN_ON((*drv)->eoi_write == kvm_guest_apic_eoi_write);
> +			(*drv)->eoi_write = kvm_guest_apic_eoi_write;
> +		}
> +	}
> +

is rather disgusting I have to say.

WTH is the KVM code meddling with core x86 apic driver data 
structures directly? At minimum factor this out and create a 
proper apic.c function which is EXPORT_SYMBOL_GPL() exported or 
so...

Thanks,

	Ingo

Re: [PATCH] apic: fix kvm build on UP without IOAPIC

[Michael S. Tsirkin]

On Fri, Jul 06, 2012 at 04:12:23PM +0200, Ingo Molnar wrote:
> 
> * Marcelo Tosatti <mtosatti@redhat.com> wrote:
> 
> > On Fri, Jul 06, 2012 at 01:13:14PM +0200, Ingo Molnar wrote:
> > > 
> > > * H. Peter Anvin <hpa@zytor.com> wrote:
> > > 
> > > > On 07/01/2012 08:05 AM, Michael S. Tsirkin wrote:
> > > > >On UP i386, when APIC is disabled
> > > > ># CONFIG_X86_UP_APIC is not set
> > > > ># CONFIG_PCI_IOAPIC is not set
> > > > >
> > > > >code looking at apicdrivers never has any effect but it
> > > > >still gets compiled in. In particular, this causes
> > > > >build failures with kvm, but it generally bloats the kernel
> > > > >unnecessarily.
> > > > >
> > > > >Fix by defining both __apicdrivers and __apicdrivers_end
> > > > >to be NULL when CONFIG_X86_LOCAL_APIC is unset: I verified
> > > > >that as the result any loop scanning __apicdrivers gets optimized out by
> > > > >the compiler.
> > > > >
> > > > >Warning: a .config with apic disabled doesn't seem to boot
> > > > >for me (even without this patch). Still verifying why,
> > > > >meanwhile this patch is compile-tested only.
> > > > >
> > > > >Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
> > > > >---
> > > > >
> > > > >Note: if this patch makes sense, can x86 maintainers
> > > > >please ACK applying it through the kvm tree, since that is
> > > > >where we see the issue that it addresses?
> > > > >Avi, Marcelo, maybe you can carry this in kvm/linux-next as a temporary
> > > > >measure so that linux-next builds?
> > > > >
> > > > 
> > > > I'm not happy about that as a workflow, but since you guys have an
> > > > immediate problem I guess we can do that.
> > > 
> > > I'm rather unhappy about this workflow - we've got quite a few 
> > > apic bits in the x86 tree this cycle as well and need extra 
> > > external interaction, not.
> > > 
> > > Which KVM tree commit caused this, could someone please give a 
> > > lkml link or quote it here? It's not referenced in the fix patch 
> > > either.
> > > 
> > > Thanks,
> > > 
> > > 	Ingo
> > 
> > This tree (kvm.git next):
> > 
> > http://git.kernel.org/?p=virt/kvm/kvm.git;a=shortlog;h=refs/heads/next
> > 
> > Introduced by this commit:
> > 
> > http://git.kernel.org/?p=virt/kvm/kvm.git;a=commit;h=ab9cf4996bb989983e73da894b8dd0239aa2c3c2
> 
> This bit:
> 
> > +	if (kvm_para_has_feature(KVM_FEATURE_PV_EOI)) {
> > +		struct apic **drv;
> > +
> > +		for (drv = __apicdrivers; drv < __apicdrivers_end; drv++) {
> > +			/* Should happen once for each apic */
> > +			WARN_ON((*drv)->eoi_write == kvm_guest_apic_eoi_write);
> > +			(*drv)->eoi_write = kvm_guest_apic_eoi_write;
> > +		}
> > +	}
> > +
> 
> is rather disgusting I have to say.
> 
> WTH is the KVM code meddling with core x86 apic driver data 
> structures directly? At minimum factor this out and create a 
> proper apic.c function which is EXPORT_SYMBOL_GPL() exported or 
> so...
> 
> Thanks,
> 
> 	Ingo

OK, so apic_set_eoi_write()?

-- 
MST

Re: [PATCH] apic: fix kvm build on UP without IOAPIC

[Ingo Molnar]

* Michael S. Tsirkin <mst@redhat.com> wrote:

> On Fri, Jul 06, 2012 at 04:12:23PM +0200, Ingo Molnar wrote:
> > 
> > * Marcelo Tosatti <mtosatti@redhat.com> wrote:
> > 
> > > On Fri, Jul 06, 2012 at 01:13:14PM +0200, Ingo Molnar wrote:
> > > > 
> > > > * H. Peter Anvin <hpa@zytor.com> wrote:
> > > > 
> > > > > On 07/01/2012 08:05 AM, Michael S. Tsirkin wrote:
> > > > > >On UP i386, when APIC is disabled
> > > > > ># CONFIG_X86_UP_APIC is not set
> > > > > ># CONFIG_PCI_IOAPIC is not set
> > > > > >
> > > > > >code looking at apicdrivers never has any effect but it
> > > > > >still gets compiled in. In particular, this causes
> > > > > >build failures with kvm, but it generally bloats the kernel
> > > > > >unnecessarily.
> > > > > >
> > > > > >Fix by defining both __apicdrivers and __apicdrivers_end
> > > > > >to be NULL when CONFIG_X86_LOCAL_APIC is unset: I verified
> > > > > >that as the result any loop scanning __apicdrivers gets optimized out by
> > > > > >the compiler.
> > > > > >
> > > > > >Warning: a .config with apic disabled doesn't seem to boot
> > > > > >for me (even without this patch). Still verifying why,
> > > > > >meanwhile this patch is compile-tested only.
> > > > > >
> > > > > >Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
> > > > > >---
> > > > > >
> > > > > >Note: if this patch makes sense, can x86 maintainers
> > > > > >please ACK applying it through the kvm tree, since that is
> > > > > >where we see the issue that it addresses?
> > > > > >Avi, Marcelo, maybe you can carry this in kvm/linux-next as a temporary
> > > > > >measure so that linux-next builds?
> > > > > >
> > > > > 
> > > > > I'm not happy about that as a workflow, but since you guys have an
> > > > > immediate problem I guess we can do that.
> > > > 
> > > > I'm rather unhappy about this workflow - we've got quite a few 
> > > > apic bits in the x86 tree this cycle as well and need extra 
> > > > external interaction, not.
> > > > 
> > > > Which KVM tree commit caused this, could someone please give a 
> > > > lkml link or quote it here? It's not referenced in the fix patch 
> > > > either.
> > > > 
> > > > Thanks,
> > > > 
> > > > 	Ingo
> > > 
> > > This tree (kvm.git next):
> > > 
> > > http://git.kernel.org/?p=virt/kvm/kvm.git;a=shortlog;h=refs/heads/next
> > > 
> > > Introduced by this commit:
> > > 
> > > http://git.kernel.org/?p=virt/kvm/kvm.git;a=commit;h=ab9cf4996bb989983e73da894b8dd0239aa2c3c2
> > 
> > This bit:
> > 
> > > +	if (kvm_para_has_feature(KVM_FEATURE_PV_EOI)) {
> > > +		struct apic **drv;
> > > +
> > > +		for (drv = __apicdrivers; drv < __apicdrivers_end; drv++) {
> > > +			/* Should happen once for each apic */
> > > +			WARN_ON((*drv)->eoi_write == kvm_guest_apic_eoi_write);
> > > +			(*drv)->eoi_write = kvm_guest_apic_eoi_write;
> > > +		}
> > > +	}
> > > +
> > 
> > is rather disgusting I have to say.
> > 
> > WTH is the KVM code meddling with core x86 apic driver data 
> > structures directly? At minimum factor this out and create a 
> > proper apic.c function which is EXPORT_SYMBOL_GPL() exported or 
> > so...
> > 
> > Thanks,
> > 
> > 	Ingo
> 
> OK, so apic_set_eoi_write()?

Yes, with a changelog comment analyzing the design decisions and 
locking here - what happens if actual APIC driver use races with 
this update on SMP, why is it all safe, etc?

Thanks,

	Ingo