pci_get_subsys: GFP_KERNEL allocations with IRQs disabled

pci_get_subsys: GFP_KERNEL allocations with IRQs disabled

[Fengguang Wu]

On Tue, Jul 31, 2012 at 05:18:11PM -0700, Paul E. McKenney wrote:
> On Tue, Jul 31, 2012 at 08:09:38PM -0400, Steven Rostedt wrote:
> > On Tue, 2012-07-31 at 16:57 -0700, Paul E. McKenney wrote:
> > 
> > > > What was the next lines? I bet you it was "PASSED". Which means it did
> > > > not fail. This is the second bug you found that has to do with RCU being
> > > > called in 'idle'. The one that Paul posted a patch for.
> > > 
> > > Though it needs another patch to actually use it in the right place...
> > 
> > Right. Something like this:
> 
> Looks good to me!
 
With all 3 patches applied, the warning on __update_max_tr finally
goes away. Thanks!

However, this unrelated warning still reliably remains (the same config).
I think it's pci_get_subsys() triggered this assert:

        /*
         * Oi! Can't be having __GFP_FS allocations with IRQs disabled.
         */
        if (DEBUG_LOCKS_WARN_ON(irqs_disabled_flags(flags)))
                return;

[   91.282131] machine restart
[   91.283895] ------------[ cut here ]------------
[   91.284731] WARNING: at /c/wfg/linux/kernel/lockdep.c:2739 lockdep_trace_alloc+0x1fb/0x210()
[   91.286132] Modules linked in:
[   91.286703] Pid: 697, comm: reboot Not tainted 3.5.0-00024-g01ff5db-dirty #4
[   91.287859] Call Trace:
[   91.288289]  [<81050148>] warn_slowpath_common+0xb8/0x100
[   91.289338]  [<8110acdb>] ? lockdep_trace_alloc+0x1fb/0x210
[   91.290264]  [<8110acdb>] ? lockdep_trace_alloc+0x1fb/0x210
[   91.291161]  [<810501ce>] warn_slowpath_null+0x3e/0x50
[   91.292042]  [<8110acdb>] lockdep_trace_alloc+0x1fb/0x210
[   91.292934]  [<81228e25>] kmem_cache_alloc_trace+0x55/0x600
[   91.292934]  [<813025ca>] ? kobject_put+0x9a/0x160
[   91.292934]  [<814e95e0>] ? klist_iter_exit+0x30/0x50
[   91.292934]  [<81405881>] ? bus_find_device+0xf1/0x120
[   91.292934]  [<81361a3c>] ? pci_get_subsys+0x11c/0x1b0
[   91.292934]  [<81361a3c>] pci_get_subsys+0x11c/0x1b0
[   91.292934]  [<81361afe>] pci_get_device+0x2e/0x40
[   91.292934]  [<81033e25>] mach_reboot_fixups+0xa5/0xd0
[   91.292934]  [<81027611>] native_machine_emergency_restart+0x1f1/0x590
[   91.292934]  [<814f2e00>] ? printk+0x4b/0x5b
[   91.292934]  [<810269ef>] native_machine_restart+0x6f/0x80
[   91.292934]  [<810271cc>] machine_restart+0x1c/0x30
[   91.292934]  [<810886e0>] kernel_restart+0x70/0xc0
[   91.292934]  [<81088a85>] sys_reboot+0x325/0x380
[   91.292934]  [<811f796c>] ? handle_pte_fault+0xdc/0x1740
[   91.292934]  [<811f93e7>] ? handle_mm_fault+0x417/0x4a0
[   91.292934]  [<8103e07b>] ? do_page_fault+0x7fb/0xb30
[   91.292934]  [<810b33e7>] ? up_read+0x37/0x70
[   91.292934]  [<8103e07b>] ? do_page_fault+0x7fb/0xb30
[   91.292934]  [<8123c063>] ? do_sys_open+0x3a3/0x3f0
[   91.292934]  [<8123c063>] ? do_sys_open+0x3a3/0x3f0
[   91.292934]  [<810b0270>] ? update_rmtp+0xe0/0xe0
[   91.292934]  [<8150376e>] ? restore_all+0xf/0xf
[   91.292934]  [<8103d880>] ? vmalloc_sync_all+0x320/0x320
[   91.292934]  [<81109fca>] ? trace_hardirqs_on_caller+0x28a/0x380
[   91.292934]  [<81311594>] ? trace_hardirqs_on_thunk+0xc/0x10
[   91.292934]  [<81503735>] syscall_call+0x7/0xb

Thanks,
Fengguang

Re: pci_get_subsys: GFP_KERNEL allocations with IRQs disabled

[Fengguang Wu]

Feng,

> I think it's pci_get_subsys() triggered this assert:
> 
>         /*
>          * Oi! Can't be having __GFP_FS allocations with IRQs disabled.
>          */
>         if (DEBUG_LOCKS_WARN_ON(irqs_disabled_flags(flags)))
>                 return;

It's bisected down to this commit:

commit 55c844a4dd16a4d1fdc0cf2a283ec631a02ec448
Author:     Feng Tang <feng.tang@intel.com>
AuthorDate: Wed May 30 23:15:41 2012 +0800
Commit:     Ingo Molnar <mingo@kernel.org>
CommitDate: Wed Jun 6 12:03:23 2012 +0200

    x86/reboot: Fix a warning message triggered by stop_other_cpus()

Thanks,
Fengguang

> [   91.282131] machine restart
> [   91.283895] ------------[ cut here ]------------
> [   91.284731] WARNING: at /c/wfg/linux/kernel/lockdep.c:2739 lockdep_trace_alloc+0x1fb/0x210()
> [   91.286132] Modules linked in:
> [   91.286703] Pid: 697, comm: reboot Not tainted 3.5.0-00024-g01ff5db-dirty #4
> [   91.287859] Call Trace:
> [   91.288289]  [<81050148>] warn_slowpath_common+0xb8/0x100
> [   91.289338]  [<8110acdb>] ? lockdep_trace_alloc+0x1fb/0x210
> [   91.290264]  [<8110acdb>] ? lockdep_trace_alloc+0x1fb/0x210
> [   91.291161]  [<810501ce>] warn_slowpath_null+0x3e/0x50
> [   91.292042]  [<8110acdb>] lockdep_trace_alloc+0x1fb/0x210
> [   91.292934]  [<81228e25>] kmem_cache_alloc_trace+0x55/0x600
> [   91.292934]  [<813025ca>] ? kobject_put+0x9a/0x160
> [   91.292934]  [<814e95e0>] ? klist_iter_exit+0x30/0x50
> [   91.292934]  [<81405881>] ? bus_find_device+0xf1/0x120
> [   91.292934]  [<81361a3c>] ? pci_get_subsys+0x11c/0x1b0
> [   91.292934]  [<81361a3c>] pci_get_subsys+0x11c/0x1b0
> [   91.292934]  [<81361afe>] pci_get_device+0x2e/0x40
> [   91.292934]  [<81033e25>] mach_reboot_fixups+0xa5/0xd0
> [   91.292934]  [<81027611>] native_machine_emergency_restart+0x1f1/0x590
> [   91.292934]  [<814f2e00>] ? printk+0x4b/0x5b
> [   91.292934]  [<810269ef>] native_machine_restart+0x6f/0x80
> [   91.292934]  [<810271cc>] machine_restart+0x1c/0x30
> [   91.292934]  [<810886e0>] kernel_restart+0x70/0xc0
> [   91.292934]  [<81088a85>] sys_reboot+0x325/0x380
> [   91.292934]  [<811f796c>] ? handle_pte_fault+0xdc/0x1740
> [   91.292934]  [<811f93e7>] ? handle_mm_fault+0x417/0x4a0
> [   91.292934]  [<8103e07b>] ? do_page_fault+0x7fb/0xb30
> [   91.292934]  [<810b33e7>] ? up_read+0x37/0x70
> [   91.292934]  [<8103e07b>] ? do_page_fault+0x7fb/0xb30
> [   91.292934]  [<8123c063>] ? do_sys_open+0x3a3/0x3f0
> [   91.292934]  [<8123c063>] ? do_sys_open+0x3a3/0x3f0
> [   91.292934]  [<810b0270>] ? update_rmtp+0xe0/0xe0
> [   91.292934]  [<8150376e>] ? restore_all+0xf/0xf
> [   91.292934]  [<8103d880>] ? vmalloc_sync_all+0x320/0x320
> [   91.292934]  [<81109fca>] ? trace_hardirqs_on_caller+0x28a/0x380
> [   91.292934]  [<81311594>] ? trace_hardirqs_on_thunk+0xc/0x10
> [   91.292934]  [<81503735>] syscall_call+0x7/0xb
> 
> Thanks,
> Fengguang

Re: pci_get_subsys: GFP_KERNEL allocations with IRQs disabled

[Feng Tang]

Hi Fengguang,


On Wed, 22 Aug 2012 10:50:08 +0800
Fengguang Wu <fengguang.wu@intel.com> wrote:

> Feng,
> 
> > I think it's pci_get_subsys() triggered this assert:
> > 
> >         /*
> >          * Oi! Can't be having __GFP_FS allocations with IRQs disabled.
> >          */
> >         if (DEBUG_LOCKS_WARN_ON(irqs_disabled_flags(flags)))
> >                 return;
> 
> It's bisected down to this commit:
> 
> commit 55c844a4dd16a4d1fdc0cf2a283ec631a02ec448
> Author:     Feng Tang <feng.tang@intel.com>
> AuthorDate: Wed May 30 23:15:41 2012 +0800
> Commit:     Ingo Molnar <mingo@kernel.org>
> CommitDate: Wed Jun 6 12:03:23 2012 +0200
> 
>     x86/reboot: Fix a warning message triggered by stop_other_cpus()
> 
> Thanks,
> Fengguang

Thanks for the bisection.

Revert my commit should be a solution, but can we simply make the pci_device_id
a local on stack one instead of using sleepable kmalloc for it, as this
sounds fragile when pci_get_subsys get called in a late system reboot stage?

Thanks,
Feng

------------
diff --git a/drivers/pci/search.c b/drivers/pci/search.c
index 993d4a0..e5ccede 100644
--- a/drivers/pci/search.c
+++ b/drivers/pci/search.c
@@ -246,7 +246,7 @@ struct pci_dev *pci_get_subsys(unsigned int vendor, unsigned int device,
 			       struct pci_dev *from)
 {
 	struct pci_dev *pdev;
-	struct pci_device_id *id;
+	struct pci_device_id id;
 
 	/*
 	 * pci_find_subsys() can be called on the ide_setup() path,
@@ -257,17 +257,12 @@ struct pci_dev *pci_get_subsys(unsigned int vendor, unsigned int device,
 	if (unlikely(no_pci_devices()))
 		return NULL;
 
-	id = kzalloc(sizeof(*id), GFP_KERNEL);
-	if (!id)
-		return NULL;
-	id->vendor = vendor;
-	id->device = device;
-	id->subvendor = ss_vendor;
-	id->subdevice = ss_device;
-
-	pdev = pci_get_dev_by_id(id, from);
-	kfree(id);
+	id.vendor = vendor;
+	id.device = device;
+	id.subvendor = ss_vendor;
+	id.subdevice = ss_device;
 
+	pdev = pci_get_dev_by_id(&id, from);
 	return pdev;
 }
 


> 
> > [   91.282131] machine restart
> > [   91.283895] ------------[ cut here ]------------
> > [   91.284731] WARNING: at /c/wfg/linux/kernel/lockdep.c:2739 lockdep_trace_alloc+0x1fb/0x210()
> > [   91.286132] Modules linked in:
> > [   91.286703] Pid: 697, comm: reboot Not tainted 3.5.0-00024-g01ff5db-dirty #4
> > [   91.287859] Call Trace:
> > [   91.288289]  [<81050148>] warn_slowpath_common+0xb8/0x100
> > [   91.289338]  [<8110acdb>] ? lockdep_trace_alloc+0x1fb/0x210
> > [   91.290264]  [<8110acdb>] ? lockdep_trace_alloc+0x1fb/0x210
> > [   91.291161]  [<810501ce>] warn_slowpath_null+0x3e/0x50
> > [   91.292042]  [<8110acdb>] lockdep_trace_alloc+0x1fb/0x210
> > [   91.292934]  [<81228e25>] kmem_cache_alloc_trace+0x55/0x600
> > [   91.292934]  [<813025ca>] ? kobject_put+0x9a/0x160
> > [   91.292934]  [<814e95e0>] ? klist_iter_exit+0x30/0x50
> > [   91.292934]  [<81405881>] ? bus_find_device+0xf1/0x120
> > [   91.292934]  [<81361a3c>] ? pci_get_subsys+0x11c/0x1b0
> > [   91.292934]  [<81361a3c>] pci_get_subsys+0x11c/0x1b0
> > [   91.292934]  [<81361afe>] pci_get_device+0x2e/0x40
> > [   91.292934]  [<81033e25>] mach_reboot_fixups+0xa5/0xd0
> > [   91.292934]  [<81027611>] native_machine_emergency_restart+0x1f1/0x590
> > [   91.292934]  [<814f2e00>] ? printk+0x4b/0x5b
> > [   91.292934]  [<810269ef>] native_machine_restart+0x6f/0x80
> > [   91.292934]  [<810271cc>] machine_restart+0x1c/0x30
> > [   91.292934]  [<810886e0>] kernel_restart+0x70/0xc0
> > [   91.292934]  [<81088a85>] sys_reboot+0x325/0x380
> > [   91.292934]  [<811f796c>] ? handle_pte_fault+0xdc/0x1740
> > [   91.292934]  [<811f93e7>] ? handle_mm_fault+0x417/0x4a0
> > [   91.292934]  [<8103e07b>] ? do_page_fault+0x7fb/0xb30
> > [   91.292934]  [<810b33e7>] ? up_read+0x37/0x70
> > [   91.292934]  [<8103e07b>] ? do_page_fault+0x7fb/0xb30
> > [   91.292934]  [<8123c063>] ? do_sys_open+0x3a3/0x3f0
> > [   91.292934]  [<8123c063>] ? do_sys_open+0x3a3/0x3f0
> > [   91.292934]  [<810b0270>] ? update_rmtp+0xe0/0xe0
> > [   91.292934]  [<8150376e>] ? restore_all+0xf/0xf
> > [   91.292934]  [<8103d880>] ? vmalloc_sync_all+0x320/0x320
> > [   91.292934]  [<81109fca>] ? trace_hardirqs_on_caller+0x28a/0x380
> > [   91.292934]  [<81311594>] ? trace_hardirqs_on_thunk+0xc/0x10
> > [   91.292934]  [<81503735>] syscall_call+0x7/0xb
> > 
> > Thanks,
> > Fengguang

Re: pci_get_subsys: GFP_KERNEL allocations with IRQs disabled

[Fengguang Wu]

On Wed, Aug 22, 2012 at 03:49:08PM +0800, Tang, Feng wrote:
> Hi Fengguang,
> 
> 
> On Wed, 22 Aug 2012 10:50:08 +0800
> Fengguang Wu <fengguang.wu@intel.com> wrote:
> 
> > Feng,
> > 
> > > I think it's pci_get_subsys() triggered this assert:
> > > 
> > >         /*
> > >          * Oi! Can't be having __GFP_FS allocations with IRQs disabled.
> > >          */
> > >         if (DEBUG_LOCKS_WARN_ON(irqs_disabled_flags(flags)))
> > >                 return;
> > 
> > It's bisected down to this commit:
> > 
> > commit 55c844a4dd16a4d1fdc0cf2a283ec631a02ec448
> > Author:     Feng Tang <feng.tang@intel.com>
> > AuthorDate: Wed May 30 23:15:41 2012 +0800
> > Commit:     Ingo Molnar <mingo@kernel.org>
> > CommitDate: Wed Jun 6 12:03:23 2012 +0200
> > 
> >     x86/reboot: Fix a warning message triggered by stop_other_cpus()
> > 
> > Thanks,
> > Fengguang
> 
> Thanks for the bisection.
> 
> Revert my commit should be a solution, but can we simply make the pci_device_id
> a local on stack one instead of using sleepable kmalloc for it, as this
> sounds fragile when pci_get_subsys get called in a late system reboot stage?

Good idea! I like this simple solution. It will sure fix the warning.

Reviewed-by: Fengguang Wu <fengguang.wu@intel.com>

Thanks,
Fengguang

> ------------
> diff --git a/drivers/pci/search.c b/drivers/pci/search.c
> index 993d4a0..e5ccede 100644
> --- a/drivers/pci/search.c
> +++ b/drivers/pci/search.c
> @@ -246,7 +246,7 @@ struct pci_dev *pci_get_subsys(unsigned int vendor, unsigned int device,
>  			       struct pci_dev *from)
>  {
>  	struct pci_dev *pdev;
> -	struct pci_device_id *id;
> +	struct pci_device_id id;
>  
>  	/*
>  	 * pci_find_subsys() can be called on the ide_setup() path,
> @@ -257,17 +257,12 @@ struct pci_dev *pci_get_subsys(unsigned int vendor, unsigned int device,
>  	if (unlikely(no_pci_devices()))
>  		return NULL;
>  
> -	id = kzalloc(sizeof(*id), GFP_KERNEL);
> -	if (!id)
> -		return NULL;
> -	id->vendor = vendor;
> -	id->device = device;
> -	id->subvendor = ss_vendor;
> -	id->subdevice = ss_device;
> -
> -	pdev = pci_get_dev_by_id(id, from);
> -	kfree(id);
> +	id.vendor = vendor;
> +	id.device = device;
> +	id.subvendor = ss_vendor;
> +	id.subdevice = ss_device;
>  
> +	pdev = pci_get_dev_by_id(&id, from);
>  	return pdev;
>  }
>  
> 
> 
> > 
> > > [   91.282131] machine restart
> > > [   91.283895] ------------[ cut here ]------------
> > > [   91.284731] WARNING: at /c/wfg/linux/kernel/lockdep.c:2739 lockdep_trace_alloc+0x1fb/0x210()
> > > [   91.286132] Modules linked in:
> > > [   91.286703] Pid: 697, comm: reboot Not tainted 3.5.0-00024-g01ff5db-dirty #4
> > > [   91.287859] Call Trace:
> > > [   91.288289]  [<81050148>] warn_slowpath_common+0xb8/0x100
> > > [   91.289338]  [<8110acdb>] ? lockdep_trace_alloc+0x1fb/0x210
> > > [   91.290264]  [<8110acdb>] ? lockdep_trace_alloc+0x1fb/0x210
> > > [   91.291161]  [<810501ce>] warn_slowpath_null+0x3e/0x50
> > > [   91.292042]  [<8110acdb>] lockdep_trace_alloc+0x1fb/0x210
> > > [   91.292934]  [<81228e25>] kmem_cache_alloc_trace+0x55/0x600
> > > [   91.292934]  [<813025ca>] ? kobject_put+0x9a/0x160
> > > [   91.292934]  [<814e95e0>] ? klist_iter_exit+0x30/0x50
> > > [   91.292934]  [<81405881>] ? bus_find_device+0xf1/0x120
> > > [   91.292934]  [<81361a3c>] ? pci_get_subsys+0x11c/0x1b0
> > > [   91.292934]  [<81361a3c>] pci_get_subsys+0x11c/0x1b0
> > > [   91.292934]  [<81361afe>] pci_get_device+0x2e/0x40
> > > [   91.292934]  [<81033e25>] mach_reboot_fixups+0xa5/0xd0
> > > [   91.292934]  [<81027611>] native_machine_emergency_restart+0x1f1/0x590
> > > [   91.292934]  [<814f2e00>] ? printk+0x4b/0x5b
> > > [   91.292934]  [<810269ef>] native_machine_restart+0x6f/0x80
> > > [   91.292934]  [<810271cc>] machine_restart+0x1c/0x30
> > > [   91.292934]  [<810886e0>] kernel_restart+0x70/0xc0
> > > [   91.292934]  [<81088a85>] sys_reboot+0x325/0x380
> > > [   91.292934]  [<811f796c>] ? handle_pte_fault+0xdc/0x1740
> > > [   91.292934]  [<811f93e7>] ? handle_mm_fault+0x417/0x4a0
> > > [   91.292934]  [<8103e07b>] ? do_page_fault+0x7fb/0xb30
> > > [   91.292934]  [<810b33e7>] ? up_read+0x37/0x70
> > > [   91.292934]  [<8103e07b>] ? do_page_fault+0x7fb/0xb30
> > > [   91.292934]  [<8123c063>] ? do_sys_open+0x3a3/0x3f0
> > > [   91.292934]  [<8123c063>] ? do_sys_open+0x3a3/0x3f0
> > > [   91.292934]  [<810b0270>] ? update_rmtp+0xe0/0xe0
> > > [   91.292934]  [<8150376e>] ? restore_all+0xf/0xf
> > > [   91.292934]  [<8103d880>] ? vmalloc_sync_all+0x320/0x320
> > > [   91.292934]  [<81109fca>] ? trace_hardirqs_on_caller+0x28a/0x380
> > > [   91.292934]  [<81311594>] ? trace_hardirqs_on_thunk+0xc/0x10
> > > [   91.292934]  [<81503735>] syscall_call+0x7/0xb
> > > 
> > > Thanks,
> > > Fengguang

Re: pci_get_subsys: GFP_KERNEL allocations with IRQs disabled

[Bjorn Helgaas]

On Wed, Aug 22, 2012 at 12:49 AM, Feng Tang <feng.tang@intel.com> wrote:
> Hi Fengguang,
>
>
> On Wed, 22 Aug 2012 10:50:08 +0800
> Fengguang Wu <fengguang.wu@intel.com> wrote:
>
>> Feng,
>>
>> > I think it's pci_get_subsys() triggered this assert:
>> >
>> >         /*
>> >          * Oi! Can't be having __GFP_FS allocations with IRQs disabled.
>> >          */
>> >         if (DEBUG_LOCKS_WARN_ON(irqs_disabled_flags(flags)))
>> >                 return;
>>
>> It's bisected down to this commit:
>>
>> commit 55c844a4dd16a4d1fdc0cf2a283ec631a02ec448
>> Author:     Feng Tang <feng.tang@intel.com>
>> AuthorDate: Wed May 30 23:15:41 2012 +0800
>> Commit:     Ingo Molnar <mingo@kernel.org>
>> CommitDate: Wed Jun 6 12:03:23 2012 +0200
>>
>>     x86/reboot: Fix a warning message triggered by stop_other_cpus()
>>
>> Thanks,
>> Fengguang
>
> Thanks for the bisection.
>
> Revert my commit should be a solution, but can we simply make the pci_device_id
> a local on stack one instead of using sleepable kmalloc for it, as this
> sounds fragile when pci_get_subsys get called in a late system reboot stage?

I think this is a great idea.  Can you make this a real patch, with a
changelog and Signed-off-by?

We should also remove the obsolete comment about early boot.  I'm not
sure the no_pci_devices() check is needed, either.  And we can make
the same simplification in pci_get_class().

> ------------
> diff --git a/drivers/pci/search.c b/drivers/pci/search.c
> index 993d4a0..e5ccede 100644
> --- a/drivers/pci/search.c
> +++ b/drivers/pci/search.c
> @@ -246,7 +246,7 @@ struct pci_dev *pci_get_subsys(unsigned int vendor, unsigned int device,
>                                struct pci_dev *from)
>  {
>         struct pci_dev *pdev;
> -       struct pci_device_id *id;
> +       struct pci_device_id id;
>
>         /*
>          * pci_find_subsys() can be called on the ide_setup() path,
> @@ -257,17 +257,12 @@ struct pci_dev *pci_get_subsys(unsigned int vendor, unsigned int device,
>         if (unlikely(no_pci_devices()))
>                 return NULL;
>
> -       id = kzalloc(sizeof(*id), GFP_KERNEL);
> -       if (!id)
> -               return NULL;
> -       id->vendor = vendor;
> -       id->device = device;
> -       id->subvendor = ss_vendor;
> -       id->subdevice = ss_device;
> -
> -       pdev = pci_get_dev_by_id(id, from);
> -       kfree(id);
> +       id.vendor = vendor;
> +       id.device = device;
> +       id.subvendor = ss_vendor;
> +       id.subdevice = ss_device;
>
> +       pdev = pci_get_dev_by_id(&id, from);

No need for "pdev" here, since we don't have to free anything.

>         return pdev;
>  }

Re: pci_get_subsys: GFP_KERNEL allocations with IRQs disabled

[Feng Tang]

Hi Bjorn,

On Wed, 22 Aug 2012 11:02:52 -0700
Bjorn Helgaas <bhelgaas@google.com> wrote:

> On Wed, Aug 22, 2012 at 12:49 AM, Feng Tang <feng.tang@intel.com> wrote:
> > Hi Fengguang,
> >
> >
> > On Wed, 22 Aug 2012 10:50:08 +0800
> > Fengguang Wu <fengguang.wu@intel.com> wrote:
> >
> >> Feng,
> >>
> >> > I think it's pci_get_subsys() triggered this assert:
> >> >
> >> >         /*
> >> >          * Oi! Can't be having __GFP_FS allocations with IRQs disabled.
> >> >          */
> >> >         if (DEBUG_LOCKS_WARN_ON(irqs_disabled_flags(flags)))
> >> >                 return;
> >>
> >> It's bisected down to this commit:
> >>
> >> commit 55c844a4dd16a4d1fdc0cf2a283ec631a02ec448
> >> Author:     Feng Tang <feng.tang@intel.com>
> >> AuthorDate: Wed May 30 23:15:41 2012 +0800
> >> Commit:     Ingo Molnar <mingo@kernel.org>
> >> CommitDate: Wed Jun 6 12:03:23 2012 +0200
> >>
> >>     x86/reboot: Fix a warning message triggered by stop_other_cpus()
> >>
> >> Thanks,
> >> Fengguang
> >
> > Thanks for the bisection.
> >
> > Revert my commit should be a solution, but can we simply make the pci_device_id
> > a local on stack one instead of using sleepable kmalloc for it, as this
> > sounds fragile when pci_get_subsys get called in a late system reboot stage?
> 
> I think this is a great idea.  Can you make this a real patch, with a
> changelog and Signed-off-by?

Thanks and will do.

> 
> We should also remove the obsolete comment about early boot.  I'm not
> sure the no_pci_devices() check is needed, either.  And we can make
> the same simplification in pci_get_class().

Will check the no_pci_devices() part, and try to make it a separate
patch for easy reverting in case of error.

> 
> > ------------
> > diff --git a/drivers/pci/search.c b/drivers/pci/search.c
> > index 993d4a0..e5ccede 100644
> > --- a/drivers/pci/search.c
> > +++ b/drivers/pci/search.c
> > @@ -246,7 +246,7 @@ struct pci_dev *pci_get_subsys(unsigned int vendor, unsigned int device,
> >                                struct pci_dev *from)
> >  {
> > +       id.vendor = vendor;
> > +       id.device = device;
> > +       id.subvendor = ss_vendor;
> > +       id.subdevice = ss_device;
> >
> > +       pdev = pci_get_dev_by_id(&id, from);
> 
> No need for "pdev" here, since we don't have to free anything.

ok, will directly return it.

Thanks,
Feng

[PATCH 1/2] PCI: Use local parameter pci_device_id for pci_get_subsys/class()

[Feng Tang]

>From 57a28ee5e7662ca28ba4c793aa037d64bd082dee Mon Sep 17 00:00:00 2001
From: Feng Tang <feng.tang@intel.com>
Date: Wed, 22 Aug 2012 15:41:51 +0800
Subject: [PATCH 1/2] PCI: Use local parameter pci_device_id for pci_get_subsys/class()

This fixes a kernel warning https://lkml.org/lkml/2012/7/31/682

pci_get_subsys() may get called in late system reboot stage, using
a sleepable kmalloc() sounds fragile and will casue a kernel warning
with my recent commmit 55c844a "x86/reboot: Fix a warning message
triggered by stop_other_cpus()" which disable local interrupt in
late system shutdown/reboot phase. Using a local parameter instead
will fix it and make it eligible for calling forom atomic context.

Do the same change for the pci_get_class() as suggeted by Bjorn Helgaas

Bisected-by: Fengguang Wu <fengguang.wu@intel.com>
Signed-off-by: Feng Tang <feng.tang@intel.com>
Reviewed-by: Fengguang Wu <fengguang.wu@intel.com>
---
 drivers/pci/search.c |   35 +++++++++++------------------------
 1 files changed, 11 insertions(+), 24 deletions(-)

diff --git a/drivers/pci/search.c b/drivers/pci/search.c
index 993d4a0..78a08b1 100644
--- a/drivers/pci/search.c
+++ b/drivers/pci/search.c
@@ -245,8 +245,7 @@ struct pci_dev *pci_get_subsys(unsigned int vendor, unsigned int device,
 			       unsigned int ss_vendor, unsigned int ss_device,
 			       struct pci_dev *from)
 {
-	struct pci_dev *pdev;
-	struct pci_device_id *id;
+	struct pci_device_id id;
 
 	/*
 	 * pci_find_subsys() can be called on the ide_setup() path,
@@ -257,18 +256,12 @@ struct pci_dev *pci_get_subsys(unsigned int vendor, unsigned int device,
 	if (unlikely(no_pci_devices()))
 		return NULL;
 
-	id = kzalloc(sizeof(*id), GFP_KERNEL);
-	if (!id)
-		return NULL;
-	id->vendor = vendor;
-	id->device = device;
-	id->subvendor = ss_vendor;
-	id->subdevice = ss_device;
-
-	pdev = pci_get_dev_by_id(id, from);
-	kfree(id);
+	id.vendor = vendor;
+	id.device = device;
+	id.subvendor = ss_vendor;
+	id.subdevice = ss_device;
 
-	return pdev;
+	return pci_get_dev_by_id(&id, from);
 }
 
 /**
@@ -307,19 +300,13 @@ pci_get_device(unsigned int vendor, unsigned int device, struct pci_dev *from)
  */
 struct pci_dev *pci_get_class(unsigned int class, struct pci_dev *from)
 {
-	struct pci_dev *dev;
-	struct pci_device_id *id;
+	struct pci_device_id id;
 
-	id = kzalloc(sizeof(*id), GFP_KERNEL);
-	if (!id)
-		return NULL;
-	id->vendor = id->device = id->subvendor = id->subdevice = PCI_ANY_ID;
-	id->class_mask = PCI_ANY_ID;
-	id->class = class;
+	id.vendor = id.device = id.subvendor = id.subdevice = PCI_ANY_ID;
+	id.class_mask = PCI_ANY_ID;
+	id.class = class;
 
-	dev = pci_get_dev_by_id(id, from);
-	kfree(id);
-	return dev;
+	return pci_get_dev_by_id(&id, from);
 }
 
 /**
-- 
1.7.1

Re: [PATCH 1/2] PCI: Use local parameter pci_device_id for pci_get_subsys/class()

[Yinghai Lu]

On Thu, Aug 23, 2012 at 12:45 AM, Feng Tang <feng.tang@intel.com> wrote:
> From 57a28ee5e7662ca28ba4c793aa037d64bd082dee Mon Sep 17 00:00:00 2001
> From: Feng Tang <feng.tang@intel.com>
> Date: Wed, 22 Aug 2012 15:41:51 +0800
> Subject: [PATCH 1/2] PCI: Use local parameter pci_device_id for pci_get_subsys/class()
>
> This fixes a kernel warning https://lkml.org/lkml/2012/7/31/682
>
> pci_get_subsys() may get called in late system reboot stage, using
> a sleepable kmalloc() sounds fragile and will casue a kernel warning
> with my recent commmit 55c844a "x86/reboot: Fix a warning message
> triggered by stop_other_cpus()" which disable local interrupt in
> late system shutdown/reboot phase. Using a local parameter instead
> will fix it and make it eligible for calling forom atomic context.
>
> Do the same change for the pci_get_class() as suggeted by Bjorn Helgaas
>
> Bisected-by: Fengguang Wu <fengguang.wu@intel.com>
> Signed-off-by: Feng Tang <feng.tang@intel.com>
> Reviewed-by: Fengguang Wu <fengguang.wu@intel.com>
> ---
>  drivers/pci/search.c |   35 +++++++++++------------------------
>  1 files changed, 11 insertions(+), 24 deletions(-)
>
> diff --git a/drivers/pci/search.c b/drivers/pci/search.c
> index 993d4a0..78a08b1 100644
> --- a/drivers/pci/search.c
> +++ b/drivers/pci/search.c
> @@ -245,8 +245,7 @@ struct pci_dev *pci_get_subsys(unsigned int vendor, unsigned int device,
>                                unsigned int ss_vendor, unsigned int ss_device,
>                                struct pci_dev *from)
>  {
> -       struct pci_dev *pdev;
> -       struct pci_device_id *id;
> +       struct pci_device_id id;
>
>         /*
>          * pci_find_subsys() can be called on the ide_setup() path,
> @@ -257,18 +256,12 @@ struct pci_dev *pci_get_subsys(unsigned int vendor, unsigned int device,
>         if (unlikely(no_pci_devices()))
>                 return NULL;
>
> -       id = kzalloc(sizeof(*id), GFP_KERNEL);
> -       if (!id)
> -               return NULL;
> -       id->vendor = vendor;
> -       id->device = device;
> -       id->subvendor = ss_vendor;
> -       id->subdevice = ss_device;
> -
> -       pdev = pci_get_dev_by_id(id, from);
> -       kfree(id);
> +       id.vendor = vendor;
> +       id.device = device;
> +       id.subvendor = ss_vendor;
> +       id.subdevice = ss_device;
>
> -       return pdev;
> +       return pci_get_dev_by_id(&id, from);
>  }
>
>  /**
> @@ -307,19 +300,13 @@ pci_get_device(unsigned int vendor, unsigned int device, struct pci_dev *from)
>   */
>  struct pci_dev *pci_get_class(unsigned int class, struct pci_dev *from)
>  {
> -       struct pci_dev *dev;
> -       struct pci_device_id *id;
> +       struct pci_device_id id;
>
> -       id = kzalloc(sizeof(*id), GFP_KERNEL);
> -       if (!id)
> -               return NULL;
> -       id->vendor = id->device = id->subvendor = id->subdevice = PCI_ANY_ID;
> -       id->class_mask = PCI_ANY_ID;
> -       id->class = class;
> +       id.vendor = id.device = id.subvendor = id.subdevice = PCI_ANY_ID;
> +       id.class_mask = PCI_ANY_ID;
> +       id.class = class;
>
> -       dev = pci_get_dev_by_id(id, from);
> -       kfree(id);
> -       return dev;
> +       return pci_get_dev_by_id(&id, from);
>  }
>
>  /**

with this one in pci/next pci config in /sys are not created.

10:~ # lspci -tv
pcilib: Cannot open /sys/bus/pci/devices/0000:00:03.0/config
lspci: Unable to read the standard configuration space header of
device 0000:00:03.0
pcilib: Cannot open /sys/bus/pci/devices/0000:00:02.0/config
lspci: Unable to read the standard configuration space header of
device 0000:00:02.0
pcilib: Cannot open /sys/bus/pci/devices/0000:00:01.3/config
lspci: Unable to read the standard configuration space header of
device 0000:00:01.3
pcilib: Cannot open /sys/bus/pci/devices/0000:00:01.1/config
lspci: Unable to read the standard configuration space header of
device 0000:00:01.1
pcilib: Cannot open /sys/bus/pci/devices/0000:00:01.0/config
lspci: Unable to read the standard configuration space header of
device 0000:00:01.0
pcilib: Cannot open /sys/bus/pci/devices/0000:00:00.0/config
lspci: Unable to read the standard configuration space header of
device 0000:00:00.0
-[0000:00]-

bisected to this commit

ccee7d23102f5e5765ec24779c5b77472af8f79e is the first bad commit
commit ccee7d23102f5e5765ec24779c5b77472af8f79e
Author: Feng Tang <feng.tang@intel.com>
Date:   Thu Aug 23 15:45:03 2012 +0800

    PCI: Use pci_device_id on stack for pci_get_subsys/class() to avoid kmalloc

    This fixes a kernel warning https://lkml.org/lkml/2012/7/31/682

    pci_get_subsys() may get called in late system reboot stage, using
    a sleepable kmalloc() sounds fragile and will cause a kernel warning
    with my recent commmit 55c844a "x86/reboot: Fix a warning message
    triggered by stop_other_cpus()" which disable local interrupt in
    late system shutdown/reboot phase. Using a local parameter instead
    will fix it and make it eligible for calling forom atomic context.

    Do the same change for the pci_get_class() as suggested by Bjorn Helgaas

    [bhelgaas: changelog]
    Bisected-by: Fengguang Wu <fengguang.wu@intel.com>
    Signed-off-by: Feng Tang <feng.tang@intel.com>
    Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
    Reviewed-by: Fengguang Wu <fengguang.wu@intel.com>

:040000 040000 dee62a035816b73abc68e40de8f21c7349efc4cb
70b2a6258bffa1ab963bd650d8f5d02da774fbce M	drivers

so the stack get overrun ?

Bjorn, I think it is this one that cause lspci broken that I mentioned
during meeting at San Diego.

Thanks

Yinghai

Re: [PATCH 1/2] PCI: Use local parameter pci_device_id for pci_get_subsys/class()

[Yinghai Lu]

On Fri, Sep 7, 2012 at 6:00 PM, Yinghai Lu <yinghai@kernel.org> wrote:
> On Thu, Aug 23, 2012 at 12:45 AM, Feng Tang <feng.tang@intel.com> wrote:
>> From 57a28ee5e7662ca28ba4c793aa037d64bd082dee Mon Sep 17 00:00:00 2001
>> From: Feng Tang <feng.tang@intel.com>
>> Date: Wed, 22 Aug 2012 15:41:51 +0800
>> Subject: [PATCH 1/2] PCI: Use local parameter pci_device_id for pci_get_subsys/class()
>>
>> This fixes a kernel warning https://lkml.org/lkml/2012/7/31/682
>>
>> pci_get_subsys() may get called in late system reboot stage, using
>> a sleepable kmalloc() sounds fragile and will casue a kernel warning
>> with my recent commmit 55c844a "x86/reboot: Fix a warning message
>> triggered by stop_other_cpus()" which disable local interrupt in
>> late system shutdown/reboot phase. Using a local parameter instead
>> will fix it and make it eligible for calling forom atomic context.
>>
>> Do the same change for the pci_get_class() as suggeted by Bjorn Helgaas
>>
>> Bisected-by: Fengguang Wu <fengguang.wu@intel.com>
>> Signed-off-by: Feng Tang <feng.tang@intel.com>
>> Reviewed-by: Fengguang Wu <fengguang.wu@intel.com>
>> ---
>>  drivers/pci/search.c |   35 +++++++++++------------------------
>>  1 files changed, 11 insertions(+), 24 deletions(-)
>>
>> diff --git a/drivers/pci/search.c b/drivers/pci/search.c
>> index 993d4a0..78a08b1 100644
>> --- a/drivers/pci/search.c
>> +++ b/drivers/pci/search.c
>> @@ -245,8 +245,7 @@ struct pci_dev *pci_get_subsys(unsigned int vendor, unsigned int device,
>>                                unsigned int ss_vendor, unsigned int ss_device,
>>                                struct pci_dev *from)
>>  {
>> -       struct pci_dev *pdev;
>> -       struct pci_device_id *id;
>> +       struct pci_device_id id;
>>
>>         /*
>>          * pci_find_subsys() can be called on the ide_setup() path,
>> @@ -257,18 +256,12 @@ struct pci_dev *pci_get_subsys(unsigned int vendor, unsigned int device,
>>         if (unlikely(no_pci_devices()))
>>                 return NULL;
>>
>> -       id = kzalloc(sizeof(*id), GFP_KERNEL);
>> -       if (!id)
>> -               return NULL;
>> -       id->vendor = vendor;
>> -       id->device = device;
>> -       id->subvendor = ss_vendor;
>> -       id->subdevice = ss_device;
>> -
>> -       pdev = pci_get_dev_by_id(id, from);
>> -       kfree(id);
>> +       id.vendor = vendor;
>> +       id.device = device;
>> +       id.subvendor = ss_vendor;
>> +       id.subdevice = ss_device;
>>
>> -       return pdev;
>> +       return pci_get_dev_by_id(&id, from);
>>  }
>>
>>  /**
>> @@ -307,19 +300,13 @@ pci_get_device(unsigned int vendor, unsigned int device, struct pci_dev *from)
>>   */
>>  struct pci_dev *pci_get_class(unsigned int class, struct pci_dev *from)
>>  {
>> -       struct pci_dev *dev;
>> -       struct pci_device_id *id;
>> +       struct pci_device_id id;
>>
>> -       id = kzalloc(sizeof(*id), GFP_KERNEL);
>> -       if (!id)
>> -               return NULL;
>> -       id->vendor = id->device = id->subvendor = id->subdevice = PCI_ANY_ID;
>> -       id->class_mask = PCI_ANY_ID;
>> -       id->class = class;
>> +       id.vendor = id.device = id.subvendor = id.subdevice = PCI_ANY_ID;
>> +       id.class_mask = PCI_ANY_ID;
>> +       id.class = class;
>>
>> -       dev = pci_get_dev_by_id(id, from);
>> -       kfree(id);
>> -       return dev;
>> +       return pci_get_dev_by_id(&id, from);
>>  }
>>
>>  /**
>
> with this one in pci/next pci config in /sys are not created.
>
> 10:~ # lspci -tv
> pcilib: Cannot open /sys/bus/pci/devices/0000:00:03.0/config
> lspci: Unable to read the standard configuration space header of
> device 0000:00:03.0
> pcilib: Cannot open /sys/bus/pci/devices/0000:00:02.0/config
> lspci: Unable to read the standard configuration space header of
> device 0000:00:02.0
> pcilib: Cannot open /sys/bus/pci/devices/0000:00:01.3/config
> lspci: Unable to read the standard configuration space header of
> device 0000:00:01.3
> pcilib: Cannot open /sys/bus/pci/devices/0000:00:01.1/config
> lspci: Unable to read the standard configuration space header of
> device 0000:00:01.1
> pcilib: Cannot open /sys/bus/pci/devices/0000:00:01.0/config
> lspci: Unable to read the standard configuration space header of
> device 0000:00:01.0
> pcilib: Cannot open /sys/bus/pci/devices/0000:00:00.0/config
> lspci: Unable to read the standard configuration space header of
> device 0000:00:00.0
> -[0000:00]-
>
> bisected to this commit
>
> ccee7d23102f5e5765ec24779c5b77472af8f79e is the first bad commit
> commit ccee7d23102f5e5765ec24779c5b77472af8f79e
> Author: Feng Tang <feng.tang@intel.com>
> Date:   Thu Aug 23 15:45:03 2012 +0800
>
>     PCI: Use pci_device_id on stack for pci_get_subsys/class() to avoid kmalloc
>
>     This fixes a kernel warning https://lkml.org/lkml/2012/7/31/682
>
>     pci_get_subsys() may get called in late system reboot stage, using
>     a sleepable kmalloc() sounds fragile and will cause a kernel warning
>     with my recent commmit 55c844a "x86/reboot: Fix a warning message
>     triggered by stop_other_cpus()" which disable local interrupt in
>     late system shutdown/reboot phase. Using a local parameter instead
>     will fix it and make it eligible for calling forom atomic context.
>
>     Do the same change for the pci_get_class() as suggested by Bjorn Helgaas
>
>     [bhelgaas: changelog]
>     Bisected-by: Fengguang Wu <fengguang.wu@intel.com>
>     Signed-off-by: Feng Tang <feng.tang@intel.com>
>     Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
>     Reviewed-by: Fengguang Wu <fengguang.wu@intel.com>
>
> :040000 040000 dee62a035816b73abc68e40de8f21c7349efc4cb
> 70b2a6258bffa1ab963bd650d8f5d02da774fbce M      drivers
>
> so the stack get overrun ?
>
> Bjorn, I think it is this one that cause lspci broken that I mentioned
> during meeting at San Diego.
>

Greg,

Any reason for using kmalloc instead of local variable during your
rewriting pci search code?

commit 95247b57ed844511a212265b45cf9a919753aea1
Author: Greg Kroah-Hartman <gregkh@suse.de>
Date:   Wed Feb 13 11:03:58 2008 -0800

    PCI: clean up search.c a lot

    This cleans up the search.c file, now using the pci list of devices that
    are created for the driver core, instead of relying on our separate list
    of devices.  It's better to use the functions already created for this
    kind of thing, instead of rolling our own all the time.

    This work is done in anticipation of getting rid of that second list of
    pci devices all together.

    And it ends up saving code, always a nice benefit.

    This also removes one compiler warning for when CONFIG_PCI_LEGACY is
    enabled as we no longer internally use the deprecated functions anymore.

    Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>

Re: [PATCH 1/2] PCI: Use local parameter pci_device_id for pci_get_subsys/class()

[Greg Kroah-Hartman]

On Fri, Sep 07, 2012 at 06:32:48PM -0700, Yinghai Lu wrote:
> Greg,
> 
> Any reason for using kmalloc instead of local variable during your
> rewriting pci search code?
> 
> commit 95247b57ed844511a212265b45cf9a919753aea1
> Author: Greg Kroah-Hartman <gregkh@suse.de>
> Date:   Wed Feb 13 11:03:58 2008 -0800

Seriously?  Something I wrote 4 years ago?  I really have no idea,
sorry.

greg k-h

Re: [PATCH 1/2] PCI: Use local parameter pci_device_id for pci_get_subsys/class()

[Fengguang Wu]

On Fri, Sep 07, 2012 at 06:32:48PM -0700, Yinghai Lu wrote:

> > with this one in pci/next pci config in /sys are not created.
> >
> > 10:~ # lspci -tv
> > pcilib: Cannot open /sys/bus/pci/devices/0000:00:03.0/config
> > lspci: Unable to read the standard configuration space header of
> > device 0000:00:03.0
> > pcilib: Cannot open /sys/bus/pci/devices/0000:00:02.0/config
> > lspci: Unable to read the standard configuration space header of
> > device 0000:00:02.0
> > pcilib: Cannot open /sys/bus/pci/devices/0000:00:01.3/config
> > lspci: Unable to read the standard configuration space header of
> > device 0000:00:01.3
> > pcilib: Cannot open /sys/bus/pci/devices/0000:00:01.1/config
> > lspci: Unable to read the standard configuration space header of
> > device 0000:00:01.1
> > pcilib: Cannot open /sys/bus/pci/devices/0000:00:01.0/config
> > lspci: Unable to read the standard configuration space header of
> > device 0000:00:01.0
> > pcilib: Cannot open /sys/bus/pci/devices/0000:00:00.0/config
> > lspci: Unable to read the standard configuration space header of
> > device 0000:00:00.0
> > -[0000:00]-
> >
> > bisected to this commit
> >
> > ccee7d23102f5e5765ec24779c5b77472af8f79e is the first bad commit
> > commit ccee7d23102f5e5765ec24779c5b77472af8f79e
> > Author: Feng Tang <feng.tang@intel.com>
> > Date:   Thu Aug 23 15:45:03 2012 +0800
> >
> >     PCI: Use pci_device_id on stack for pci_get_subsys/class() to avoid kmalloc
> >
> >     This fixes a kernel warning https://lkml.org/lkml/2012/7/31/682
> >
> >     pci_get_subsys() may get called in late system reboot stage, using
> >     a sleepable kmalloc() sounds fragile and will cause a kernel warning
> >     with my recent commmit 55c844a "x86/reboot: Fix a warning message
> >     triggered by stop_other_cpus()" which disable local interrupt in
> >     late system shutdown/reboot phase. Using a local parameter instead
> >     will fix it and make it eligible for calling forom atomic context.
> >
> >     Do the same change for the pci_get_class() as suggested by Bjorn Helgaas
> >
> >     [bhelgaas: changelog]
> >     Bisected-by: Fengguang Wu <fengguang.wu@intel.com>
> >     Signed-off-by: Feng Tang <feng.tang@intel.com>
> >     Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
> >     Reviewed-by: Fengguang Wu <fengguang.wu@intel.com>
> >
> > :040000 040000 dee62a035816b73abc68e40de8f21c7349efc4cb
> > 70b2a6258bffa1ab963bd650d8f5d02da774fbce M      drivers
> >
> > so the stack get overrun ?
> >
> > Bjorn, I think it is this one that cause lspci broken that I mentioned
> > during meeting at San Diego.

This makes lspci work again on my side. The caveat is, kzalloc() will
zero out all data while the new local variable leaves some data
uninitialized.

diff --git a/drivers/pci/search.c b/drivers/pci/search.c
index 78a08b1..9148b6e 100644
--- a/drivers/pci/search.c
+++ b/drivers/pci/search.c
@@ -245,7 +245,12 @@ struct pci_dev *pci_get_subsys(unsigned int vendor, unsigned int device,
 			       unsigned int ss_vendor, unsigned int ss_device,
 			       struct pci_dev *from)
 {
-	struct pci_device_id id;
+	struct pci_device_id id = {
+		.vendor = vendor,
+		.device = device,
+		.subvendor = ss_vendor,
+		.subdevice = ss_device,
+	};
 
 	/*
 	 * pci_find_subsys() can be called on the ide_setup() path,
@@ -256,11 +261,6 @@ struct pci_dev *pci_get_subsys(unsigned int vendor, unsigned int device,
 	if (unlikely(no_pci_devices()))
 		return NULL;
 
-	id.vendor = vendor;
-	id.device = device;
-	id.subvendor = ss_vendor;
-	id.subdevice = ss_device;
-
 	return pci_get_dev_by_id(&id, from);
 }
 
@@ -300,11 +300,14 @@ pci_get_device(unsigned int vendor, unsigned int device, struct pci_dev *from)
  */
 struct pci_dev *pci_get_class(unsigned int class, struct pci_dev *from)
 {
-	struct pci_device_id id;
-
-	id.vendor = id.device = id.subvendor = id.subdevice = PCI_ANY_ID;
-	id.class_mask = PCI_ANY_ID;
-	id.class = class;
+	struct pci_device_id id = {
+		.vendor = PCI_ANY_ID,
+		.device = PCI_ANY_ID,
+		.subvendor = PCI_ANY_ID,
+		.subdevice = PCI_ANY_ID,
+		.class_mask = PCI_ANY_ID,
+		.class = class,
+	};
 
 	return pci_get_dev_by_id(&id, from);
 }

Re: [PATCH 1/2] PCI: Use local parameter pci_device_id for pci_get_subsys/class()

[Yinghai Lu]

On Sat, Sep 8, 2012 at 6:42 AM, Fengguang Wu <fengguang.wu@intel.com> wrote:
> On Fri, Sep 07, 2012 at 06:32:48PM -0700, Yinghai Lu wrote:
>
>> > with this one in pci/next pci config in /sys are not created.
>> >
>> > 10:~ # lspci -tv
>> > pcilib: Cannot open /sys/bus/pci/devices/0000:00:03.0/config
>> > lspci: Unable to read the standard configuration space header of
>> > device 0000:00:03.0
>> > pcilib: Cannot open /sys/bus/pci/devices/0000:00:02.0/config
>> > lspci: Unable to read the standard configuration space header of
>> > device 0000:00:02.0
>> > pcilib: Cannot open /sys/bus/pci/devices/0000:00:01.3/config
>> > lspci: Unable to read the standard configuration space header of
>> > device 0000:00:01.3
>> > pcilib: Cannot open /sys/bus/pci/devices/0000:00:01.1/config
>> > lspci: Unable to read the standard configuration space header of
>> > device 0000:00:01.1
>> > pcilib: Cannot open /sys/bus/pci/devices/0000:00:01.0/config
>> > lspci: Unable to read the standard configuration space header of
>> > device 0000:00:01.0
>> > pcilib: Cannot open /sys/bus/pci/devices/0000:00:00.0/config
>> > lspci: Unable to read the standard configuration space header of
>> > device 0000:00:00.0
>> > -[0000:00]-
>> >
>> > bisected to this commit
>> >
>> > ccee7d23102f5e5765ec24779c5b77472af8f79e is the first bad commit
>> > commit ccee7d23102f5e5765ec24779c5b77472af8f79e
>> > Author: Feng Tang <feng.tang@intel.com>
>> > Date:   Thu Aug 23 15:45:03 2012 +0800
>> >
>> >     PCI: Use pci_device_id on stack for pci_get_subsys/class() to avoid kmalloc
>> >
>> >     This fixes a kernel warning https://lkml.org/lkml/2012/7/31/682
>> >
>> >     pci_get_subsys() may get called in late system reboot stage, using
>> >     a sleepable kmalloc() sounds fragile and will cause a kernel warning
>> >     with my recent commmit 55c844a "x86/reboot: Fix a warning message
>> >     triggered by stop_other_cpus()" which disable local interrupt in
>> >     late system shutdown/reboot phase. Using a local parameter instead
>> >     will fix it and make it eligible for calling forom atomic context.
>> >
>> >     Do the same change for the pci_get_class() as suggested by Bjorn Helgaas
>> >
>> >     [bhelgaas: changelog]
>> >     Bisected-by: Fengguang Wu <fengguang.wu@intel.com>
>> >     Signed-off-by: Feng Tang <feng.tang@intel.com>
>> >     Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
>> >     Reviewed-by: Fengguang Wu <fengguang.wu@intel.com>
>> >
>> > :040000 040000 dee62a035816b73abc68e40de8f21c7349efc4cb
>> > 70b2a6258bffa1ab963bd650d8f5d02da774fbce M      drivers
>> >
>> > so the stack get overrun ?
>> >
>> > Bjorn, I think it is this one that cause lspci broken that I mentioned
>> > during meeting at San Diego.
>
> This makes lspci work again on my side. The caveat is, kzalloc() will
> zero out all data while the new local variable leaves some data
> uninitialized.
>
> diff --git a/drivers/pci/search.c b/drivers/pci/search.c
> index 78a08b1..9148b6e 100644
> --- a/drivers/pci/search.c
> +++ b/drivers/pci/search.c
> @@ -245,7 +245,12 @@ struct pci_dev *pci_get_subsys(unsigned int vendor, unsigned int device,
>                                unsigned int ss_vendor, unsigned int ss_device,
>                                struct pci_dev *from)
>  {
> -       struct pci_device_id id;
> +       struct pci_device_id id = {
> +               .vendor = vendor,
> +               .device = device,
> +               .subvendor = ss_vendor,
> +               .subdevice = ss_device,
> +       };
>
>         /*
>          * pci_find_subsys() can be called on the ide_setup() path,
> @@ -256,11 +261,6 @@ struct pci_dev *pci_get_subsys(unsigned int vendor, unsigned int device,
>         if (unlikely(no_pci_devices()))
>                 return NULL;
>
> -       id.vendor = vendor;
> -       id.device = device;
> -       id.subvendor = ss_vendor;
> -       id.subdevice = ss_device;
> -

yes, here forget to clear .class and .class_mask

>         return pci_get_dev_by_id(&id, from);
>  }
>
> @@ -300,11 +300,14 @@ pci_get_device(unsigned int vendor, unsigned int device, struct pci_dev *from)
>   */
>  struct pci_dev *pci_get_class(unsigned int class, struct pci_dev *from)
>  {
> -       struct pci_device_id id;
> -
> -       id.vendor = id.device = id.subvendor = id.subdevice = PCI_ANY_ID;
> -       id.class_mask = PCI_ANY_ID;
> -       id.class = class;
> +       struct pci_device_id id = {
> +               .vendor = PCI_ANY_ID,
> +               .device = PCI_ANY_ID,
> +               .subvendor = PCI_ANY_ID,
> +               .subdevice = PCI_ANY_ID,
> +               .class_mask = PCI_ANY_ID,
> +               .class = class,
> +       };
>
>         return pci_get_dev_by_id(&id, from);
>  }

Re: [PATCH 1/2] PCI: Use local parameter pci_device_id for pci_get_subsys/class()

[Feng Tang]

On Sat, 8 Sep 2012 21:42:20 +0800
Fengguang Wu <fengguang.wu@intel.com> wrote:

> On Fri, Sep 07, 2012 at 06:32:48PM -0700, Yinghai Lu wrote:
> 
> > > with this one in pci/next pci config in /sys are not created.
> > >
> > > 10:~ # lspci -tv
> > > pcilib: Cannot open /sys/bus/pci/devices/0000:00:03.0/config
> > > lspci: Unable to read the standard configuration space header of
> > > device 0000:00:03.0
> > > pcilib: Cannot open /sys/bus/pci/devices/0000:00:02.0/config
> > > lspci: Unable to read the standard configuration space header of
> > > device 0000:00:02.0
> > > pcilib: Cannot open /sys/bus/pci/devices/0000:00:01.3/config
> > > lspci: Unable to read the standard configuration space header of
> > > device 0000:00:01.3
> > > pcilib: Cannot open /sys/bus/pci/devices/0000:00:01.1/config
> > > lspci: Unable to read the standard configuration space header of
> > > device 0000:00:01.1
> > > pcilib: Cannot open /sys/bus/pci/devices/0000:00:01.0/config
> > > lspci: Unable to read the standard configuration space header of
> > > device 0000:00:01.0
> > > pcilib: Cannot open /sys/bus/pci/devices/0000:00:00.0/config
> > > lspci: Unable to read the standard configuration space header of
> > > device 0000:00:00.0
> > > -[0000:00]-
> > >
> > > bisected to this commit
> > >
> > > ccee7d23102f5e5765ec24779c5b77472af8f79e is the first bad commit
> > > commit ccee7d23102f5e5765ec24779c5b77472af8f79e
> > > Author: Feng Tang <feng.tang@intel.com>
> > > Date:   Thu Aug 23 15:45:03 2012 +0800
> > >
> > >     PCI: Use pci_device_id on stack for pci_get_subsys/class() to avoid kmalloc
> > >
> > >     This fixes a kernel warning https://lkml.org/lkml/2012/7/31/682
> > >
> > >     pci_get_subsys() may get called in late system reboot stage, using
> > >     a sleepable kmalloc() sounds fragile and will cause a kernel warning
> > >     with my recent commmit 55c844a "x86/reboot: Fix a warning message
> > >     triggered by stop_other_cpus()" which disable local interrupt in
> > >     late system shutdown/reboot phase. Using a local parameter instead
> > >     will fix it and make it eligible for calling forom atomic context.
> > >
> > >     Do the same change for the pci_get_class() as suggested by Bjorn Helgaas
> > >
> > >     [bhelgaas: changelog]
> > >     Bisected-by: Fengguang Wu <fengguang.wu@intel.com>
> > >     Signed-off-by: Feng Tang <feng.tang@intel.com>
> > >     Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
> > >     Reviewed-by: Fengguang Wu <fengguang.wu@intel.com>
> > >
> > > :040000 040000 dee62a035816b73abc68e40de8f21c7349efc4cb
> > > 70b2a6258bffa1ab963bd650d8f5d02da774fbce M      drivers
> > >
> > > so the stack get overrun ?
> > >
> > > Bjorn, I think it is this one that cause lspci broken that I mentioned
> > > during meeting at San Diego.
> 
> This makes lspci work again on my side. The caveat is, kzalloc() will
> zero out all data while the new local variable leaves some data
> uninitialized.

Yes, thanks for the quick root cause and fix to the bug in my code.

- Feng

Re: [PATCH 1/2] PCI: Use local parameter pci_device_id for pci_get_subsys/class()

[Yinghai Lu]

On Sat, Sep 8, 2012 at 8:34 AM, Feng Tang <feng.tang@intel.com> wrote:
>> This makes lspci work again on my side. The caveat is, kzalloc() will
>> zero out all data while the new local variable leaves some data
>> uninitialized.
>
> Yes, thanks for the quick root cause and fix to the bug in my code.

Can you resubmit your patch with two extra "memset" line?

Thanks

Yinghai

Re: [PATCH 1/2] PCI: Use local parameter pci_device_id for pci_get_subsys/class()

[Bjorn Helgaas]

On Sat, Sep 08, 2012 at 11:40:52AM -0700, Yinghai Lu wrote:
> On Sat, Sep 8, 2012 at 8:34 AM, Feng Tang <feng.tang@intel.com> wrote:
> >> This makes lspci work again on my side. The caveat is, kzalloc() will
> >> zero out all data while the new local variable leaves some data
> >> uninitialized.
> >
> > Yes, thanks for the quick root cause and fix to the bug in my code.
> 
> Can you resubmit your patch with two extra "memset" line?

I updated the patch as follows and rebased my "next" branch to include it:

commit e664f5bd55247bba3a6ebd61f83d6c9cd87ce0de
Author: Feng Tang <feng.tang@intel.com>
Date:   Thu Aug 23 15:45:03 2012 +0800

    PCI: Use pci_device_id on stack for pci_get_subsys/class() to avoid kmalloc
    
    This fixes a kernel warning https://lkml.org/lkml/2012/7/31/682
    
    pci_get_subsys() may get called in late system reboot stage, using
    a sleepable kmalloc() sounds fragile and will cause a kernel warning
    with my recent commmit 55c844a "x86/reboot: Fix a warning message
    triggered by stop_other_cpus()" which disable local interrupt in
    late system shutdown/reboot phase. Using a local parameter instead
    will fix it and make it eligible for calling forom atomic context.
    
    Do the same change for the pci_get_class() as suggested by Bjorn Helgaas
    
    [bhelgaas: changelog, clear pci_device_id on stack with memset()]
    Bisected-by: Fengguang Wu <fengguang.wu@intel.com>
    Signed-off-by: Feng Tang <feng.tang@intel.com>
    Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
    Reviewed-by: Fengguang Wu <fengguang.wu@intel.com>

diff --git a/drivers/pci/search.c b/drivers/pci/search.c
index 993d4a0..e0a0310 100644
--- a/drivers/pci/search.c
+++ b/drivers/pci/search.c
@@ -245,8 +245,7 @@ struct pci_dev *pci_get_subsys(unsigned int vendor, unsigned int device,
 			       unsigned int ss_vendor, unsigned int ss_device,
 			       struct pci_dev *from)
 {
-	struct pci_dev *pdev;
-	struct pci_device_id *id;
+	struct pci_device_id id;
 
 	/*
 	 * pci_find_subsys() can be called on the ide_setup() path,
@@ -257,18 +256,13 @@ struct pci_dev *pci_get_subsys(unsigned int vendor, unsigned int device,
 	if (unlikely(no_pci_devices()))
 		return NULL;
 
-	id = kzalloc(sizeof(*id), GFP_KERNEL);
-	if (!id)
-		return NULL;
-	id->vendor = vendor;
-	id->device = device;
-	id->subvendor = ss_vendor;
-	id->subdevice = ss_device;
-
-	pdev = pci_get_dev_by_id(id, from);
-	kfree(id);
+	memset(&id, 0, sizeof(id));
+	id.vendor = vendor;
+	id.device = device;
+	id.subvendor = ss_vendor;
+	id.subdevice = ss_device;
 
-	return pdev;
+	return pci_get_dev_by_id(&id, from);
 }
 
 /**
@@ -307,19 +301,14 @@ pci_get_device(unsigned int vendor, unsigned int device, struct pci_dev *from)
  */
 struct pci_dev *pci_get_class(unsigned int class, struct pci_dev *from)
 {
-	struct pci_dev *dev;
-	struct pci_device_id *id;
+	struct pci_device_id id;
 
-	id = kzalloc(sizeof(*id), GFP_KERNEL);
-	if (!id)
-		return NULL;
-	id->vendor = id->device = id->subvendor = id->subdevice = PCI_ANY_ID;
-	id->class_mask = PCI_ANY_ID;
-	id->class = class;
+	memset(&id, 0, sizeof(id));
+	id.vendor = id.device = id.subvendor = id.subdevice = PCI_ANY_ID;
+	id.class_mask = PCI_ANY_ID;
+	id.class = class;
 
-	dev = pci_get_dev_by_id(id, from);
-	kfree(id);
-	return dev;
+	return pci_get_dev_by_id(&id, from);
 }
 
 /**

[PATCH 2/2] PCI: Remove the obsolete no_pci_devices() check

[Feng Tang]

>From 9f2f3bbdf65f669e091c72b9648a4a0394ce28f5 Mon Sep 17 00:00:00 2001
From: Feng Tang <feng.tang@intel.com>
Date: Thu, 23 Aug 2012 14:55:48 +0800
Subject: [PATCH 2/2] PCI: Remove the obsolete no_pci_devices() check

In function pci_get_subsys() there is a check:

	/*
	 * pci_find_subsys() can be called on the ide_setup() path,
	 * super-early in boot.  But the down_read() will enable local
	 * interrupts, which can cause some machines to crash.  So here we
	 * detect and flag that situation and bail out early.
	 */
	if (unlikely(no_pci_devices()))
		return NULL;

But there is no ide_setup() now, and no down_read() either, which
makes the check absolete. So remove it.

Signed-off-by: Feng Tang <feng.tang@intel.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
---
 drivers/pci/search.c |    9 ---------
 1 files changed, 0 insertions(+), 9 deletions(-)

diff --git a/drivers/pci/search.c b/drivers/pci/search.c
index 78a08b1..e6e604f 100644
--- a/drivers/pci/search.c
+++ b/drivers/pci/search.c
@@ -247,15 +247,6 @@ struct pci_dev *pci_get_subsys(unsigned int vendor, unsigned int device,
 {
 	struct pci_device_id id;
 
-	/*
-	 * pci_find_subsys() can be called on the ide_setup() path,
-	 * super-early in boot.  But the down_read() will enable local
-	 * interrupts, which can cause some machines to crash.  So here we
-	 * detect and flag that situation and bail out early.
-	 */
-	if (unlikely(no_pci_devices()))
-		return NULL;
-
 	id.vendor = vendor;
 	id.device = device;
 	id.subvendor = ss_vendor;
-- 
1.7.1