[PATCH] cpufreq, store_scaling_governor requires policy->rwsem to be held for duration of changing governors [v2]

[PATCH] cpufreq, store_scaling_governor requires policy->rwsem to be held for duration of changing governors [v2]

[Prarit Bhargava]

[v2]: Fixed wording of commit, and added additional testing information.

A while ago we added a test to mimic some of our users' userspace governor
programs which monitor system behaviour and will switch governors on the
fly.  The decision process for this includes looking at time of day,
expected system load, etc.  For some time now we have had reports of
system panics in the cpufreq code when using the userspace governor
utility.

The userspace utility writes
/sys/devices/system/cpu/cpuX/cpufreq/scaling_governor and sets the
governor.  In some cases this can happen rapidly, and under heavy load
there are occasions where the changes are delayed.  This can mean that
several governor changes may occur within a short period of time.

This has exposed a bug in the store_scaling_governor path.  When the sysfs
file is written to,

store()
	->down_write(&policy->rwsem);
	->store_scaling_governor()
		-> cpufreq_set_policy()
			up_write(&policy->rwsem);
			__cpufreq_governor(policy, CPUFREQ_GOV_POLICY_EXIT);
			down_write(&policy->rwsem);

The release of the policy->rwsem results in a situation where another
write to the scaling_governor file can now start and overwrite pointers
and cause corruption.

ex)

 Oops: 0000 [#1] SMP
 Modules linked in: sg e1000e igb iTCO_wdt iTCO_vendor_support coretemp kvm_intel ptp kvm pps_core i7300_edac edac_core ses lpc_ich enclosure ioatdma mfd_core shpchp dca i2c_i801 serio_raw acpi_cpufreq pcspkr mperf nfsd auth_rpcgss nfs_acl lockd sunrpc xfs libcrc32c sr_mod cdrom sd_mod ata_generic crc_t10dif crct10dif_common pata_acpi radeon i2c_algo_bit drm_kms_helper ttm drm ata_piix libata i2c_core megaraid_sas dm_mirror dm_region_hash dm_log dm_mod
 CPU: 7 PID: 565 Comm: kworker/7:2 Not tainted 3.10.0-121.el7.x86_64 #1
 Hardware name: Intel MP Server/S7000FC4UR, BIOS SFC4UR.86B.01.00.0031.021820102042 02/18/2010
 Workqueue: events od_dbs_timer
 task: ffff88046544cfa0 ti: ffff880465438000 task.ti: ffff880465438000
 RIP: 0010:[<ffffffff81489bed>]  [<ffffffff81489bed>] od_dbs_timer+0x2d/0x160
 RSP: 0018:ffff880465439de0  EFLAGS: 00010282
 RAX: ffff88007f03d500 RBX: 0000000000010de0 RCX: 0000000000000006
 RDX: 0000000000000006 RSI: 7040000000000000 RDI: ffff88046f2f0e08
 e1000e 0000:06:00.0: irq 79 for MSI/MSI-X
 IPv6: ADDRCONF(NETDEV_UP): enp6s0f0: link is not ready
 RBP: ffff880465439e18 R08: ffff88046f2f0e10 R09: dfedcb48370f0e08
 R10: dfedcb48370f0e08 R11: 0000000000000000 R12: ffff880465a69200
 R13: 0000000000000000 R14: ffff88046f2f0e08 R15: 00000000000001c0
 e1000e 0000:06:00.1: irq 80 for MSI/MSI-X
 FS:  0000000000000000(0000) GS:ffff88046f2e0000(0000) knlGS:0000000000000000
 CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
 CR2: 0000000000000010 CR3: 0000000466be5000 CR4: 00000000000007e0
 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
 DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
 Stack:
  ffff88045ef8d140 ffff880468e1ad80 ffff88046f2f0e08 ffff880465a69200
  ffff88046f2f3e00 ffff88046f2f7f00 00000000000001c0 ffff880465439e60
  ffffffff8107e02b 000000006f2f3e18 0000000000000000 ffff88046f2f3e18
 Call Trace:
  [<ffffffff8107e02b>] process_one_work+0x17b/0x460
  [<ffffffff8107edfb>] worker_thread+0x11b/0x400
  [<ffffffff8107ece0>] ? rescuer_thread+0x400/0x400
  [<ffffffff81085aef>] kthread+0xcf/0xe0
  [<ffffffff81085a20>] ? kthread_create_on_node+0x140/0x140
 e1000e 0000:06:00.1: irq 80 for MSI/MSI-X
 IPv6: ADDRCONF(NETDEV_UP): enp6s0f1: link is not ready
  [<ffffffff815fc76c>] ret_from_fork+0x7c/0xb0
  [<ffffffff81085a20>] ? kthread_create_on_node+0x140/0x140
 Code: 66 66 90 55 48 89 e5 41 57 41 56 49 89 fe 41 55 41 54 53 48 c7 c3 e0 0d 01 00 48 83 ec 10 48 8b 47 f8 8b 50 14 4c 8b 68 40 89 d1 <4d> 8b 7d 10 89 55 cc 48 03 1c cd 80 78 9e 81 48 8d 83 a8 00 00
 RIP  [<ffffffff81489bed>] od_dbs_timer+0x2d/0x160
  RSP <ffff880465439de0>
 CR2: 0000000000000010
 BUG: unable to handle kernel  ---[ end trace c9a1ca82e01a4294 ]---
 Kernel panic - not syncing: Fatal exception
NULL pointer dereference at 0000000000000014
 IP: [<ffffffff81489be4>] od_dbs_timer+0x24/0x160
 PGD 46661d067 PUD 465540067 PMD 0
 Oops: 0000 [#2] SMP
 Modules linked in: sg e1000e igb iTCO_wdt iTCO_vendor_support coretemp kvm_intel ptp kvm pps_core i7300_edac edac_core ses lpc_ich enclosure ioatdma mfd_core shpchp dca i2c_i801 serio_raw acpi_cpufreq pcspkr mperf nfsd auth_rpcgss nfs_acl lockd sunrpc xfs libcrc32c sr_mod cdrom sd_mod ata_generic crc_t10dif crct10dif_common pata_acpi radeon i2c_algo_bit drm_kms_helper ttm drm ata_piix libata i2c_core megaraid_sas dm_mirror dm_region_hash dm_log dm_mod
 CPU: 6 PID: 506 Comm: kworker/6:2 Tainted: G      D     --------------   3.10.0-121.el7.x86_64 #1
 Hardware name: Intel MP Server/S7000FC4UR, BIOS SFC4UR.86B.01.00.0031.021820102042 02/18/2010
 Workqueue: events od_dbs_timer
 task: ffff880465448b60 ti: ffff880463730000 task.ti: ffff880463730000
 RIP: 0010:[<ffffffff81489be4>]  [<ffffffff81489be4>] od_dbs_timer+0x24/0x160
 RSP: 0018:ffff880463731de0  EFLAGS: 00010282
 RAX: 0000000000000000 RBX: 0000000000010de0 RCX: dead000000200200
 RDX: 0000000000000001 RSI: 7040000000000000 RDI: ffff88046f2d0e08
 RBP: ffff880463731e18 R08: ffff88046f2d0e10 R09: dfedcb50370d0e08
 R10: dfedcb50370d0e08 R11: 0000000000000000 R12: ffff880465bd2b80
 R13: ffff88046f2d3e00 R14: ffff88046f2d0e08 R15: 0000000000000180
 FS:  0000000000000000(0000) GS:ffff88046f2c0000(0000) knlGS:0000000000000000
 CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
 CR2: 0000000000000014 CR3: 0000000466be5000 CR4: 00000000000007e0
 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
 DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
 Stack:
  ffff88045ef8d140 ffff880468dcf1c0 ffff88046f2d0e08 ffff880465bd2b80
  ffff88046f2d3e00 ffff88046f2d7f00 0000000000000180 ffff880463731e60
  ffffffff8107e02b 000000006f2d3e18 0000000000000000 ffff88046f2d3e18
 Call Trace:
  [<ffffffff8107e02b>] process_one_work+0x17b/0x460
  [<ffffffff8107edfb>] worker_thread+0x11b/0x400
  [<ffffffff8107ece0>] ? rescuer_thread+0x400/0x400
  [<ffffffff81085aef>] kthread+0xcf/0xe0
  [<ffffffff81085a20>] ? kthread_create_on_node+0x140/0x140
  [<ffffffff815fc76c>] ret_from_fork+0x7c/0xb0
  [<ffffffff81085a20>] ? kthread_create_on_node+0x140/0x140
 Code: 1f 84 00 00 00 00 00 66 66 66 66 90 55 48 89 e5 41 57 41 56 49 89 fe 41 55 41 54 53 48 c7 c3 e0 0d 01 00 48 83 ec 10 48 8b 47 f8 <8b> 50 14 4c 8b 68 40 89 d1 4d 8b 7d 10 89 55 cc 48 03 1c cd 80
 RIP  [<ffffffff81489be4>] od_dbs_timer+0x24/0x160
  RSP <ffff880463731de0>
 CR2: 0000000000000014
 drm_kms_helper: panic occurred, switching back to text console

There are other panics associated with this, including list corruption and
various flavors of panics in the od_dbs_timer & work code.

As a test I introduced a separate mutex around the cpufreq_set_policy() call.
After doing so system behaves as expected, and the panics no longer happen.
I narrowed the mutex down to the "governor switch" code in
cpufreq_set_policy() and saw that everything stabilized.

I then reverted commit 955ef4833574636819cd269cfbae12f79cbde63a, cpufreq: Drop
rwsem lock around CPUFREQ_GOV_POLICY_EXIT, which added the dropping of the
policy->rwsem and again, the panics no longer occurred.

The closer I looked at commit 955ef483, the more questions I have.  The first
thing is that it appears that the stacktrace includes function calls that
are not, and never have been, part of the linux.git tree, ie) the call trace
shows

            [<c0055a79>] lock_acquire+0x61/0xbc
            [<c00fabf1>] sysfs_addrm_finish+0xc1/0x128
            [<c00f9819>] sysfs_hash_and_remove+0x35/0x64
            [<c00fbe6f>] remove_files.isra.0+0x1b/0x24
            [<c00fbea5>] sysfs_remove_group+0x2d/0xa8
            [<c02f9a0b>] cpufreq_governor_interactive+0x13b/0x35c
            [<c02f61df>] __cpufreq_governor+0x2b/0x8c
            [<c02f6579>] __cpufreq_set_policy+0xa9/0xf8
            [<c02f6b75>] store_scaling_governor+0x61/0x100
            [<c02f6f4d>] store+0x39/0x60
            [<c00f9b81>] sysfs_write_file+0xed/0x114
            [<c00b3fd1>] vfs_write+0x65/0xd8
            [<c00b424b>] sys_write+0x2f/0x50
            [<c000cdc1>] ret_fast_syscall+0x1/0x52

and the top part of the stack from cpufreq_governor_interactive() appear to
be for a driver that is not in the linux.git tree.  Google search does show
that it exists in various android trees, however, my concern is that the "core"
scaling governors are now broken because of an out tree driver.

It seems like the patch fixed this problem incorrectly by breaking the existing
locking.  The whole point of the policy->rwsem is to single thread updates
to the policy struct; this is no longer the case after that commit as we
are breaking the locking right in the middle of a policy->governor update.
The fix should have been to the cpufreq_governor_interactive out-of-tree
driver and fix the locking there in its release code.  In any case, the current
linux.git code no longer can reproduce the original failure; the locking in the
sysfs release code has changed.

In case anyone is interested in reproducing, the reproducer I used was

i=0
while [ True ]; do
        i=$((i+1))
        echo "ondemand" > /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor &
        echo "performance" > /sys/devices/system/cpu/cpu1/cpufreq/scaling_governor &
        echo "ondemand" > /sys/devices/system/cpu/cpu1/cpufreq/scaling_governor  &
        echo "performance" > /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor &
        if [ $((i % 100)) = 0 ]; then
                num_ps=$(ps aux | grep doit | wc -l)
                echo "$i running, $num_ps outstanding"
        fi
done

and the system usually panics within 500 iterations.

I have tested this extensively across 15 systems of various configurations.
In the unpatched kernel I see oopses usually within 500 iterations.  In the
patched kernel which fixes the broken locking, I can go several hours
without any issues.

This patch effectively reverts commit 955ef483.

Cc: "Rafael J. Wysocki" <rjw@rjwysocki.net>
Cc: Viresh Kumar <viresh.kumar@linaro.org>
Cc: Lenny Szubowicz <lszubowi@redhat.com>
Cc: linux-pm@vger.kernel.org
Signed-off-by: Prarit Bhargava <prarit@redhat.com>
---
 drivers/cpufreq/cpufreq.c |    8 +-------
 include/linux/cpufreq.h   |    4 ----
 2 files changed, 1 insertion(+), 11 deletions(-)

diff --git a/drivers/cpufreq/cpufreq.c b/drivers/cpufreq/cpufreq.c
index 6f02485..4869f0e 100644
--- a/drivers/cpufreq/cpufreq.c
+++ b/drivers/cpufreq/cpufreq.c
@@ -2198,12 +2198,8 @@ static int cpufreq_set_policy(struct cpufreq_policy *policy,
 	/* save old, working values */
 	old_gov = policy->governor;
 	/* end old governor */
-	if (old_gov) {
-		__cpufreq_governor(policy, CPUFREQ_GOV_STOP);
-		up_write(&policy->rwsem);
+	if (old_gov)
 		__cpufreq_governor(policy, CPUFREQ_GOV_POLICY_EXIT);
-		down_write(&policy->rwsem);
-	}
 
 	/* start new governor */
 	policy->governor = new_policy->governor;
@@ -2211,9 +2207,7 @@ static int cpufreq_set_policy(struct cpufreq_policy *policy,
 		if (!__cpufreq_governor(policy, CPUFREQ_GOV_START))
 			goto out;
 
-		up_write(&policy->rwsem);
 		__cpufreq_governor(policy, CPUFREQ_GOV_POLICY_EXIT);
-		down_write(&policy->rwsem);
 	}
 
 	/* new governor failed, so re-start old one */
diff --git a/include/linux/cpufreq.h b/include/linux/cpufreq.h
index 8f8ae95..b450c43 100644
--- a/include/linux/cpufreq.h
+++ b/include/linux/cpufreq.h
@@ -100,10 +100,6 @@ struct cpufreq_policy {
 	 * - Any routine that will write to the policy structure and/or may take away
 	 *   the policy altogether (eg. CPU hotplug), will hold this lock in write
 	 *   mode before doing so.
-	 *
-	 * Additional rules:
-	 * - Lock should not be held across
-	 *     __cpufreq_governor(data, CPUFREQ_GOV_POLICY_EXIT);
 	 */
 	struct rw_semaphore	rwsem;
 
-- 
1.7.9.3

Re: [PATCH] cpufreq, store_scaling_governor requires policy->rwsem to be held for duration of changing governors [v2]

[Rafael J. Wysocki]

On Tuesday, July 29, 2014 07:46:02 AM Prarit Bhargava wrote:
> [v2]: Fixed wording of commit, and added additional testing information.
> 
> A while ago we added a test to mimic some of our users' userspace governor
> programs which monitor system behaviour and will switch governors on the
> fly.  The decision process for this includes looking at time of day,
> expected system load, etc.  For some time now we have had reports of
> system panics in the cpufreq code when using the userspace governor
> utility.
> 
> The userspace utility writes
> /sys/devices/system/cpu/cpuX/cpufreq/scaling_governor and sets the
> governor.  In some cases this can happen rapidly, and under heavy load
> there are occasions where the changes are delayed.  This can mean that
> several governor changes may occur within a short period of time.
> 
> This has exposed a bug in the store_scaling_governor path.  When the sysfs
> file is written to,
> 
> store()
> 	->down_write(&policy->rwsem);
> 	->store_scaling_governor()
> 		-> cpufreq_set_policy()
> 			up_write(&policy->rwsem);
> 			__cpufreq_governor(policy, CPUFREQ_GOV_POLICY_EXIT);
> 			down_write(&policy->rwsem);
> 
> The release of the policy->rwsem results in a situation where another
> write to the scaling_governor file can now start and overwrite pointers
> and cause corruption.
> 
> ex)
> 
>  Oops: 0000 [#1] SMP
>  Modules linked in: sg e1000e igb iTCO_wdt iTCO_vendor_support coretemp kvm_intel ptp kvm pps_core i7300_edac edac_core ses lpc_ich enclosure ioatdma mfd_core shpchp dca i2c_i801 serio_raw acpi_cpufreq pcspkr mperf nfsd auth_rpcgss nfs_acl lockd sunrpc xfs libcrc32c sr_mod cdrom sd_mod ata_generic crc_t10dif crct10dif_common pata_acpi radeon i2c_algo_bit drm_kms_helper ttm drm ata_piix libata i2c_core megaraid_sas dm_mirror dm_region_hash dm_log dm_mod
>  CPU: 7 PID: 565 Comm: kworker/7:2 Not tainted 3.10.0-121.el7.x86_64 #1
>  Hardware name: Intel MP Server/S7000FC4UR, BIOS SFC4UR.86B.01.00.0031.021820102042 02/18/2010
>  Workqueue: events od_dbs_timer
>  task: ffff88046544cfa0 ti: ffff880465438000 task.ti: ffff880465438000
>  RIP: 0010:[<ffffffff81489bed>]  [<ffffffff81489bed>] od_dbs_timer+0x2d/0x160
>  RSP: 0018:ffff880465439de0  EFLAGS: 00010282
>  RAX: ffff88007f03d500 RBX: 0000000000010de0 RCX: 0000000000000006
>  RDX: 0000000000000006 RSI: 7040000000000000 RDI: ffff88046f2f0e08
>  e1000e 0000:06:00.0: irq 79 for MSI/MSI-X
>  IPv6: ADDRCONF(NETDEV_UP): enp6s0f0: link is not ready
>  RBP: ffff880465439e18 R08: ffff88046f2f0e10 R09: dfedcb48370f0e08
>  R10: dfedcb48370f0e08 R11: 0000000000000000 R12: ffff880465a69200
>  R13: 0000000000000000 R14: ffff88046f2f0e08 R15: 00000000000001c0
>  e1000e 0000:06:00.1: irq 80 for MSI/MSI-X
>  FS:  0000000000000000(0000) GS:ffff88046f2e0000(0000) knlGS:0000000000000000
>  CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
>  CR2: 0000000000000010 CR3: 0000000466be5000 CR4: 00000000000007e0
>  DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
>  DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
>  Stack:
>   ffff88045ef8d140 ffff880468e1ad80 ffff88046f2f0e08 ffff880465a69200
>   ffff88046f2f3e00 ffff88046f2f7f00 00000000000001c0 ffff880465439e60
>   ffffffff8107e02b 000000006f2f3e18 0000000000000000 ffff88046f2f3e18
>  Call Trace:
>   [<ffffffff8107e02b>] process_one_work+0x17b/0x460
>   [<ffffffff8107edfb>] worker_thread+0x11b/0x400
>   [<ffffffff8107ece0>] ? rescuer_thread+0x400/0x400
>   [<ffffffff81085aef>] kthread+0xcf/0xe0
>   [<ffffffff81085a20>] ? kthread_create_on_node+0x140/0x140
>  e1000e 0000:06:00.1: irq 80 for MSI/MSI-X
>  IPv6: ADDRCONF(NETDEV_UP): enp6s0f1: link is not ready
>   [<ffffffff815fc76c>] ret_from_fork+0x7c/0xb0
>   [<ffffffff81085a20>] ? kthread_create_on_node+0x140/0x140
>  Code: 66 66 90 55 48 89 e5 41 57 41 56 49 89 fe 41 55 41 54 53 48 c7 c3 e0 0d 01 00 48 83 ec 10 48 8b 47 f8 8b 50 14 4c 8b 68 40 89 d1 <4d> 8b 7d 10 89 55 cc 48 03 1c cd 80 78 9e 81 48 8d 83 a8 00 00
>  RIP  [<ffffffff81489bed>] od_dbs_timer+0x2d/0x160
>   RSP <ffff880465439de0>
>  CR2: 0000000000000010
>  BUG: unable to handle kernel  ---[ end trace c9a1ca82e01a4294 ]---
>  Kernel panic - not syncing: Fatal exception
> NULL pointer dereference at 0000000000000014
>  IP: [<ffffffff81489be4>] od_dbs_timer+0x24/0x160
>  PGD 46661d067 PUD 465540067 PMD 0
>  Oops: 0000 [#2] SMP
>  Modules linked in: sg e1000e igb iTCO_wdt iTCO_vendor_support coretemp kvm_intel ptp kvm pps_core i7300_edac edac_core ses lpc_ich enclosure ioatdma mfd_core shpchp dca i2c_i801 serio_raw acpi_cpufreq pcspkr mperf nfsd auth_rpcgss nfs_acl lockd sunrpc xfs libcrc32c sr_mod cdrom sd_mod ata_generic crc_t10dif crct10dif_common pata_acpi radeon i2c_algo_bit drm_kms_helper ttm drm ata_piix libata i2c_core megaraid_sas dm_mirror dm_region_hash dm_log dm_mod
>  CPU: 6 PID: 506 Comm: kworker/6:2 Tainted: G      D     --------------   3.10.0-121.el7.x86_64 #1
>  Hardware name: Intel MP Server/S7000FC4UR, BIOS SFC4UR.86B.01.00.0031.021820102042 02/18/2010
>  Workqueue: events od_dbs_timer
>  task: ffff880465448b60 ti: ffff880463730000 task.ti: ffff880463730000
>  RIP: 0010:[<ffffffff81489be4>]  [<ffffffff81489be4>] od_dbs_timer+0x24/0x160
>  RSP: 0018:ffff880463731de0  EFLAGS: 00010282
>  RAX: 0000000000000000 RBX: 0000000000010de0 RCX: dead000000200200
>  RDX: 0000000000000001 RSI: 7040000000000000 RDI: ffff88046f2d0e08
>  RBP: ffff880463731e18 R08: ffff88046f2d0e10 R09: dfedcb50370d0e08
>  R10: dfedcb50370d0e08 R11: 0000000000000000 R12: ffff880465bd2b80
>  R13: ffff88046f2d3e00 R14: ffff88046f2d0e08 R15: 0000000000000180
>  FS:  0000000000000000(0000) GS:ffff88046f2c0000(0000) knlGS:0000000000000000
>  CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
>  CR2: 0000000000000014 CR3: 0000000466be5000 CR4: 00000000000007e0
>  DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
>  DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
>  Stack:
>   ffff88045ef8d140 ffff880468dcf1c0 ffff88046f2d0e08 ffff880465bd2b80
>   ffff88046f2d3e00 ffff88046f2d7f00 0000000000000180 ffff880463731e60
>   ffffffff8107e02b 000000006f2d3e18 0000000000000000 ffff88046f2d3e18
>  Call Trace:
>   [<ffffffff8107e02b>] process_one_work+0x17b/0x460
>   [<ffffffff8107edfb>] worker_thread+0x11b/0x400
>   [<ffffffff8107ece0>] ? rescuer_thread+0x400/0x400
>   [<ffffffff81085aef>] kthread+0xcf/0xe0
>   [<ffffffff81085a20>] ? kthread_create_on_node+0x140/0x140
>   [<ffffffff815fc76c>] ret_from_fork+0x7c/0xb0
>   [<ffffffff81085a20>] ? kthread_create_on_node+0x140/0x140
>  Code: 1f 84 00 00 00 00 00 66 66 66 66 90 55 48 89 e5 41 57 41 56 49 89 fe 41 55 41 54 53 48 c7 c3 e0 0d 01 00 48 83 ec 10 48 8b 47 f8 <8b> 50 14 4c 8b 68 40 89 d1 4d 8b 7d 10 89 55 cc 48 03 1c cd 80
>  RIP  [<ffffffff81489be4>] od_dbs_timer+0x24/0x160
>   RSP <ffff880463731de0>
>  CR2: 0000000000000014
>  drm_kms_helper: panic occurred, switching back to text console
> 
> There are other panics associated with this, including list corruption and
> various flavors of panics in the od_dbs_timer & work code.
> 
> As a test I introduced a separate mutex around the cpufreq_set_policy() call.
> After doing so system behaves as expected, and the panics no longer happen.
> I narrowed the mutex down to the "governor switch" code in
> cpufreq_set_policy() and saw that everything stabilized.
> 
> I then reverted commit 955ef4833574636819cd269cfbae12f79cbde63a, cpufreq: Drop
> rwsem lock around CPUFREQ_GOV_POLICY_EXIT, which added the dropping of the
> policy->rwsem and again, the panics no longer occurred.
> 
> The closer I looked at commit 955ef483, the more questions I have.  The first
> thing is that it appears that the stacktrace includes function calls that
> are not, and never have been, part of the linux.git tree, ie) the call trace
> shows
> 
>             [<c0055a79>] lock_acquire+0x61/0xbc
>             [<c00fabf1>] sysfs_addrm_finish+0xc1/0x128
>             [<c00f9819>] sysfs_hash_and_remove+0x35/0x64
>             [<c00fbe6f>] remove_files.isra.0+0x1b/0x24
>             [<c00fbea5>] sysfs_remove_group+0x2d/0xa8
>             [<c02f9a0b>] cpufreq_governor_interactive+0x13b/0x35c
>             [<c02f61df>] __cpufreq_governor+0x2b/0x8c
>             [<c02f6579>] __cpufreq_set_policy+0xa9/0xf8
>             [<c02f6b75>] store_scaling_governor+0x61/0x100
>             [<c02f6f4d>] store+0x39/0x60
>             [<c00f9b81>] sysfs_write_file+0xed/0x114
>             [<c00b3fd1>] vfs_write+0x65/0xd8
>             [<c00b424b>] sys_write+0x2f/0x50
>             [<c000cdc1>] ret_fast_syscall+0x1/0x52
> 
> and the top part of the stack from cpufreq_governor_interactive() appear to
> be for a driver that is not in the linux.git tree.  Google search does show
> that it exists in various android trees, however, my concern is that the "core"
> scaling governors are now broken because of an out tree driver.
> 
> It seems like the patch fixed this problem incorrectly by breaking the existing
> locking.  The whole point of the policy->rwsem is to single thread updates
> to the policy struct; this is no longer the case after that commit as we
> are breaking the locking right in the middle of a policy->governor update.
> The fix should have been to the cpufreq_governor_interactive out-of-tree
> driver and fix the locking there in its release code.  In any case, the current
> linux.git code no longer can reproduce the original failure; the locking in the
> sysfs release code has changed.
> 
> In case anyone is interested in reproducing, the reproducer I used was
> 
> i=0
> while [ True ]; do
>         i=$((i+1))
>         echo "ondemand" > /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor &
>         echo "performance" > /sys/devices/system/cpu/cpu1/cpufreq/scaling_governor &
>         echo "ondemand" > /sys/devices/system/cpu/cpu1/cpufreq/scaling_governor  &
>         echo "performance" > /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor &
>         if [ $((i % 100)) = 0 ]; then
>                 num_ps=$(ps aux | grep doit | wc -l)
>                 echo "$i running, $num_ps outstanding"
>         fi
> done
> 
> and the system usually panics within 500 iterations.
> 
> I have tested this extensively across 15 systems of various configurations.
> In the unpatched kernel I see oopses usually within 500 iterations.  In the
> patched kernel which fixes the broken locking, I can go several hours
> without any issues.
> 
> This patch effectively reverts commit 955ef483.

OK, I'm convinced by this.

I suppose we should push it for -stable from 3.10 through 3.15.x, right?

> Cc: "Rafael J. Wysocki" <rjw@rjwysocki.net>
> Cc: Viresh Kumar <viresh.kumar@linaro.org>
> Cc: Lenny Szubowicz <lszubowi@redhat.com>
> Cc: linux-pm@vger.kernel.org
> Signed-off-by: Prarit Bhargava <prarit@redhat.com>
> ---
>  drivers/cpufreq/cpufreq.c |    8 +-------
>  include/linux/cpufreq.h   |    4 ----
>  2 files changed, 1 insertion(+), 11 deletions(-)
> 
> diff --git a/drivers/cpufreq/cpufreq.c b/drivers/cpufreq/cpufreq.c
> index 6f02485..4869f0e 100644
> --- a/drivers/cpufreq/cpufreq.c
> +++ b/drivers/cpufreq/cpufreq.c
> @@ -2198,12 +2198,8 @@ static int cpufreq_set_policy(struct cpufreq_policy *policy,
>  	/* save old, working values */
>  	old_gov = policy->governor;
>  	/* end old governor */
> -	if (old_gov) {
> -		__cpufreq_governor(policy, CPUFREQ_GOV_STOP);
> -		up_write(&policy->rwsem);
> +	if (old_gov)
>  		__cpufreq_governor(policy, CPUFREQ_GOV_POLICY_EXIT);
> -		down_write(&policy->rwsem);
> -	}
>  
>  	/* start new governor */
>  	policy->governor = new_policy->governor;
> @@ -2211,9 +2207,7 @@ static int cpufreq_set_policy(struct cpufreq_policy *policy,
>  		if (!__cpufreq_governor(policy, CPUFREQ_GOV_START))
>  			goto out;
>  
> -		up_write(&policy->rwsem);
>  		__cpufreq_governor(policy, CPUFREQ_GOV_POLICY_EXIT);
> -		down_write(&policy->rwsem);
>  	}
>  
>  	/* new governor failed, so re-start old one */
> diff --git a/include/linux/cpufreq.h b/include/linux/cpufreq.h
> index 8f8ae95..b450c43 100644
> --- a/include/linux/cpufreq.h
> +++ b/include/linux/cpufreq.h
> @@ -100,10 +100,6 @@ struct cpufreq_policy {
>  	 * - Any routine that will write to the policy structure and/or may take away
>  	 *   the policy altogether (eg. CPU hotplug), will hold this lock in write
>  	 *   mode before doing so.
> -	 *
> -	 * Additional rules:
> -	 * - Lock should not be held across
> -	 *     __cpufreq_governor(data, CPUFREQ_GOV_POLICY_EXIT);
>  	 */
>  	struct rw_semaphore	rwsem;
>  
> 

-- 
I speak only for myself.
Rafael J. Wysocki, Intel Open Source Technology Center.

Re: [PATCH] cpufreq, store_scaling_governor requires policy->rwsem to be held for duration of changing governors [v2]

[Prarit Bhargava]

On 07/29/2014 08:03 PM, Rafael J. Wysocki wrote:
> On Tuesday, July 29, 2014 07:46:02 AM Prarit Bhargava wrote:
>> [v2]: Fixed wording of commit, and added additional testing information.
>>
>> A while ago we added a test to mimic some of our users' userspace governor
>> programs which monitor system behaviour and will switch governors on the
>> fly.  The decision process for this includes looking at time of day,
>> expected system load, etc.  For some time now we have had reports of
>> system panics in the cpufreq code when using the userspace governor
>> utility.
>>
>> The userspace utility writes
>> /sys/devices/system/cpu/cpuX/cpufreq/scaling_governor and sets the
>> governor.  In some cases this can happen rapidly, and under heavy load
>> there are occasions where the changes are delayed.  This can mean that
>> several governor changes may occur within a short period of time.
>>
>> This has exposed a bug in the store_scaling_governor path.  When the sysfs
>> file is written to,
>>
>> store()
>> 	->down_write(&policy->rwsem);
>> 	->store_scaling_governor()
>> 		-> cpufreq_set_policy()
>> 			up_write(&policy->rwsem);
>> 			__cpufreq_governor(policy, CPUFREQ_GOV_POLICY_EXIT);
>> 			down_write(&policy->rwsem);
>>
>> The release of the policy->rwsem results in a situation where another
>> write to the scaling_governor file can now start and overwrite pointers
>> and cause corruption.
>>
>> ex)
>>
>>  Oops: 0000 [#1] SMP
>>  Modules linked in: sg e1000e igb iTCO_wdt iTCO_vendor_support coretemp kvm_intel ptp kvm pps_core i7300_edac edac_core ses lpc_ich enclosure ioatdma mfd_core shpchp dca i2c_i801 serio_raw acpi_cpufreq pcspkr mperf nfsd auth_rpcgss nfs_acl lockd sunrpc xfs libcrc32c sr_mod cdrom sd_mod ata_generic crc_t10dif crct10dif_common pata_acpi radeon i2c_algo_bit drm_kms_helper ttm drm ata_piix libata i2c_core megaraid_sas dm_mirror dm_region_hash dm_log dm_mod
>>  CPU: 7 PID: 565 Comm: kworker/7:2 Not tainted 3.10.0-121.el7.x86_64 #1
>>  Hardware name: Intel MP Server/S7000FC4UR, BIOS SFC4UR.86B.01.00.0031.021820102042 02/18/2010
>>  Workqueue: events od_dbs_timer
>>  task: ffff88046544cfa0 ti: ffff880465438000 task.ti: ffff880465438000
>>  RIP: 0010:[<ffffffff81489bed>]  [<ffffffff81489bed>] od_dbs_timer+0x2d/0x160
>>  RSP: 0018:ffff880465439de0  EFLAGS: 00010282
>>  RAX: ffff88007f03d500 RBX: 0000000000010de0 RCX: 0000000000000006
>>  RDX: 0000000000000006 RSI: 7040000000000000 RDI: ffff88046f2f0e08
>>  e1000e 0000:06:00.0: irq 79 for MSI/MSI-X
>>  IPv6: ADDRCONF(NETDEV_UP): enp6s0f0: link is not ready
>>  RBP: ffff880465439e18 R08: ffff88046f2f0e10 R09: dfedcb48370f0e08
>>  R10: dfedcb48370f0e08 R11: 0000000000000000 R12: ffff880465a69200
>>  R13: 0000000000000000 R14: ffff88046f2f0e08 R15: 00000000000001c0
>>  e1000e 0000:06:00.1: irq 80 for MSI/MSI-X
>>  FS:  0000000000000000(0000) GS:ffff88046f2e0000(0000) knlGS:0000000000000000
>>  CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
>>  CR2: 0000000000000010 CR3: 0000000466be5000 CR4: 00000000000007e0
>>  DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
>>  DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
>>  Stack:
>>   ffff88045ef8d140 ffff880468e1ad80 ffff88046f2f0e08 ffff880465a69200
>>   ffff88046f2f3e00 ffff88046f2f7f00 00000000000001c0 ffff880465439e60
>>   ffffffff8107e02b 000000006f2f3e18 0000000000000000 ffff88046f2f3e18
>>  Call Trace:
>>   [<ffffffff8107e02b>] process_one_work+0x17b/0x460
>>   [<ffffffff8107edfb>] worker_thread+0x11b/0x400
>>   [<ffffffff8107ece0>] ? rescuer_thread+0x400/0x400
>>   [<ffffffff81085aef>] kthread+0xcf/0xe0
>>   [<ffffffff81085a20>] ? kthread_create_on_node+0x140/0x140
>>  e1000e 0000:06:00.1: irq 80 for MSI/MSI-X
>>  IPv6: ADDRCONF(NETDEV_UP): enp6s0f1: link is not ready
>>   [<ffffffff815fc76c>] ret_from_fork+0x7c/0xb0
>>   [<ffffffff81085a20>] ? kthread_create_on_node+0x140/0x140
>>  Code: 66 66 90 55 48 89 e5 41 57 41 56 49 89 fe 41 55 41 54 53 48 c7 c3 e0 0d 01 00 48 83 ec 10 48 8b 47 f8 8b 50 14 4c 8b 68 40 89 d1 <4d> 8b 7d 10 89 55 cc 48 03 1c cd 80 78 9e 81 48 8d 83 a8 00 00
>>  RIP  [<ffffffff81489bed>] od_dbs_timer+0x2d/0x160
>>   RSP <ffff880465439de0>
>>  CR2: 0000000000000010
>>  BUG: unable to handle kernel  ---[ end trace c9a1ca82e01a4294 ]---
>>  Kernel panic - not syncing: Fatal exception
>> NULL pointer dereference at 0000000000000014
>>  IP: [<ffffffff81489be4>] od_dbs_timer+0x24/0x160
>>  PGD 46661d067 PUD 465540067 PMD 0
>>  Oops: 0000 [#2] SMP
>>  Modules linked in: sg e1000e igb iTCO_wdt iTCO_vendor_support coretemp kvm_intel ptp kvm pps_core i7300_edac edac_core ses lpc_ich enclosure ioatdma mfd_core shpchp dca i2c_i801 serio_raw acpi_cpufreq pcspkr mperf nfsd auth_rpcgss nfs_acl lockd sunrpc xfs libcrc32c sr_mod cdrom sd_mod ata_generic crc_t10dif crct10dif_common pata_acpi radeon i2c_algo_bit drm_kms_helper ttm drm ata_piix libata i2c_core megaraid_sas dm_mirror dm_region_hash dm_log dm_mod
>>  CPU: 6 PID: 506 Comm: kworker/6:2 Tainted: G      D     --------------   3.10.0-121.el7.x86_64 #1
>>  Hardware name: Intel MP Server/S7000FC4UR, BIOS SFC4UR.86B.01.00.0031.021820102042 02/18/2010
>>  Workqueue: events od_dbs_timer
>>  task: ffff880465448b60 ti: ffff880463730000 task.ti: ffff880463730000
>>  RIP: 0010:[<ffffffff81489be4>]  [<ffffffff81489be4>] od_dbs_timer+0x24/0x160
>>  RSP: 0018:ffff880463731de0  EFLAGS: 00010282
>>  RAX: 0000000000000000 RBX: 0000000000010de0 RCX: dead000000200200
>>  RDX: 0000000000000001 RSI: 7040000000000000 RDI: ffff88046f2d0e08
>>  RBP: ffff880463731e18 R08: ffff88046f2d0e10 R09: dfedcb50370d0e08
>>  R10: dfedcb50370d0e08 R11: 0000000000000000 R12: ffff880465bd2b80
>>  R13: ffff88046f2d3e00 R14: ffff88046f2d0e08 R15: 0000000000000180
>>  FS:  0000000000000000(0000) GS:ffff88046f2c0000(0000) knlGS:0000000000000000
>>  CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
>>  CR2: 0000000000000014 CR3: 0000000466be5000 CR4: 00000000000007e0
>>  DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
>>  DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
>>  Stack:
>>   ffff88045ef8d140 ffff880468dcf1c0 ffff88046f2d0e08 ffff880465bd2b80
>>   ffff88046f2d3e00 ffff88046f2d7f00 0000000000000180 ffff880463731e60
>>   ffffffff8107e02b 000000006f2d3e18 0000000000000000 ffff88046f2d3e18
>>  Call Trace:
>>   [<ffffffff8107e02b>] process_one_work+0x17b/0x460
>>   [<ffffffff8107edfb>] worker_thread+0x11b/0x400
>>   [<ffffffff8107ece0>] ? rescuer_thread+0x400/0x400
>>   [<ffffffff81085aef>] kthread+0xcf/0xe0
>>   [<ffffffff81085a20>] ? kthread_create_on_node+0x140/0x140
>>   [<ffffffff815fc76c>] ret_from_fork+0x7c/0xb0
>>   [<ffffffff81085a20>] ? kthread_create_on_node+0x140/0x140
>>  Code: 1f 84 00 00 00 00 00 66 66 66 66 90 55 48 89 e5 41 57 41 56 49 89 fe 41 55 41 54 53 48 c7 c3 e0 0d 01 00 48 83 ec 10 48 8b 47 f8 <8b> 50 14 4c 8b 68 40 89 d1 4d 8b 7d 10 89 55 cc 48 03 1c cd 80
>>  RIP  [<ffffffff81489be4>] od_dbs_timer+0x24/0x160
>>   RSP <ffff880463731de0>
>>  CR2: 0000000000000014
>>  drm_kms_helper: panic occurred, switching back to text console
>>
>> There are other panics associated with this, including list corruption and
>> various flavors of panics in the od_dbs_timer & work code.
>>
>> As a test I introduced a separate mutex around the cpufreq_set_policy() call.
>> After doing so system behaves as expected, and the panics no longer happen.
>> I narrowed the mutex down to the "governor switch" code in
>> cpufreq_set_policy() and saw that everything stabilized.
>>
>> I then reverted commit 955ef4833574636819cd269cfbae12f79cbde63a, cpufreq: Drop
>> rwsem lock around CPUFREQ_GOV_POLICY_EXIT, which added the dropping of the
>> policy->rwsem and again, the panics no longer occurred.
>>
>> The closer I looked at commit 955ef483, the more questions I have.  The first
>> thing is that it appears that the stacktrace includes function calls that
>> are not, and never have been, part of the linux.git tree, ie) the call trace
>> shows
>>
>>             [<c0055a79>] lock_acquire+0x61/0xbc
>>             [<c00fabf1>] sysfs_addrm_finish+0xc1/0x128
>>             [<c00f9819>] sysfs_hash_and_remove+0x35/0x64
>>             [<c00fbe6f>] remove_files.isra.0+0x1b/0x24
>>             [<c00fbea5>] sysfs_remove_group+0x2d/0xa8
>>             [<c02f9a0b>] cpufreq_governor_interactive+0x13b/0x35c
>>             [<c02f61df>] __cpufreq_governor+0x2b/0x8c
>>             [<c02f6579>] __cpufreq_set_policy+0xa9/0xf8
>>             [<c02f6b75>] store_scaling_governor+0x61/0x100
>>             [<c02f6f4d>] store+0x39/0x60
>>             [<c00f9b81>] sysfs_write_file+0xed/0x114
>>             [<c00b3fd1>] vfs_write+0x65/0xd8
>>             [<c00b424b>] sys_write+0x2f/0x50
>>             [<c000cdc1>] ret_fast_syscall+0x1/0x52
>>
>> and the top part of the stack from cpufreq_governor_interactive() appear to
>> be for a driver that is not in the linux.git tree.  Google search does show
>> that it exists in various android trees, however, my concern is that the "core"
>> scaling governors are now broken because of an out tree driver.
>>
>> It seems like the patch fixed this problem incorrectly by breaking the existing
>> locking.  The whole point of the policy->rwsem is to single thread updates
>> to the policy struct; this is no longer the case after that commit as we
>> are breaking the locking right in the middle of a policy->governor update.
>> The fix should have been to the cpufreq_governor_interactive out-of-tree
>> driver and fix the locking there in its release code.  In any case, the current
>> linux.git code no longer can reproduce the original failure; the locking in the
>> sysfs release code has changed.
>>
>> In case anyone is interested in reproducing, the reproducer I used was
>>
>> i=0
>> while [ True ]; do
>>         i=$((i+1))
>>         echo "ondemand" > /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor &
>>         echo "performance" > /sys/devices/system/cpu/cpu1/cpufreq/scaling_governor &
>>         echo "ondemand" > /sys/devices/system/cpu/cpu1/cpufreq/scaling_governor  &
>>         echo "performance" > /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor &
>>         if [ $((i % 100)) = 0 ]; then
>>                 num_ps=$(ps aux | grep doit | wc -l)
>>                 echo "$i running, $num_ps outstanding"
>>         fi
>> done
>>
>> and the system usually panics within 500 iterations.
>>
>> I have tested this extensively across 15 systems of various configurations.
>> In the unpatched kernel I see oopses usually within 500 iterations.  In the
>> patched kernel which fixes the broken locking, I can go several hours
>> without any issues.
>>
>> This patch effectively reverts commit 955ef483.
> 
> OK, I'm convinced by this.
> 
> I suppose we should push it for -stable from 3.10 through 3.15.x, right?

Rafael, I think that is a good idea.  I'm not sure what the protocol is for
adding stable@kernel.org though ...

P.

Re: [PATCH] cpufreq, store_scaling_governor requires policy->rwsem to be held for duration of changing governors [v2]

[Rafael J. Wysocki]

On Wednesday, July 30, 2014 10:18:25 AM Prarit Bhargava wrote:
> 
> On 07/29/2014 08:03 PM, Rafael J. Wysocki wrote:
> > On Tuesday, July 29, 2014 07:46:02 AM Prarit Bhargava wrote:

[cut]

> >> This patch effectively reverts commit 955ef483.
> > 
> > OK, I'm convinced by this.
> > 
> > I suppose we should push it for -stable from 3.10 through 3.15.x, right?
> 
> Rafael, I think that is a good idea.  I'm not sure what the protocol is for
> adding stable@kernel.org though ...

I'll take care of this, thanks!

-- 
I speak only for myself.
Rafael J. Wysocki, Intel Open Source Technology Center.

Re: [PATCH] cpufreq, store_scaling_governor requires policy->rwsem to be held for duration of changing governors [v2]

[Saravana Kannan]

On 07/30/2014 02:40 PM, Rafael J. Wysocki wrote:
> On Wednesday, July 30, 2014 10:18:25 AM Prarit Bhargava wrote:
>>
>> On 07/29/2014 08:03 PM, Rafael J. Wysocki wrote:
>>> On Tuesday, July 29, 2014 07:46:02 AM Prarit Bhargava wrote:
>
> [cut]
>
>>>> This patch effectively reverts commit 955ef483.

The issue reported in this patch is valid. We are seeing that internally 
too. I believe I reported it in another thread (within the past month).

However, the original patch fixes a real deadlock issue (I'm too tired 
to look it up now). We can revet the original, but it's going to bring 
back the original issue. I just want to make sure Prarit and Raphael 
realize this before proceeding.

I do have plans for a proper fix for the mainline (not stable branches), 
but plan to do that after the current set of suspend/hotplug patches go 
through. The fix would be easier to make after that.

>>>
>>> OK, I'm convinced by this.
>>>
>>> I suppose we should push it for -stable from 3.10 through 3.15.x, right?
>>
>> Rafael, I think that is a good idea.  I'm not sure what the protocol is for
>> adding stable@kernel.org though ...
>
> I'll take care of this, thanks!
>

But you aren't going to pull the in for the next release, right?

-Saravana

-- 
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
hosted by The Linux Foundation

Re: [PATCH] cpufreq, store_scaling_governor requires policy->rwsem to be held for duration of changing governors [v2]

[Saravana Kannan]

On 07/30/2014 07:16 PM, Rafael J. Wysocki wrote:
> On Wednesday, July 30, 2014 06:36:00 PM Saravana Kannan wrote:
>> On 07/30/2014 02:40 PM, Rafael J. Wysocki wrote:
>>> On Wednesday, July 30, 2014 10:18:25 AM Prarit Bhargava wrote:
>>>>
>>>> On 07/29/2014 08:03 PM, Rafael J. Wysocki wrote:
>>>>> On Tuesday, July 29, 2014 07:46:02 AM Prarit Bhargava wrote:
>>>
>>> [cut]
>>>
>>>>>> This patch effectively reverts commit 955ef483.
>>
>> The issue reported in this patch is valid. We are seeing that internally
>> too. I believe I reported it in another thread (within the past month).
>>
>> However, the original patch fixes a real deadlock issue (I'm too tired
>> to look it up now). We can revet the original, but it's going to bring
>> back the original issue. I just want to make sure Prarit and Raphael
>> realize this before proceeding.
>>
>> I do have plans for a proper fix for the mainline (not stable branches),
>> but plan to do that after the current set of suspend/hotplug patches go
>> through. The fix would be easier to make after that.
>>
>>>>>
>>>>> OK, I'm convinced by this.
>>>>>
>>>>> I suppose we should push it for -stable from 3.10 through 3.15.x, right?
>>>>
>>>> Rafael, I think that is a good idea.  I'm not sure what the protocol is for
>>>> adding stable@kernel.org though ...
>>>
>>> I'll take care of this, thanks!
>>>
>>
>> But you aren't going to pull the in for the next release, right?
>
> What do you mean?
>

Reverting the commit will bring back another dead lock issue. So, you 
don't want to revert it on mainline. Do I still not make sense because 
I'm not using the right terms?

-Saravana

-- 
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
hosted by The Linux Foundation

Re: [PATCH] cpufreq, store_scaling_governor requires policy->rwsem to be held for duration of changing governors [v2]

[Rafael J. Wysocki]

On Wednesday, July 30, 2014 06:36:00 PM Saravana Kannan wrote:
> On 07/30/2014 02:40 PM, Rafael J. Wysocki wrote:
> > On Wednesday, July 30, 2014 10:18:25 AM Prarit Bhargava wrote:
> >>
> >> On 07/29/2014 08:03 PM, Rafael J. Wysocki wrote:
> >>> On Tuesday, July 29, 2014 07:46:02 AM Prarit Bhargava wrote:
> >
> > [cut]
> >
> >>>> This patch effectively reverts commit 955ef483.
> 
> The issue reported in this patch is valid. We are seeing that internally 
> too. I believe I reported it in another thread (within the past month).
> 
> However, the original patch fixes a real deadlock issue (I'm too tired 
> to look it up now). We can revet the original, but it's going to bring 
> back the original issue. I just want to make sure Prarit and Raphael 
> realize this before proceeding.
> 
> I do have plans for a proper fix for the mainline (not stable branches), 
> but plan to do that after the current set of suspend/hotplug patches go 
> through. The fix would be easier to make after that.
> 
> >>>
> >>> OK, I'm convinced by this.
> >>>
> >>> I suppose we should push it for -stable from 3.10 through 3.15.x, right?
> >>
> >> Rafael, I think that is a good idea.  I'm not sure what the protocol is for
> >> adding stable@kernel.org though ...
> >
> > I'll take care of this, thanks!
> >
> 
> But you aren't going to pull the in for the next release, right?

What do you mean?

Rafael

Re: [PATCH] cpufreq, store_scaling_governor requires policy->rwsem to be held for duration of changing governors [v2]

[Prarit Bhargava]

On 07/30/2014 10:16 PM, Rafael J. Wysocki wrote:
> On Wednesday, July 30, 2014 06:36:00 PM Saravana Kannan wrote:
>> On 07/30/2014 02:40 PM, Rafael J. Wysocki wrote:
>>> On Wednesday, July 30, 2014 10:18:25 AM Prarit Bhargava wrote:
>>>>
>>>> On 07/29/2014 08:03 PM, Rafael J. Wysocki wrote:
>>>>> On Tuesday, July 29, 2014 07:46:02 AM Prarit Bhargava wrote:
>>>
>>> [cut]
>>>
>>>>>> This patch effectively reverts commit 955ef483.
>>
>> The issue reported in this patch is valid. We are seeing that internally 
>> too. I believe I reported it in another thread (within the past month).
>>
>> However, the original patch fixes a real deadlock issue (I'm too tired 
>> to look it up now). We can revet the original, but it's going to bring 
>> back the original issue. I just want to make sure Prarit and Raphael 
>> realize this before proceeding.

Hi Saravana,

Thanks for your input.  I went back to the code and confirmed my original
statement about this patch.

Note: in a previous email I erroneously wrote "buffer->mutex" when I should
have identified the lock as sysfs_mutex.  Sorry 'bout that, and apologies
for any confusion that may have caused.

>From my commit message:

"In any case, the current linux.git code no longer can reproduce the original
failure; the locking in the sysfs release code has changed."

The original patch attempted to fix this deadlock:

A cpufreq driver on a file read did:

    -> #0 (&per_cpu(cpu_policy_rwsem, cpu)){+++++.}:
            [<c0055253>] __lock_acquire+0xef3/0x13dc
            [<c0055a79>] lock_acquire+0x61/0xbc
            [<c03ee1f5>] down_read+0x25/0x30
            [<c02f6179>] lock_policy_rwsem_read+0x25/0x34
            [<c02f6edd>] show+0x21/0x58
            [<c00f9c0f>] sysfs_read_file+0x67/0xcc
            [<c00b40a7>] vfs_read+0x63/0xd8
            [<c00b41fb>] sys_read+0x2f/0x50
            [<c000cdc1>] ret_fast_syscall+0x1/0x52

lock(s_active#41) [ which is actually the acquisition of sysfs_mutex ]
lock(&per_cpu(cpu_policy_rwsem, cpu));

and on the governor switch (notably the EXIT of the existing governor), the
opposite occurs

    -> #1 (s_active#41){++++.+}:
            [<c0055a79>] lock_acquire+0x61/0xbc
            [<c00fabf1>] sysfs_addrm_finish+0xc1/0x128
            [<c00f9819>] sysfs_hash_and_remove+0x35/0x64
            [<c00fbe6f>] remove_files.isra.0+0x1b/0x24
            [<c00fbea5>] sysfs_remove_group+0x2d/0xa8
            [<c02f9a0b>] cpufreq_governor_interactive+0x13b/0x35c
            [<c02f61df>] __cpufreq_governor+0x2b/0x8c
            [<c02f6579>] __cpufreq_set_policy+0xa9/0xf8
            [<c02f6b75>] store_scaling_governor+0x61/0x100
            [<c02f6f4d>] store+0x39/0x60
            [<c00f9b81>] sysfs_write_file+0xed/0x114
            [<c00b3fd1>] vfs_write+0x65/0xd8
            [<c00b424b>] sys_write+0x2f/0x50
            [<c000cdc1>] ret_fast_syscall+0x1/0x52


lock(&per_cpu(cpu_policy_rwsem, cpu));
lock(s_active#41) [ which is actually the acquisition of sysfs_mutex ]

The sysfs_mutex no longer blocks in the sysfs path, and I have built with
LOCKDEP on and off to confirm that I do not see any tracebacks or hangs.  I
tested this by doing a few reads of the current governor, and then doing a
governor switch (to at least initiate the LOCKDEP warning).  IIUC the traceback
above that is the way to reproduce this LOCKDEP warning.

IOW ... this deadlock situation no longer exists in the code (at least AFAICT).

Secondly, restoring the locking (and keeping in mind the semantic of the
read-write semaphore, specifically that all reads must be done before the write
lock is acquired by a thread) modifies the code such that the above situation (a
read and a write) can no longer occur in this code.

P.

Re: [PATCH] cpufreq, store_scaling_governor requires policy->rwsem to be held for duration of changing governors [v2]

[Prarit Bhargava]

On 07/31/2014 06:16 AM, Prarit Bhargava wrote:
> 
> 
> On 07/30/2014 10:16 PM, Rafael J. Wysocki wrote:
>> On Wednesday, July 30, 2014 06:36:00 PM Saravana Kannan wrote:
>>> On 07/30/2014 02:40 PM, Rafael J. Wysocki wrote:
>>>> On Wednesday, July 30, 2014 10:18:25 AM Prarit Bhargava wrote:
>>>>>
>>>>> On 07/29/2014 08:03 PM, Rafael J. Wysocki wrote:
>>>>>> On Tuesday, July 29, 2014 07:46:02 AM Prarit Bhargava wrote:
>>>>
>>>> [cut]
>>>>
>>>>>>> This patch effectively reverts commit 955ef483.
>>>
>>> The issue reported in this patch is valid. We are seeing that internally 
>>> too. I believe I reported it in another thread (within the past month).
>>>
>>> However, the original patch fixes a real deadlock issue (I'm too tired 
>>> to look it up now). We can revet the original, but it's going to bring 
>>> back the original issue. I just want to make sure Prarit and Raphael 
>>> realize this before proceeding.
> 
> Hi Saravana,
> 
> Thanks for your input.  I went back to the code and confirmed my original
> statement about this patch.
> 
> Note: in a previous email I erroneously wrote "buffer->mutex" when I should
> have identified the lock as sysfs_mutex.  Sorry 'bout that, and apologies
> for any confusion that may have caused.
> 
> From my commit message:
> 
> "In any case, the current linux.git code no longer can reproduce the original
> failure; the locking in the sysfs release code has changed."
> 
> The original patch attempted to fix this deadlock:
> 
> A cpufreq driver on a file read did:
> 
>     -> #0 (&per_cpu(cpu_policy_rwsem, cpu)){+++++.}:
>             [<c0055253>] __lock_acquire+0xef3/0x13dc
>             [<c0055a79>] lock_acquire+0x61/0xbc
>             [<c03ee1f5>] down_read+0x25/0x30
>             [<c02f6179>] lock_policy_rwsem_read+0x25/0x34
>             [<c02f6edd>] show+0x21/0x58
>             [<c00f9c0f>] sysfs_read_file+0x67/0xcc
>             [<c00b40a7>] vfs_read+0x63/0xd8
>             [<c00b41fb>] sys_read+0x2f/0x50
>             [<c000cdc1>] ret_fast_syscall+0x1/0x52
> 
> lock(s_active#41) [ which is actually the acquisition of sysfs_mutex ]
> lock(&per_cpu(cpu_policy_rwsem, cpu));
> 
> and on the governor switch (notably the EXIT of the existing governor), the
> opposite occurs
> 
>     -> #1 (s_active#41){++++.+}:
>             [<c0055a79>] lock_acquire+0x61/0xbc
>             [<c00fabf1>] sysfs_addrm_finish+0xc1/0x128
>             [<c00f9819>] sysfs_hash_and_remove+0x35/0x64
>             [<c00fbe6f>] remove_files.isra.0+0x1b/0x24
>             [<c00fbea5>] sysfs_remove_group+0x2d/0xa8
>             [<c02f9a0b>] cpufreq_governor_interactive+0x13b/0x35c
>             [<c02f61df>] __cpufreq_governor+0x2b/0x8c
>             [<c02f6579>] __cpufreq_set_policy+0xa9/0xf8
>             [<c02f6b75>] store_scaling_governor+0x61/0x100
>             [<c02f6f4d>] store+0x39/0x60
>             [<c00f9b81>] sysfs_write_file+0xed/0x114
>             [<c00b3fd1>] vfs_write+0x65/0xd8
>             [<c00b424b>] sys_write+0x2f/0x50
>             [<c000cdc1>] ret_fast_syscall+0x1/0x52
> 
> 
> lock(&per_cpu(cpu_policy_rwsem, cpu));
> lock(s_active#41) [ which is actually the acquisition of sysfs_mutex ]
> 
> The sysfs_mutex no longer blocks in the sysfs path, and I have built with
> LOCKDEP on and off to confirm that I do not see any tracebacks or hangs.  I
> tested this by doing a few reads of the current governor, and then doing a
> governor switch (to at least initiate the LOCKDEP warning).  IIUC the traceback
> above that is the way to reproduce this LOCKDEP warning.

^^^ this should not be taken as 'I did only a few reads ...'.  I tested quite
extensively across 15 different systems and added a read of the scaling_governor
files in my little reproducer.

P.

Re: [PATCH] cpufreq, store_scaling_governor requires policy->rwsem to be held for duration of changing governors [v2]

[Prarit Bhargava]

On 07/30/2014 10:16 PM, Rafael J. Wysocki wrote:
> On Wednesday, July 30, 2014 06:36:00 PM Saravana Kannan wrote:
>> On 07/30/2014 02:40 PM, Rafael J. Wysocki wrote:
>>> On Wednesday, July 30, 2014 10:18:25 AM Prarit Bhargava wrote:
>>>>
>>>> On 07/29/2014 08:03 PM, Rafael J. Wysocki wrote:
>>>>> On Tuesday, July 29, 2014 07:46:02 AM Prarit Bhargava wrote:
>>>
>>> [cut]
>>>
>>>>>> This patch effectively reverts commit 955ef483.
>>
>> The issue reported in this patch is valid. We are seeing that internally 
>> too. I believe I reported it in another thread (within the past month).
>>
>> However, the original patch fixes a real deadlock issue (I'm too tired 
>> to look it up now). We can revet the original, but it's going to bring 
>> back the original issue. I just want to make sure Prarit and Raphael 
>> realize this before proceeding.
>>
>> I do have plans for a proper fix for the mainline (not stable branches), 
>> but plan to do that after the current set of suspend/hotplug patches go 
>> through. The fix would be easier to make after that.
>>
>>>>>
>>>>> OK, I'm convinced by this.
>>>>>
>>>>> I suppose we should push it for -stable from 3.10 through 3.15.x, right?
>>>>
>>>> Rafael, I think that is a good idea.  I'm not sure what the protocol is for
>>>> adding stable@kernel.org though ...

Rafael, let me (again) re-write the patch description.  I think Saravana has
raised an important issue that I have not clearly identified why it is safe to
remove this code in my patch description.  Also, I want to clearly identify the
appropriate -stable releases to push this out to.

I'll submit a [v3] later today or tomorrow.

P.

> 
> Rafael
> 

Re: [PATCH] cpufreq, store_scaling_governor requires policy->rwsem to be held for duration of changing governors [v2]

[Rafael J. Wysocki]

On Thursday, July 31, 2014 06:23:18 AM Prarit Bhargava wrote:
> 
> On 07/30/2014 10:16 PM, Rafael J. Wysocki wrote:
> > On Wednesday, July 30, 2014 06:36:00 PM Saravana Kannan wrote:
> >> On 07/30/2014 02:40 PM, Rafael J. Wysocki wrote:
> >>> On Wednesday, July 30, 2014 10:18:25 AM Prarit Bhargava wrote:
> >>>>
> >>>> On 07/29/2014 08:03 PM, Rafael J. Wysocki wrote:
> >>>>> On Tuesday, July 29, 2014 07:46:02 AM Prarit Bhargava wrote:
> >>>
> >>> [cut]
> >>>
> >>>>>> This patch effectively reverts commit 955ef483.
> >>
> >> The issue reported in this patch is valid. We are seeing that internally 
> >> too. I believe I reported it in another thread (within the past month).
> >>
> >> However, the original patch fixes a real deadlock issue (I'm too tired 
> >> to look it up now). We can revet the original, but it's going to bring 
> >> back the original issue. I just want to make sure Prarit and Raphael 
> >> realize this before proceeding.
> >>
> >> I do have plans for a proper fix for the mainline (not stable branches), 
> >> but plan to do that after the current set of suspend/hotplug patches go 
> >> through. The fix would be easier to make after that.
> >>
> >>>>>
> >>>>> OK, I'm convinced by this.
> >>>>>
> >>>>> I suppose we should push it for -stable from 3.10 through 3.15.x, right?
> >>>>
> >>>> Rafael, I think that is a good idea.  I'm not sure what the protocol is for
> >>>> adding stable@kernel.org though ...
> 
> Rafael, let me (again) re-write the patch description.  I think Saravana has
> raised an important issue that I have not clearly identified why it is safe to
> remove this code in my patch description.  Also, I want to clearly identify the
> appropriate -stable releases to push this out to.
> 
> I'll submit a [v3] later today or tomorrow.

In any case that's too late for 3.16 final, unless there's an -rc8.

Thanks for doing that work!

Rafael

Re: [PATCH] cpufreq, store_scaling_governor requires policy->rwsem to be held for duration of changing governors [v2]

[Prarit Bhargava]

On 07/31/2014 12:36 PM, Rafael J. Wysocki wrote:
> On Thursday, July 31, 2014 06:23:18 AM Prarit Bhargava wrote:
>>
>> On 07/30/2014 10:16 PM, Rafael J. Wysocki wrote:
>>> On Wednesday, July 30, 2014 06:36:00 PM Saravana Kannan wrote:
>>>> On 07/30/2014 02:40 PM, Rafael J. Wysocki wrote:
>>>>> On Wednesday, July 30, 2014 10:18:25 AM Prarit Bhargava wrote:
>>>>>>
>>>>>> On 07/29/2014 08:03 PM, Rafael J. Wysocki wrote:
>>>>>>> On Tuesday, July 29, 2014 07:46:02 AM Prarit Bhargava wrote:
>>>>>
>>>>> [cut]
>>>>>
>>>>>>>> This patch effectively reverts commit 955ef483.
>>>>
>>>> The issue reported in this patch is valid. We are seeing that internally 
>>>> too. I believe I reported it in another thread (within the past month).
>>>>
>>>> However, the original patch fixes a real deadlock issue (I'm too tired 
>>>> to look it up now). We can revet the original, but it's going to bring 
>>>> back the original issue. I just want to make sure Prarit and Raphael 
>>>> realize this before proceeding.
>>>>
>>>> I do have plans for a proper fix for the mainline (not stable branches), 
>>>> but plan to do that after the current set of suspend/hotplug patches go 
>>>> through. The fix would be easier to make after that.
>>>>
>>>>>>>
>>>>>>> OK, I'm convinced by this.
>>>>>>>
>>>>>>> I suppose we should push it for -stable from 3.10 through 3.15.x, right?
>>>>>>
>>>>>> Rafael, I think that is a good idea.  I'm not sure what the protocol is for
>>>>>> adding stable@kernel.org though ...
>>
>> Rafael, let me (again) re-write the patch description.  I think Saravana has
>> raised an important issue that I have not clearly identified why it is safe to
>> remove this code in my patch description.  Also, I want to clearly identify the
>> appropriate -stable releases to push this out to.
>>
>> I'll submit a [v3] later today or tomorrow.
> 
> In any case that's too late for 3.16 final, unless there's an -rc8.
> 
> Thanks for doing that work!

Ugh ... I tried this (yet another) large system and hit another panic :(.

I'm investigating now, and I'm hoping this is just something "new".

P.

> 
> Rafael
> 

Re: [PATCH] cpufreq, store_scaling_governor requires policy->rwsem to be held for duration of changing governors [v2]

[Prarit Bhargava]

On 07/31/2014 02:38 PM, Rafael J. Wysocki wrote:
> On Thursday, July 31, 2014 01:57:29 PM Prarit Bhargava wrote:
>>
>> On 07/31/2014 12:36 PM, Rafael J. Wysocki wrote:
>>> On Thursday, July 31, 2014 06:23:18 AM Prarit Bhargava wrote:
>>>>
>>>> On 07/30/2014 10:16 PM, Rafael J. Wysocki wrote:
>>>>> On Wednesday, July 30, 2014 06:36:00 PM Saravana Kannan wrote:
>>>>>> On 07/30/2014 02:40 PM, Rafael J. Wysocki wrote:
>>>>>>> On Wednesday, July 30, 2014 10:18:25 AM Prarit Bhargava wrote:
>>>>>>>>
>>>>>>>> On 07/29/2014 08:03 PM, Rafael J. Wysocki wrote:
>>>>>>>>> On Tuesday, July 29, 2014 07:46:02 AM Prarit Bhargava wrote:
>>>>>>>
>>>>>>> [cut]
>>>>>>>
>>>>>>>>>> This patch effectively reverts commit 955ef483.
>>>>>>
>>>>>> The issue reported in this patch is valid. We are seeing that internally 
>>>>>> too. I believe I reported it in another thread (within the past month).
>>>>>>
>>>>>> However, the original patch fixes a real deadlock issue (I'm too tired 
>>>>>> to look it up now). We can revet the original, but it's going to bring 
>>>>>> back the original issue. I just want to make sure Prarit and Raphael 
>>>>>> realize this before proceeding.
>>>>>>
>>>>>> I do have plans for a proper fix for the mainline (not stable branches), 
>>>>>> but plan to do that after the current set of suspend/hotplug patches go 
>>>>>> through. The fix would be easier to make after that.
>>>>>>
>>>>>>>>>
>>>>>>>>> OK, I'm convinced by this.
>>>>>>>>>
>>>>>>>>> I suppose we should push it for -stable from 3.10 through 3.15.x, right?
>>>>>>>>
>>>>>>>> Rafael, I think that is a good idea.  I'm not sure what the protocol is for
>>>>>>>> adding stable@kernel.org though ...
>>>>
>>>> Rafael, let me (again) re-write the patch description.  I think Saravana has
>>>> raised an important issue that I have not clearly identified why it is safe to
>>>> remove this code in my patch description.  Also, I want to clearly identify the
>>>> appropriate -stable releases to push this out to.
>>>>
>>>> I'll submit a [v3] later today or tomorrow.
>>>
>>> In any case that's too late for 3.16 final, unless there's an -rc8.
>>>
>>> Thanks for doing that work!
>>
>> Ugh ... I tried this (yet another) large system and hit another panic :(.
>>
>> I'm investigating now, and I'm hoping this is just something "new".
> 
> Well, I've applied your patch as is and I can push it to Linus.
> 
> However, if you want to update the changelog, I'll not do that, but in that
> case the patch will have to wait for the next week.

Rafael, please let it wait for next week.  I _need_ to make sure this is correct
and I'd rather not pushed something half-done.

Thanks,

P.

> 
> Rafael
> 

Re: [PATCH] cpufreq, store_scaling_governor requires policy->rwsem to be held for duration of changing governors [v2]

[Rafael J. Wysocki]

On Thursday, July 31, 2014 01:57:29 PM Prarit Bhargava wrote:
> 
> On 07/31/2014 12:36 PM, Rafael J. Wysocki wrote:
> > On Thursday, July 31, 2014 06:23:18 AM Prarit Bhargava wrote:
> >>
> >> On 07/30/2014 10:16 PM, Rafael J. Wysocki wrote:
> >>> On Wednesday, July 30, 2014 06:36:00 PM Saravana Kannan wrote:
> >>>> On 07/30/2014 02:40 PM, Rafael J. Wysocki wrote:
> >>>>> On Wednesday, July 30, 2014 10:18:25 AM Prarit Bhargava wrote:
> >>>>>>
> >>>>>> On 07/29/2014 08:03 PM, Rafael J. Wysocki wrote:
> >>>>>>> On Tuesday, July 29, 2014 07:46:02 AM Prarit Bhargava wrote:
> >>>>>
> >>>>> [cut]
> >>>>>
> >>>>>>>> This patch effectively reverts commit 955ef483.
> >>>>
> >>>> The issue reported in this patch is valid. We are seeing that internally 
> >>>> too. I believe I reported it in another thread (within the past month).
> >>>>
> >>>> However, the original patch fixes a real deadlock issue (I'm too tired 
> >>>> to look it up now). We can revet the original, but it's going to bring 
> >>>> back the original issue. I just want to make sure Prarit and Raphael 
> >>>> realize this before proceeding.
> >>>>
> >>>> I do have plans for a proper fix for the mainline (not stable branches), 
> >>>> but plan to do that after the current set of suspend/hotplug patches go 
> >>>> through. The fix would be easier to make after that.
> >>>>
> >>>>>>>
> >>>>>>> OK, I'm convinced by this.
> >>>>>>>
> >>>>>>> I suppose we should push it for -stable from 3.10 through 3.15.x, right?
> >>>>>>
> >>>>>> Rafael, I think that is a good idea.  I'm not sure what the protocol is for
> >>>>>> adding stable@kernel.org though ...
> >>
> >> Rafael, let me (again) re-write the patch description.  I think Saravana has
> >> raised an important issue that I have not clearly identified why it is safe to
> >> remove this code in my patch description.  Also, I want to clearly identify the
> >> appropriate -stable releases to push this out to.
> >>
> >> I'll submit a [v3] later today or tomorrow.
> > 
> > In any case that's too late for 3.16 final, unless there's an -rc8.
> > 
> > Thanks for doing that work!
> 
> Ugh ... I tried this (yet another) large system and hit another panic :(.
> 
> I'm investigating now, and I'm hoping this is just something "new".

Well, I've applied your patch as is and I can push it to Linus.

However, if you want to update the changelog, I'll not do that, but in that
case the patch will have to wait for the next week.

Rafael

Re: [PATCH] cpufreq, store_scaling_governor requires policy->rwsem to be held for duration of changing governors [v2]

[Saravana Kannan]

On 07/31/2014 11:26 AM, Prarit Bhargava wrote:
>
>
> On 07/31/2014 02:38 PM, Rafael J. Wysocki wrote:
>> On Thursday, July 31, 2014 01:57:29 PM Prarit Bhargava wrote:
>>>
>>> On 07/31/2014 12:36 PM, Rafael J. Wysocki wrote:
>>>> On Thursday, July 31, 2014 06:23:18 AM Prarit Bhargava wrote:
>>>>>
>>>>> On 07/30/2014 10:16 PM, Rafael J. Wysocki wrote:
>>>>>> On Wednesday, July 30, 2014 06:36:00 PM Saravana Kannan wrote:
>>>>>>> On 07/30/2014 02:40 PM, Rafael J. Wysocki wrote:
>>>>>>>> On Wednesday, July 30, 2014 10:18:25 AM Prarit Bhargava wrote:
>>>>>>>>>
>>>>>>>>> On 07/29/2014 08:03 PM, Rafael J. Wysocki wrote:
>>>>>>>>>> On Tuesday, July 29, 2014 07:46:02 AM Prarit Bhargava wrote:
>>>>>>>>
>>>>>>>> [cut]
>>>>>>>>
>>>>>>>>>>> This patch effectively reverts commit 955ef483.
>>>>>>>
>>>>>>> The issue reported in this patch is valid. We are seeing that internally
>>>>>>> too. I believe I reported it in another thread (within the past month).
>>>>>>>
>>>>>>> However, the original patch fixes a real deadlock issue (I'm too tired
>>>>>>> to look it up now). We can revet the original, but it's going to bring
>>>>>>> back the original issue. I just want to make sure Prarit and Raphael
>>>>>>> realize this before proceeding.
>>>>>>>
>>>>>>> I do have plans for a proper fix for the mainline (not stable branches),
>>>>>>> but plan to do that after the current set of suspend/hotplug patches go
>>>>>>> through. The fix would be easier to make after that.
>>>>>>>
>>>>>>>>>>
>>>>>>>>>> OK, I'm convinced by this.
>>>>>>>>>>
>>>>>>>>>> I suppose we should push it for -stable from 3.10 through 3.15.x, right?
>>>>>>>>>
>>>>>>>>> Rafael, I think that is a good idea.  I'm not sure what the protocol is for
>>>>>>>>> adding stable@kernel.org though ...
>>>>>
>>>>> Rafael, let me (again) re-write the patch description.  I think Saravana has
>>>>> raised an important issue that I have not clearly identified why it is safe to
>>>>> remove this code in my patch description.  Also, I want to clearly identify the
>>>>> appropriate -stable releases to push this out to.
>>>>>
>>>>> I'll submit a [v3] later today or tomorrow.
>>>>
>>>> In any case that's too late for 3.16 final, unless there's an -rc8.
>>>>
>>>> Thanks for doing that work!
>>>
>>> Ugh ... I tried this (yet another) large system and hit another panic :(.
>>>
>>> I'm investigating now, and I'm hoping this is just something "new".
>>
>> Well, I've applied your patch as is and I can push it to Linus.
>>
>> However, if you want to update the changelog, I'll not do that, but in that
>> case the patch will have to wait for the next week.
>
> Rafael, please let it wait for next week.  I _need_ to make sure this is correct
> and I'd rather not pushed something half-done.
>

Prarit,

I'm not an expert on sysfs locking, but I would think the specific sysfs 
lock would depend on the file/attribute group. So, can you please try to 
hotplug a core in/out (to trigger the POLICY_EXIT) and then read a sysfs 
file exported by the governor? scaling_governor doesn't cut it since 
that file is not removed on policy exit event to governor. If it's 
ondemand, try reading/write it's sampling rate file.

The main problem here is upon POLICY_EXIT to the governor, the governor 
tries to remove its sysfs file. So, if you have the policy lock held 
while sending POLICY_EXIT to the governor, you'll cause the:
lock policy
lock sysfs

But trying to read the same file would cause:
lock sysfs
lock policy

-Saravana

-- 
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
hosted by The Linux Foundation

Re: [PATCH] cpufreq, store_scaling_governor requires policy->rwsem to be held for duration of changing governors [v2]

[Prarit Bhargava]

On 07/31/2014 04:24 PM, Saravana Kannan wrote:
> On 07/31/2014 11:26 AM, Prarit Bhargava wrote:
>>
>>
>> On 07/31/2014 02:38 PM, Rafael J. Wysocki wrote:
>>> On Thursday, July 31, 2014 01:57:29 PM Prarit Bhargava wrote:
>>>>
>>>> On 07/31/2014 12:36 PM, Rafael J. Wysocki wrote:
>>>>> On Thursday, July 31, 2014 06:23:18 AM Prarit Bhargava wrote:
>>>>>>
>>>>>> On 07/30/2014 10:16 PM, Rafael J. Wysocki wrote:
>>>>>>> On Wednesday, July 30, 2014 06:36:00 PM Saravana Kannan wrote:
>>>>>>>> On 07/30/2014 02:40 PM, Rafael J. Wysocki wrote:
>>>>>>>>> On Wednesday, July 30, 2014 10:18:25 AM Prarit Bhargava wrote:
>>>>>>>>>>
>>>>>>>>>> On 07/29/2014 08:03 PM, Rafael J. Wysocki wrote:
>>>>>>>>>>> On Tuesday, July 29, 2014 07:46:02 AM Prarit Bhargava wrote:
>>>>>>>>>
>>>>>>>>> [cut]
>>>>>>>>>
>>>>>>>>>>>> This patch effectively reverts commit 955ef483.
>>>>>>>>
>>>>>>>> The issue reported in this patch is valid. We are seeing that internally
>>>>>>>> too. I believe I reported it in another thread (within the past month).
>>>>>>>>
>>>>>>>> However, the original patch fixes a real deadlock issue (I'm too tired
>>>>>>>> to look it up now). We can revet the original, but it's going to bring
>>>>>>>> back the original issue. I just want to make sure Prarit and Raphael
>>>>>>>> realize this before proceeding.
>>>>>>>>
>>>>>>>> I do have plans for a proper fix for the mainline (not stable branches),
>>>>>>>> but plan to do that after the current set of suspend/hotplug patches go
>>>>>>>> through. The fix would be easier to make after that.
>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> OK, I'm convinced by this.
>>>>>>>>>>>
>>>>>>>>>>> I suppose we should push it for -stable from 3.10 through 3.15.x, right?
>>>>>>>>>>
>>>>>>>>>> Rafael, I think that is a good idea.  I'm not sure what the protocol
>>>>>>>>>> is for
>>>>>>>>>> adding stable@kernel.org though ...
>>>>>>
>>>>>> Rafael, let me (again) re-write the patch description.  I think Saravana has
>>>>>> raised an important issue that I have not clearly identified why it is
>>>>>> safe to
>>>>>> remove this code in my patch description.  Also, I want to clearly
>>>>>> identify the
>>>>>> appropriate -stable releases to push this out to.
>>>>>>
>>>>>> I'll submit a [v3] later today or tomorrow.
>>>>>
>>>>> In any case that's too late for 3.16 final, unless there's an -rc8.
>>>>>
>>>>> Thanks for doing that work!
>>>>
>>>> Ugh ... I tried this (yet another) large system and hit another panic :(.
>>>>
>>>> I'm investigating now, and I'm hoping this is just something "new".
>>>
>>> Well, I've applied your patch as is and I can push it to Linus.
>>>
>>> However, if you want to update the changelog, I'll not do that, but in that
>>> case the patch will have to wait for the next week.
>>
>> Rafael, please let it wait for next week.  I _need_ to make sure this is correct
>> and I'd rather not pushed something half-done.
>>
> 
> Prarit,
> 
> I'm not an expert on sysfs locking, but I would think the specific sysfs lock
> would depend on the file/attribute group. So, can you please try to hotplug a
> core in/out (to trigger the POLICY_EXIT) and then read a sysfs file exported by
> the governor? scaling_governor doesn't cut it since that file is not removed on
> policy exit event to governor. If it's ondemand, try reading/write it's sampling
> rate file.

Thanks Saravana -- will do.  I will get back to you shortly on this.

P.

> 
> -Saravana
> 

Re: [PATCH] cpufreq, store_scaling_governor requires policy->rwsem to be held for duration of changing governors [v2]

[Saravana Kannan]

On 07/31/2014 01:30 PM, Prarit Bhargava wrote:
>
>
> On 07/31/2014 04:24 PM, Saravana Kannan wrote:
>>
>> Prarit,
>>
>> I'm not an expert on sysfs locking, but I would think the specific sysfs lock
>> would depend on the file/attribute group. So, can you please try to hotplug a
>> core in/out (to trigger the POLICY_EXIT) and then read a sysfs file exported by
>> the governor? scaling_governor doesn't cut it since that file is not removed on
>> policy exit event to governor. If it's ondemand, try reading/write it's sampling
>> rate file.
>
> Thanks Saravana -- will do.  I will get back to you shortly on this.
>

Thanks. Btw, in case you weren't already aware of it. You'll have to 
hoplug out all the CPUs in a cluster to trigger a POLICY_EXIT for that 
cluster/policy.

-Saravana

-- 
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
hosted by The Linux Foundation

Re: [PATCH] cpufreq, store_scaling_governor requires policy->rwsem to be held for duration of changing governors [v2]

[Prarit Bhargava]

On 07/31/2014 04:38 PM, Saravana Kannan wrote:
> On 07/31/2014 01:30 PM, Prarit Bhargava wrote:
>>
>>
>> On 07/31/2014 04:24 PM, Saravana Kannan wrote:
>>>
>>> Prarit,
>>>
>>> I'm not an expert on sysfs locking, but I would think the specific sysfs lock
>>> would depend on the file/attribute group. So, can you please try to hotplug a
>>> core in/out (to trigger the POLICY_EXIT) and then read a sysfs file exported by
>>> the governor? scaling_governor doesn't cut it since that file is not removed on
>>> policy exit event to governor. If it's ondemand, try reading/write it's sampling
>>> rate file.
>>
>> Thanks Saravana -- will do.  I will get back to you shortly on this.
>>
> 
> Thanks. Btw, in case you weren't already aware of it. You'll have to hoplug out
> all the CPUs in a cluster to trigger a POLICY_EXIT for that cluster/policy.

Yep -- the affected_cpus file should show all the cpus in the policy IIRC.  One
of the systems I have has 1 cpu/policy and has 48 threads so the POLICY_EXIT is
called.

I'll put something like

while [1];
do
echo ondemand > /sys/devices/system/cpu/cpu1/cpufreq/scaling_governor
cat /sys/devices/system/cpu/cpufreq/ondemand/sampling_rate
echo 20000 > /sys/devices/system/cpu/cpufreq/ondemand/sampling_rate
cat /sys/devices/system/cpu/cpufreq/ondemand/sampling_rate
echo 0 > /sys/devices/system/cpu/cpu1/online
sleep 1
echo 1 > /sys/devices/system/cpu/cpu1/online
sleep 1
done

and let it run unless you can think of something else.  FWIW, manually trying
that yields no problems AFAICT.  The sleeps are just me being paranoid and
keeping things in sequence.

P.
> 
> -Saravana
> 

Re: [PATCH] cpufreq, store_scaling_governor requires policy->rwsem to be held for duration of changing governors [v2]

[Saravana Kannan]

On 07/31/2014 02:08 PM, Prarit Bhargava wrote:
>
>
> On 07/31/2014 04:38 PM, Saravana Kannan wrote:
>> On 07/31/2014 01:30 PM, Prarit Bhargava wrote:
>>>
>>>
>>> On 07/31/2014 04:24 PM, Saravana Kannan wrote:
>>>>
>>>> Prarit,
>>>>
>>>> I'm not an expert on sysfs locking, but I would think the specific sysfs lock
>>>> would depend on the file/attribute group. So, can you please try to hotplug a
>>>> core in/out (to trigger the POLICY_EXIT) and then read a sysfs file exported by
>>>> the governor? scaling_governor doesn't cut it since that file is not removed on
>>>> policy exit event to governor. If it's ondemand, try reading/write it's sampling
>>>> rate file.
>>>
>>> Thanks Saravana -- will do.  I will get back to you shortly on this.
>>>
>>
>> Thanks. Btw, in case you weren't already aware of it. You'll have to hoplug out
>> all the CPUs in a cluster to trigger a POLICY_EXIT for that cluster/policy.
>
> Yep -- the affected_cpus file should show all the cpus in the policy IIRC.  One
> of the systems I have has 1 cpu/policy and has 48 threads so the POLICY_EXIT is
> called.
>
> I'll put something like
>
> while [1];
> do
> echo ondemand > /sys/devices/system/cpu/cpu1/cpufreq/scaling_governor
> cat /sys/devices/system/cpu/cpufreq/ondemand/sampling_rate
> echo 20000 > /sys/devices/system/cpu/cpufreq/ondemand/sampling_rate
> cat /sys/devices/system/cpu/cpufreq/ondemand/sampling_rate
> echo 0 > /sys/devices/system/cpu/cpu1/online
> sleep 1
> echo 1 > /sys/devices/system/cpu/cpu1/online
> sleep 1
> done
>

The actual race can only happen with 2 threads. I'm just trying to 
trigger a lockdep warning here.

-Saravana

-- 
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
hosted by The Linux Foundation

Re: [PATCH] cpufreq, store_scaling_governor requires policy->rwsem to be held for duration of changing governors [v2]

[Prarit Bhargava]

On 07/31/2014 06:13 PM, Saravana Kannan wrote:
> On 07/31/2014 02:08 PM, Prarit Bhargava wrote:
>>
>>
>> On 07/31/2014 04:38 PM, Saravana Kannan wrote:
>>> On 07/31/2014 01:30 PM, Prarit Bhargava wrote:
>>>>
>>>>
>>>> On 07/31/2014 04:24 PM, Saravana Kannan wrote:
>>>>>
>>>>> Prarit,
>>>>>
>>>>> I'm not an expert on sysfs locking, but I would think the specific sysfs lock
>>>>> would depend on the file/attribute group. So, can you please try to hotplug a
>>>>> core in/out (to trigger the POLICY_EXIT) and then read a sysfs file
>>>>> exported by
>>>>> the governor? scaling_governor doesn't cut it since that file is not
>>>>> removed on
>>>>> policy exit event to governor. If it's ondemand, try reading/write it's
>>>>> sampling
>>>>> rate file.
>>>>
>>>> Thanks Saravana -- will do.  I will get back to you shortly on this.
>>>>
>>>
>>> Thanks. Btw, in case you weren't already aware of it. You'll have to hoplug out
>>> all the CPUs in a cluster to trigger a POLICY_EXIT for that cluster/policy.
>>
>> Yep -- the affected_cpus file should show all the cpus in the policy IIRC.  One
>> of the systems I have has 1 cpu/policy and has 48 threads so the POLICY_EXIT is
>> called.
>>
>> I'll put something like
>>
>> while [1];
>> do
>> echo ondemand > /sys/devices/system/cpu/cpu1/cpufreq/scaling_governor
>> cat /sys/devices/system/cpu/cpufreq/ondemand/sampling_rate
>> echo 20000 > /sys/devices/system/cpu/cpufreq/ondemand/sampling_rate
>> cat /sys/devices/system/cpu/cpufreq/ondemand/sampling_rate
>> echo 0 > /sys/devices/system/cpu/cpu1/online
>> sleep 1
>> echo 1 > /sys/devices/system/cpu/cpu1/online
>> sleep 1
>> done
>>
> 
> The actual race can only happen with 2 threads. I'm just trying to trigger a
> lockdep warning here.

I ran the above in two separate terminals with cpuset -c 0 and cpuset -c 1 to
multi-thread it all.  No deadlock or LOCKDEP trace after about 1/2 hour, so I
think we're in the clear on that concern.

P.

Re: [PATCH] cpufreq, store_scaling_governor requires policy->rwsem to be held for duration of changing governors [v2]

[Saravana Kannan]

On 07/31/2014 03:58 PM, Prarit Bhargava wrote:
>
>
> On 07/31/2014 06:13 PM, Saravana Kannan wrote:
>> On 07/31/2014 02:08 PM, Prarit Bhargava wrote:
>>>
>>>
>>> On 07/31/2014 04:38 PM, Saravana Kannan wrote:
>>>> On 07/31/2014 01:30 PM, Prarit Bhargava wrote:
>>>>>
>>>>>
>>>>> On 07/31/2014 04:24 PM, Saravana Kannan wrote:
>>>>>>
>>>>>> Prarit,
>>>>>>
>>>>>> I'm not an expert on sysfs locking, but I would think the specific sysfs lock
>>>>>> would depend on the file/attribute group. So, can you please try to hotplug a
>>>>>> core in/out (to trigger the POLICY_EXIT) and then read a sysfs file
>>>>>> exported by
>>>>>> the governor? scaling_governor doesn't cut it since that file is not
>>>>>> removed on
>>>>>> policy exit event to governor. If it's ondemand, try reading/write it's
>>>>>> sampling
>>>>>> rate file.
>>>>>
>>>>> Thanks Saravana -- will do.  I will get back to you shortly on this.
>>>>>
>>>>
>>>> Thanks. Btw, in case you weren't already aware of it. You'll have to hoplug out
>>>> all the CPUs in a cluster to trigger a POLICY_EXIT for that cluster/policy.
>>>
>>> Yep -- the affected_cpus file should show all the cpus in the policy IIRC.  One
>>> of the systems I have has 1 cpu/policy and has 48 threads so the POLICY_EXIT is
>>> called.
>>>
>>> I'll put something like
>>>
>>> while [1];
>>> do
>>> echo ondemand > /sys/devices/system/cpu/cpu1/cpufreq/scaling_governor
>>> cat /sys/devices/system/cpu/cpufreq/ondemand/sampling_rate
>>> echo 20000 > /sys/devices/system/cpu/cpufreq/ondemand/sampling_rate
>>> cat /sys/devices/system/cpu/cpufreq/ondemand/sampling_rate
>>> echo 0 > /sys/devices/system/cpu/cpu1/online
>>> sleep 1
>>> echo 1 > /sys/devices/system/cpu/cpu1/online
>>> sleep 1
>>> done
>>>
>>
>> The actual race can only happen with 2 threads. I'm just trying to trigger a
>> lockdep warning here.
>
> I ran the above in two separate terminals with cpuset -c 0 and cpuset -c 1 to
> multi-thread it all.  No deadlock or LOCKDEP trace after about 1/2 hour, so I
> think we're in the clear on that concern.
>

I wasn't convinced. So, I took some help from Stephen to test it.

It's been a while, so I didn't remember the original issue clearly when 
I gave you some test suggestions. Now that I looked at the code more 
closely, I have a proper way to reproduce the original issue.

Nack for this patch for 2 reasons:
1. You seem to have accidentally removed a GOV_STOP in your patch. We 
definitely can't do that. This broke changing governors and that's why 
your patch didn't cause any issues. Because all your governor echos were 
failing.
2. When we fixed that and actually tried a proper test (not the one I 
gave you), we reproduced the original issue.

To reproduce original issue:
Preconditions:
* lockdep is enabled
* governor per policy is enabled

Steps:
1. Set governor to ondemand.
2. Cat one of the ondemand sysfs files.
3. Change governor to conservative.

When you do that, there's an AB, BA dead lock issue with one thread 
trying to cat a governor sysfs file and another thread trying to change 
governors.

-Saravana

-- 
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
hosted by The Linux Foundation

Re: [PATCH] cpufreq, store_scaling_governor requires policy->rwsem to be held for duration of changing governors [v2]

[Prarit Bhargava]

On 07/31/2014 08:55 PM, Saravana Kannan wrote:
> On 07/31/2014 03:58 PM, Prarit Bhargava wrote:
>>
>>
>> On 07/31/2014 06:13 PM, Saravana Kannan wrote:
>>> On 07/31/2014 02:08 PM, Prarit Bhargava wrote:
>>>>
>>>>
>>>> On 07/31/2014 04:38 PM, Saravana Kannan wrote:
>>>>> On 07/31/2014 01:30 PM, Prarit Bhargava wrote:
>>>>>>
>>>>>>
>>>>>> On 07/31/2014 04:24 PM, Saravana Kannan wrote:
>>>>>>>
>>>>>>> Prarit,
>>>>>>>
>>>>>>> I'm not an expert on sysfs locking, but I would think the specific sysfs
>>>>>>> lock
>>>>>>> would depend on the file/attribute group. So, can you please try to
>>>>>>> hotplug a
>>>>>>> core in/out (to trigger the POLICY_EXIT) and then read a sysfs file
>>>>>>> exported by
>>>>>>> the governor? scaling_governor doesn't cut it since that file is not
>>>>>>> removed on
>>>>>>> policy exit event to governor. If it's ondemand, try reading/write it's
>>>>>>> sampling
>>>>>>> rate file.
>>>>>>
>>>>>> Thanks Saravana -- will do.  I will get back to you shortly on this.
>>>>>>
>>>>>
>>>>> Thanks. Btw, in case you weren't already aware of it. You'll have to hoplug
>>>>> out
>>>>> all the CPUs in a cluster to trigger a POLICY_EXIT for that cluster/policy.
>>>>
>>>> Yep -- the affected_cpus file should show all the cpus in the policy IIRC.  One
>>>> of the systems I have has 1 cpu/policy and has 48 threads so the POLICY_EXIT is
>>>> called.
>>>>
>>>> I'll put something like
>>>>
>>>> while [1];
>>>> do
>>>> echo ondemand > /sys/devices/system/cpu/cpu1/cpufreq/scaling_governor
>>>> cat /sys/devices/system/cpu/cpufreq/ondemand/sampling_rate
>>>> echo 20000 > /sys/devices/system/cpu/cpufreq/ondemand/sampling_rate
>>>> cat /sys/devices/system/cpu/cpufreq/ondemand/sampling_rate
>>>> echo 0 > /sys/devices/system/cpu/cpu1/online
>>>> sleep 1
>>>> echo 1 > /sys/devices/system/cpu/cpu1/online
>>>> sleep 1
>>>> done
>>>>
>>>
>>> The actual race can only happen with 2 threads. I'm just trying to trigger a
>>> lockdep warning here.
>>
>> I ran the above in two separate terminals with cpuset -c 0 and cpuset -c 1 to
>> multi-thread it all.  No deadlock or LOCKDEP trace after about 1/2 hour, so I
>> think we're in the clear on that concern.
>>
> 
> I wasn't convinced. So, I took some help from Stephen to test it.
> 
> It's been a while, so I didn't remember the original issue clearly when I gave
> you some test suggestions. Now that I looked at the code more closely, I have a
> proper way to reproduce the original issue.
> 
> Nack for this patch for 2 reasons:
> 1. You seem to have accidentally removed a GOV_STOP in your patch. We definitely
> can't do that. This broke changing governors and that's why your patch didn't
> cause any issues. Because all your governor echos were failing.

Yes, I mentioned that yesterday in the thread.

> 2. When we fixed that and actually tried a proper test (not the one I gave you),
> we reproduced the original issue.
> 
> To reproduce original issue:
> Preconditions:
> * lockdep is enabled
> * governor per policy is enabled

Okay.  I have that ...

> 
> Steps:
> 1. Set governor to ondemand.
> 2. Cat one of the ondemand sysfs files.
> 3. Change governor to conservative.
> 
> When you do that, there's an AB, BA dead lock issue with one thread trying to
> cat a governor sysfs file and another thread trying to change governors.

Okay, I'm going to try this too...

P.

> 
> -Saravana
> 

Re: [PATCH] cpufreq, store_scaling_governor requires policy->rwsem to be held for duration of changing governors [v2]

[Prarit Bhargava]

On 07/31/2014 08:55 PM, Saravana Kannan wrote:
> On 07/31/2014 03:58 PM, Prarit Bhargava wrote:
>>
>>
>> On 07/31/2014 06:13 PM, Saravana Kannan wrote:
>>> On 07/31/2014 02:08 PM, Prarit Bhargava wrote:
>>>>
>>>>
>>>> On 07/31/2014 04:38 PM, Saravana Kannan wrote:
>>>>> On 07/31/2014 01:30 PM, Prarit Bhargava wrote:
>>>>>>
>>>>>>
>>>>>> On 07/31/2014 04:24 PM, Saravana Kannan wrote:
>>>>>>>
>>>>>>> Prarit,
>>>>>>>
>>>>>>> I'm not an expert on sysfs locking, but I would think the specific sysfs
>>>>>>> lock
>>>>>>> would depend on the file/attribute group. So, can you please try to
>>>>>>> hotplug a
>>>>>>> core in/out (to trigger the POLICY_EXIT) and then read a sysfs file
>>>>>>> exported by
>>>>>>> the governor? scaling_governor doesn't cut it since that file is not
>>>>>>> removed on
>>>>>>> policy exit event to governor. If it's ondemand, try reading/write it's
>>>>>>> sampling
>>>>>>> rate file.
>>>>>>
>>>>>> Thanks Saravana -- will do.  I will get back to you shortly on this.
>>>>>>
>>>>>
>>>>> Thanks. Btw, in case you weren't already aware of it. You'll have to hoplug
>>>>> out
>>>>> all the CPUs in a cluster to trigger a POLICY_EXIT for that cluster/policy.
>>>>
>>>> Yep -- the affected_cpus file should show all the cpus in the policy IIRC.  One
>>>> of the systems I have has 1 cpu/policy and has 48 threads so the POLICY_EXIT is
>>>> called.
>>>>
>>>> I'll put something like
>>>>
>>>> while [1];
>>>> do
>>>> echo ondemand > /sys/devices/system/cpu/cpu1/cpufreq/scaling_governor
>>>> cat /sys/devices/system/cpu/cpufreq/ondemand/sampling_rate
>>>> echo 20000 > /sys/devices/system/cpu/cpufreq/ondemand/sampling_rate
>>>> cat /sys/devices/system/cpu/cpufreq/ondemand/sampling_rate
>>>> echo 0 > /sys/devices/system/cpu/cpu1/online
>>>> sleep 1
>>>> echo 1 > /sys/devices/system/cpu/cpu1/online
>>>> sleep 1
>>>> done
>>>>
>>>
>>> The actual race can only happen with 2 threads. I'm just trying to trigger a
>>> lockdep warning here.
>>
>> I ran the above in two separate terminals with cpuset -c 0 and cpuset -c 1 to
>> multi-thread it all.  No deadlock or LOCKDEP trace after about 1/2 hour, so I
>> think we're in the clear on that concern.
>>
> 
> I wasn't convinced. So, I took some help from Stephen to test it.
> 
> It's been a while, so I didn't remember the original issue clearly when I gave
> you some test suggestions. Now that I looked at the code more closely, I have a
> proper way to reproduce the original issue.
> 
> Nack for this patch for 2 reasons:
> 1. You seem to have accidentally removed a GOV_STOP in your patch. We definitely
> can't do that. This broke changing governors and that's why your patch didn't
> cause any issues. Because all your governor echos were failing.
> 2. When we fixed that and actually tried a proper test (not the one I gave you),
> we reproduced the original issue.
> 
> To reproduce original issue:
> Preconditions:
> * lockdep is enabled
> * governor per policy is enabled
> 
> Steps:
> 1. Set governor to ondemand.
> 2. Cat one of the ondemand sysfs files.
> 3. Change governor to conservative.

Can you send me the test and the trace of the deadlock?  I'm not creating it with:

diff --git a/drivers/cpufreq/cpufreq.c b/drivers/cpufreq/cpufreq.c
index 6f02485..fa11a7d 100644
--- a/drivers/cpufreq/cpufreq.c
+++ b/drivers/cpufreq/cpufreq.c
@@ -2200,9 +2200,7 @@ static int cpufreq_set_policy(struct cpufreq_policy *polic
        /* end old governor */
        if (old_gov) {
                __cpufreq_governor(policy, CPUFREQ_GOV_STOP);
-               up_write(&policy->rwsem);
                __cpufreq_governor(policy, CPUFREQ_GOV_POLICY_EXIT);
-               down_write(&policy->rwsem);
        }

        /* start new governor */
@@ -2211,9 +2209,7 @@ static int cpufreq_set_policy(struct cpufreq_policy *polic
                if (!__cpufreq_governor(policy, CPUFREQ_GOV_START))
                        goto out;

-               up_write(&policy->rwsem);
                __cpufreq_governor(policy, CPUFREQ_GOV_POLICY_EXIT);
-               down_write(&policy->rwsem);
        }

        /* new governor failed, so re-start old one */

> 
> When you do that, there's an AB, BA dead lock issue with one thread trying to
> cat a governor sysfs file and another thread trying to change governors.
> 



> -Saravana
> 

Re: [PATCH] cpufreq, store_scaling_governor requires policy->rwsem to be held for duration of changing governors [v2]

[Stephen Boyd]

On 08/01/14 03:27, Prarit Bhargava wrote:
>
> Can you send me the test and the trace of the deadlock?  I'm not creating it with:
>

This was with conservative as the default, and switching to ondemand

# cd /sys/devices/system/cpu/cpu2/cpufreq
# ls
affected_cpus                  scaling_available_governors
conservative                   scaling_cur_freq
cpuinfo_cur_freq               scaling_driver
cpuinfo_max_freq               scaling_governor
cpuinfo_min_freq               scaling_max_freq
cpuinfo_transition_latency     scaling_min_freq
related_cpus                   scaling_setspeed
scaling_available_frequencies  stats
# cat conservative/down_threshold
20
# echo ondemand > scaling_governor

 ======================================================
 [ INFO: possible circular locking dependency detected ]
 3.16.0-rc3-00039-ge1e38f124d87 #47 Not tainted
 -------------------------------------------------------
 sh/75 is trying to acquire lock:
  (s_active#9){++++..}, at: [<c0358a94>] kernfs_remove_by_name_ns+0x3c/0x84

 but task is already holding lock:
  (&policy->rwsem){+++++.}, at: [<c05ab1f0>] store+0x68/0xb8

 which lock already depends on the new lock.


 the existing dependency chain (in reverse order) is:

-> #1 (&policy->rwsem){+++++.}:
        [<c0359234>] kernfs_fop_open+0x138/0x298
        [<c02fa3f4>] do_dentry_open.isra.12+0x1b0/0x2f0
        [<c02fa604>] finish_open+0x20/0x38
        [<c0308d34>] do_last.isra.37+0x5ac/0xb68
        [<c03093a4>] path_openat+0xb4/0x5d8
        [<c0309bcc>] do_filp_open+0x2c/0x80
        [<c02fb558>] do_sys_open+0x10c/0x1c8
        [<c020f0a0>] ret_fast_syscall+0x0/0x48

-> #0 (s_active#9){++++..}:
        [<c0357d18>] __kernfs_remove+0x250/0x300
        [<c0358a94>] kernfs_remove_by_name_ns+0x3c/0x84
        [<c035aa78>] remove_files+0x34/0x78
        [<c035aee0>] sysfs_remove_group+0x40/0x98
        [<c05b0560>] cpufreq_governor_dbs+0x4c0/0x6ec
        [<c05abebc>] __cpufreq_governor+0x118/0x200
        [<c05ac0fc>] cpufreq_set_policy+0x158/0x2ac
        [<c05ad5e4>] store_scaling_governor+0x6c/0x94
        [<c05ab210>] store+0x88/0xb8
        [<c035a00c>] sysfs_kf_write+0x4c/0x50
        [<c03594d4>] kernfs_fop_write+0xc0/0x180
        [<c02fc5c8>] vfs_write+0xa0/0x1a8
        [<c02fc9d4>] SyS_write+0x40/0x8c
        [<c020f0a0>] ret_fast_syscall+0x0/0x48

 other info that might help us debug this:

  Possible unsafe locking scenario:

        CPU0                    CPU1
        ----                    ----
   lock(&policy->rwsem);
                                lock(s_active#9);
                                lock(&policy->rwsem);
   lock(s_active#9);

  *** DEADLOCK ***

 6 locks held by sh/75:
  #0:  (sb_writers#4){.+.+..}, at: [<c02fc6a8>] vfs_write+0x180/0x1a8
  #1:  (&of->mutex){+.+...}, at: [<c0359498>] kernfs_fop_write+0x84/0x180
  #2:  (s_active#10){.+.+..}, at: [<c03594a0>] kernfs_fop_write+0x8c/0x180
  #3:  (cpu_hotplug.lock){++++++}, at: [<c0221ef8>] get_online_cpus+0x38/0x9c
  #4:  (cpufreq_rwsem){.+.+.+}, at: [<c05ab1d8>] store+0x50/0xb8
  #5:  (&policy->rwsem){+++++.}, at: [<c05ab1f0>] store+0x68/0xb8

 stack backtrace:
 CPU: 0 PID: 75 Comm: sh Not tainted 3.16.0-rc3-00039-ge1e38f124d87 #47
 [<c0214de8>] (unwind_backtrace) from [<c02123f8>] (show_stack+0x10/0x14)
 [<c02123f8>] (show_stack) from [<c0709e5c>] (dump_stack+0x70/0xbc)
 [<c0709e5c>] (dump_stack) from [<c070722c>] (print_circular_bug+0x280/0x2d4)
 [<c070722c>] (print_circular_bug) from [<c02629cc>] (__lock_acquire+0x18d0/0x1abc)
 [<c02629cc>] (__lock_acquire) from [<c026310c>] (lock_acquire+0x9c/0x138)
 [<c026310c>] (lock_acquire) from [<c0357d18>] (__kernfs_remove+0x250/0x300)
 [<c0357d18>] (__kernfs_remove) from [<c0358a94>] (kernfs_remove_by_name_ns+0x3c/0x84)
 [<c0358a94>] (kernfs_remove_by_name_ns) from [<c035aa78>] (remove_files+0x34/0x78)
 [<c035aa78>] (remove_files) from [<c035aee0>] (sysfs_remove_group+0x40/0x98)
 [<c035aee0>] (sysfs_remove_group) from [<c05b0560>] (cpufreq_governor_dbs+0x4c0/0x6ec)
 [<c05b0560>] (cpufreq_governor_dbs) from [<c05abebc>] (__cpufreq_governor+0x118/0x200)
 [<c05abebc>] (__cpufreq_governor) from [<c05ac0fc>] (cpufreq_set_policy+0x158/0x2ac)
 [<c05ac0fc>] (cpufreq_set_policy) from [<c05ad5e4>] (store_scaling_governor+0x6c/0x94)
 [<c05ad5e4>] (store_scaling_governor) from [<c05ab210>] (store+0x88/0xb8)
 [<c05ab210>] (store) from [<c035a00c>] (sysfs_kf_write+0x4c/0x50)
 [<c035a00c>] (sysfs_kf_write) from [<c03594d4>] (kernfs_fop_write+0xc0/0x180)
 [<c03594d4>] (kernfs_fop_write) from [<c02fc5c8>] (vfs_write+0xa0/0x1a8)
 [<c02fc5c8>] (vfs_write) from [<c02fc9d4>] (SyS_write+0x40/0x8c)
 [<c02fc9d4>] (SyS_write) from [<c020f0a0>] (ret_fast_syscall+0x0/0x48)


-- 
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
hosted by The Linux Foundation

Re: [PATCH] cpufreq, store_scaling_governor requires policy->rwsem to be held for duration of changing governors [v2]

[Prarit Bhargava]

On 08/01/2014 01:18 PM, Stephen Boyd wrote:
> On 08/01/14 03:27, Prarit Bhargava wrote:
>>
>> Can you send me the test and the trace of the deadlock?  I'm not creating it with:
>>
> 
> This was with conservative as the default, and switching to ondemand
> 
> # cd /sys/devices/system/cpu/cpu2/cpufreq
> # ls
> affected_cpus                  scaling_available_governors
> conservative                   scaling_cur_freq
> cpuinfo_cur_freq               scaling_driver
> cpuinfo_max_freq               scaling_governor
> cpuinfo_min_freq               scaling_max_freq
> cpuinfo_transition_latency     scaling_min_freq
> related_cpus                   scaling_setspeed
> scaling_available_frequencies  stats
> # cat conservative/down_threshold
> 20
> # echo ondemand > scaling_governor

Thanks Stephen,

There's obviously a difference in our .configs.  I have a global conservative
directory, ie) /sys/devices/system/cpu/cpufreq/conservative instead of a per-cpu
governor file.

ie) what are your .config options for CPUFREQ?

Mine are:

#
# CPU Frequency scaling
#
CONFIG_CPU_FREQ=y
CONFIG_CPU_FREQ_GOV_COMMON=y
CONFIG_CPU_FREQ_STAT=m
CONFIG_CPU_FREQ_STAT_DETAILS=y
# CONFIG_CPU_FREQ_DEFAULT_GOV_PERFORMANCE is not set
# CONFIG_CPU_FREQ_DEFAULT_GOV_USERSPACE is not set
# CONFIG_CPU_FREQ_DEFAULT_GOV_ONDEMAND is not set
CONFIG_CPU_FREQ_DEFAULT_GOV_CONSERVATIVE=y
CONFIG_CPU_FREQ_GOV_PERFORMANCE=y
CONFIG_CPU_FREQ_GOV_POWERSAVE=y
CONFIG_CPU_FREQ_GOV_USERSPACE=y
CONFIG_CPU_FREQ_GOV_ONDEMAND=y
CONFIG_CPU_FREQ_GOV_CONSERVATIVE=y

Is there some other config option I have to set?

P.

> 
>  ======================================================
>  [ INFO: possible circular locking dependency detected ]
>  3.16.0-rc3-00039-ge1e38f124d87 #47 Not tainted
>  -------------------------------------------------------
>  sh/75 is trying to acquire lock:
>   (s_active#9){++++..}, at: [<c0358a94>] kernfs_remove_by_name_ns+0x3c/0x84
> 
>  but task is already holding lock:
>   (&policy->rwsem){+++++.}, at: [<c05ab1f0>] store+0x68/0xb8
> 
>  which lock already depends on the new lock.
> 
> 
>  the existing dependency chain (in reverse order) is:
> 
> -> #1 (&policy->rwsem){+++++.}:
>         [<c0359234>] kernfs_fop_open+0x138/0x298
>         [<c02fa3f4>] do_dentry_open.isra.12+0x1b0/0x2f0
>         [<c02fa604>] finish_open+0x20/0x38
>         [<c0308d34>] do_last.isra.37+0x5ac/0xb68
>         [<c03093a4>] path_openat+0xb4/0x5d8
>         [<c0309bcc>] do_filp_open+0x2c/0x80
>         [<c02fb558>] do_sys_open+0x10c/0x1c8
>         [<c020f0a0>] ret_fast_syscall+0x0/0x48
> 
> -> #0 (s_active#9){++++..}:
>         [<c0357d18>] __kernfs_remove+0x250/0x300
>         [<c0358a94>] kernfs_remove_by_name_ns+0x3c/0x84
>         [<c035aa78>] remove_files+0x34/0x78
>         [<c035aee0>] sysfs_remove_group+0x40/0x98
>         [<c05b0560>] cpufreq_governor_dbs+0x4c0/0x6ec
>         [<c05abebc>] __cpufreq_governor+0x118/0x200
>         [<c05ac0fc>] cpufreq_set_policy+0x158/0x2ac
>         [<c05ad5e4>] store_scaling_governor+0x6c/0x94
>         [<c05ab210>] store+0x88/0xb8
>         [<c035a00c>] sysfs_kf_write+0x4c/0x50
>         [<c03594d4>] kernfs_fop_write+0xc0/0x180
>         [<c02fc5c8>] vfs_write+0xa0/0x1a8
>         [<c02fc9d4>] SyS_write+0x40/0x8c
>         [<c020f0a0>] ret_fast_syscall+0x0/0x48
> 
>  other info that might help us debug this:
> 
>   Possible unsafe locking scenario:
> 
>         CPU0                    CPU1
>         ----                    ----
>    lock(&policy->rwsem);
>                                 lock(s_active#9);
>                                 lock(&policy->rwsem);
>    lock(s_active#9);
> 
>   *** DEADLOCK ***
> 
>  6 locks held by sh/75:
>   #0:  (sb_writers#4){.+.+..}, at: [<c02fc6a8>] vfs_write+0x180/0x1a8
>   #1:  (&of->mutex){+.+...}, at: [<c0359498>] kernfs_fop_write+0x84/0x180
>   #2:  (s_active#10){.+.+..}, at: [<c03594a0>] kernfs_fop_write+0x8c/0x180
>   #3:  (cpu_hotplug.lock){++++++}, at: [<c0221ef8>] get_online_cpus+0x38/0x9c
>   #4:  (cpufreq_rwsem){.+.+.+}, at: [<c05ab1d8>] store+0x50/0xb8
>   #5:  (&policy->rwsem){+++++.}, at: [<c05ab1f0>] store+0x68/0xb8
> 
>  stack backtrace:
>  CPU: 0 PID: 75 Comm: sh Not tainted 3.16.0-rc3-00039-ge1e38f124d87 #47
>  [<c0214de8>] (unwind_backtrace) from [<c02123f8>] (show_stack+0x10/0x14)
>  [<c02123f8>] (show_stack) from [<c0709e5c>] (dump_stack+0x70/0xbc)
>  [<c0709e5c>] (dump_stack) from [<c070722c>] (print_circular_bug+0x280/0x2d4)
>  [<c070722c>] (print_circular_bug) from [<c02629cc>] (__lock_acquire+0x18d0/0x1abc)
>  [<c02629cc>] (__lock_acquire) from [<c026310c>] (lock_acquire+0x9c/0x138)
>  [<c026310c>] (lock_acquire) from [<c0357d18>] (__kernfs_remove+0x250/0x300)
>  [<c0357d18>] (__kernfs_remove) from [<c0358a94>] (kernfs_remove_by_name_ns+0x3c/0x84)
>  [<c0358a94>] (kernfs_remove_by_name_ns) from [<c035aa78>] (remove_files+0x34/0x78)
>  [<c035aa78>] (remove_files) from [<c035aee0>] (sysfs_remove_group+0x40/0x98)
>  [<c035aee0>] (sysfs_remove_group) from [<c05b0560>] (cpufreq_governor_dbs+0x4c0/0x6ec)
>  [<c05b0560>] (cpufreq_governor_dbs) from [<c05abebc>] (__cpufreq_governor+0x118/0x200)
>  [<c05abebc>] (__cpufreq_governor) from [<c05ac0fc>] (cpufreq_set_policy+0x158/0x2ac)
>  [<c05ac0fc>] (cpufreq_set_policy) from [<c05ad5e4>] (store_scaling_governor+0x6c/0x94)
>  [<c05ad5e4>] (store_scaling_governor) from [<c05ab210>] (store+0x88/0xb8)
>  [<c05ab210>] (store) from [<c035a00c>] (sysfs_kf_write+0x4c/0x50)
>  [<c035a00c>] (sysfs_kf_write) from [<c03594d4>] (kernfs_fop_write+0xc0/0x180)
>  [<c03594d4>] (kernfs_fop_write) from [<c02fc5c8>] (vfs_write+0xa0/0x1a8)
>  [<c02fc5c8>] (vfs_write) from [<c02fc9d4>] (SyS_write+0x40/0x8c)
>  [<c02fc9d4>] (SyS_write) from [<c020f0a0>] (ret_fast_syscall+0x0/0x48)
> 
> 

Re: [PATCH] cpufreq, store_scaling_governor requires policy->rwsem to be held for duration of changing governors [v2]

[Stephen Boyd]

On 08/01/14 12:15, Prarit Bhargava wrote:
>
> On 08/01/2014 01:18 PM, Stephen Boyd wrote:
>> On 08/01/14 03:27, Prarit Bhargava wrote:
>>> Can you send me the test and the trace of the deadlock?  I'm not creating it with:
>>>
>> This was with conservative as the default, and switching to ondemand
>>
>> # cd /sys/devices/system/cpu/cpu2/cpufreq
>> # ls
>> affected_cpus                  scaling_available_governors
>> conservative                   scaling_cur_freq
>> cpuinfo_cur_freq               scaling_driver
>> cpuinfo_max_freq               scaling_governor
>> cpuinfo_min_freq               scaling_max_freq
>> cpuinfo_transition_latency     scaling_min_freq
>> related_cpus                   scaling_setspeed
>> scaling_available_frequencies  stats
>> # cat conservative/down_threshold
>> 20
>> # echo ondemand > scaling_governor
> Thanks Stephen,
>
> There's obviously a difference in our .configs.  I have a global conservative
> directory, ie) /sys/devices/system/cpu/cpufreq/conservative instead of a per-cpu
> governor file.
>
> ie) what are your .config options for CPUFREQ?
>
> Mine are:
>
> #
> # CPU Frequency scaling
> #
> CONFIG_CPU_FREQ=y
> CONFIG_CPU_FREQ_GOV_COMMON=y
> CONFIG_CPU_FREQ_STAT=m
> CONFIG_CPU_FREQ_STAT_DETAILS=y
> # CONFIG_CPU_FREQ_DEFAULT_GOV_PERFORMANCE is not set
> # CONFIG_CPU_FREQ_DEFAULT_GOV_USERSPACE is not set
> # CONFIG_CPU_FREQ_DEFAULT_GOV_ONDEMAND is not set
> CONFIG_CPU_FREQ_DEFAULT_GOV_CONSERVATIVE=y
> CONFIG_CPU_FREQ_GOV_PERFORMANCE=y
> CONFIG_CPU_FREQ_GOV_POWERSAVE=y
> CONFIG_CPU_FREQ_GOV_USERSPACE=y
> CONFIG_CPU_FREQ_GOV_ONDEMAND=y
> CONFIG_CPU_FREQ_GOV_CONSERVATIVE=y
>
> Is there some other config option I have to set?

I have the same options. The difference is that my driver has a governor
per policy. That's set with the CPUFREQ_HAVE_GOVERNOR_PER_POLICY flag.
If I remove that flag I can't trigger the lockdep splat anymore with
this sequence and your patch.

-- 
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
hosted by The Linux Foundation

Re: [PATCH] cpufreq, store_scaling_governor requires policy->rwsem to be held for duration of changing governors [v2]

[Prarit Bhargava]

On 08/01/2014 03:36 PM, Stephen Boyd wrote:
> On 08/01/14 12:15, Prarit Bhargava wrote:
>>
>> On 08/01/2014 01:18 PM, Stephen Boyd wrote:
>>> On 08/01/14 03:27, Prarit Bhargava wrote:
>>>> Can you send me the test and the trace of the deadlock?  I'm not creating it with:
>>>>
>>> This was with conservative as the default, and switching to ondemand
>>>
>>> # cd /sys/devices/system/cpu/cpu2/cpufreq
>>> # ls
>>> affected_cpus                  scaling_available_governors
>>> conservative                   scaling_cur_freq
>>> cpuinfo_cur_freq               scaling_driver
>>> cpuinfo_max_freq               scaling_governor
>>> cpuinfo_min_freq               scaling_max_freq
>>> cpuinfo_transition_latency     scaling_min_freq
>>> related_cpus                   scaling_setspeed
>>> scaling_available_frequencies  stats
>>> # cat conservative/down_threshold
>>> 20
>>> # echo ondemand > scaling_governor
>> Thanks Stephen,
>>
>> There's obviously a difference in our .configs.  I have a global conservative
>> directory, ie) /sys/devices/system/cpu/cpufreq/conservative instead of a per-cpu
>> governor file.
>>
>> ie) what are your .config options for CPUFREQ?
>>
>> Mine are:
>>
>> #
>> # CPU Frequency scaling
>> #
>> CONFIG_CPU_FREQ=y
>> CONFIG_CPU_FREQ_GOV_COMMON=y
>> CONFIG_CPU_FREQ_STAT=m
>> CONFIG_CPU_FREQ_STAT_DETAILS=y
>> # CONFIG_CPU_FREQ_DEFAULT_GOV_PERFORMANCE is not set
>> # CONFIG_CPU_FREQ_DEFAULT_GOV_USERSPACE is not set
>> # CONFIG_CPU_FREQ_DEFAULT_GOV_ONDEMAND is not set
>> CONFIG_CPU_FREQ_DEFAULT_GOV_CONSERVATIVE=y
>> CONFIG_CPU_FREQ_GOV_PERFORMANCE=y
>> CONFIG_CPU_FREQ_GOV_POWERSAVE=y
>> CONFIG_CPU_FREQ_GOV_USERSPACE=y
>> CONFIG_CPU_FREQ_GOV_ONDEMAND=y
>> CONFIG_CPU_FREQ_GOV_CONSERVATIVE=y
>>
>> Is there some other config option I have to set?
> 
> I have the same options. The difference is that my driver has a governor
> per policy. That's set with the CPUFREQ_HAVE_GOVERNOR_PER_POLICY flag.
> If I remove that flag I can't trigger the lockdep splat anymore with
> this sequence and your patch.

I see -- so you're seeing this on arm then?  If so, let me know so I can reserve
one to work on :)

Thanks!

P.
> 

Re: [PATCH] cpufreq, store_scaling_governor requires policy->rwsem to be held for duration of changing governors [v2]

[Stephen Boyd]

On 08/01/14 12:43, Prarit Bhargava wrote:
>
> On 08/01/2014 03:36 PM, Stephen Boyd wrote:
>> On 08/01/14 12:15, Prarit Bhargava wrote:
>>> On 08/01/2014 01:18 PM, Stephen Boyd wrote:
>>>> On 08/01/14 03:27, Prarit Bhargava wrote:
>>>>> Can you send me the test and the trace of the deadlock?  I'm not creating it with:
>>>>>
>>>> This was with conservative as the default, and switching to ondemand
>>>>
>>>> # cd /sys/devices/system/cpu/cpu2/cpufreq
>>>> # ls
>>>> affected_cpus                  scaling_available_governors
>>>> conservative                   scaling_cur_freq
>>>> cpuinfo_cur_freq               scaling_driver
>>>> cpuinfo_max_freq               scaling_governor
>>>> cpuinfo_min_freq               scaling_max_freq
>>>> cpuinfo_transition_latency     scaling_min_freq
>>>> related_cpus                   scaling_setspeed
>>>> scaling_available_frequencies  stats
>>>> # cat conservative/down_threshold
>>>> 20
>>>> # echo ondemand > scaling_governor
>>> Thanks Stephen,
>>>
>>> There's obviously a difference in our .configs.  I have a global conservative
>>> directory, ie) /sys/devices/system/cpu/cpufreq/conservative instead of a per-cpu
>>> governor file.
>>>
>>> ie) what are your .config options for CPUFREQ?
>>>
>>> Mine are:
>>>
>>> #
>>> # CPU Frequency scaling
>>> #
>>> CONFIG_CPU_FREQ=y
>>> CONFIG_CPU_FREQ_GOV_COMMON=y
>>> CONFIG_CPU_FREQ_STAT=m
>>> CONFIG_CPU_FREQ_STAT_DETAILS=y
>>> # CONFIG_CPU_FREQ_DEFAULT_GOV_PERFORMANCE is not set
>>> # CONFIG_CPU_FREQ_DEFAULT_GOV_USERSPACE is not set
>>> # CONFIG_CPU_FREQ_DEFAULT_GOV_ONDEMAND is not set
>>> CONFIG_CPU_FREQ_DEFAULT_GOV_CONSERVATIVE=y
>>> CONFIG_CPU_FREQ_GOV_PERFORMANCE=y
>>> CONFIG_CPU_FREQ_GOV_POWERSAVE=y
>>> CONFIG_CPU_FREQ_GOV_USERSPACE=y
>>> CONFIG_CPU_FREQ_GOV_ONDEMAND=y
>>> CONFIG_CPU_FREQ_GOV_CONSERVATIVE=y
>>>
>>> Is there some other config option I have to set?
>> I have the same options. The difference is that my driver has a governor
>> per policy. That's set with the CPUFREQ_HAVE_GOVERNOR_PER_POLICY flag.
>> If I remove that flag I can't trigger the lockdep splat anymore with
>> this sequence and your patch.
> I see -- so you're seeing this on arm then?  If so, let me know so I can reserve
> one to work on :)
>

I only have ARM to test on. You can set this flag in your cpufreq driver
if you have independently scaling CPU frequencies, so it isn't
necessarily an ARM thing.

-- 
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
hosted by The Linux Foundation

Re: [PATCH] cpufreq, store_scaling_governor requires policy->rwsem to be held for duration of changing governors [v2]

[Saravana Kannan]

On 08/01/2014 12:54 PM, Stephen Boyd wrote:
> On 08/01/14 12:43, Prarit Bhargava wrote:
>>
>> On 08/01/2014 03:36 PM, Stephen Boyd wrote:
>>> On 08/01/14 12:15, Prarit Bhargava wrote:
>>>> On 08/01/2014 01:18 PM, Stephen Boyd wrote:
>>>>> On 08/01/14 03:27, Prarit Bhargava wrote:
>>>>>> Can you send me the test and the trace of the deadlock?  I'm not creating it with:
>>>>>>
>>>>> This was with conservative as the default, and switching to ondemand
>>>>>
>>>>> # cd /sys/devices/system/cpu/cpu2/cpufreq
>>>>> # ls
>>>>> affected_cpus                  scaling_available_governors
>>>>> conservative                   scaling_cur_freq
>>>>> cpuinfo_cur_freq               scaling_driver
>>>>> cpuinfo_max_freq               scaling_governor
>>>>> cpuinfo_min_freq               scaling_max_freq
>>>>> cpuinfo_transition_latency     scaling_min_freq
>>>>> related_cpus                   scaling_setspeed
>>>>> scaling_available_frequencies  stats
>>>>> # cat conservative/down_threshold
>>>>> 20
>>>>> # echo ondemand > scaling_governor
>>>> Thanks Stephen,
>>>>
>>>> There's obviously a difference in our .configs.  I have a global conservative
>>>> directory, ie) /sys/devices/system/cpu/cpufreq/conservative instead of a per-cpu
>>>> governor file.
>>>>
>>>> ie) what are your .config options for CPUFREQ?
>>>>
>>>> Mine are:
>>>>
>>>> #
>>>> # CPU Frequency scaling
>>>> #
>>>> CONFIG_CPU_FREQ=y
>>>> CONFIG_CPU_FREQ_GOV_COMMON=y
>>>> CONFIG_CPU_FREQ_STAT=m
>>>> CONFIG_CPU_FREQ_STAT_DETAILS=y
>>>> # CONFIG_CPU_FREQ_DEFAULT_GOV_PERFORMANCE is not set
>>>> # CONFIG_CPU_FREQ_DEFAULT_GOV_USERSPACE is not set
>>>> # CONFIG_CPU_FREQ_DEFAULT_GOV_ONDEMAND is not set
>>>> CONFIG_CPU_FREQ_DEFAULT_GOV_CONSERVATIVE=y
>>>> CONFIG_CPU_FREQ_GOV_PERFORMANCE=y
>>>> CONFIG_CPU_FREQ_GOV_POWERSAVE=y
>>>> CONFIG_CPU_FREQ_GOV_USERSPACE=y
>>>> CONFIG_CPU_FREQ_GOV_ONDEMAND=y
>>>> CONFIG_CPU_FREQ_GOV_CONSERVATIVE=y
>>>>
>>>> Is there some other config option I have to set?
>>> I have the same options. The difference is that my driver has a governor
>>> per policy. That's set with the CPUFREQ_HAVE_GOVERNOR_PER_POLICY flag.
>>> If I remove that flag I can't trigger the lockdep splat anymore with
>>> this sequence and your patch.
>> I see -- so you're seeing this on arm then?  If so, let me know so I can reserve
>> one to work on :)
>>
>
> I only have ARM to test on. You can set this flag in your cpufreq driver
> if you have independently scaling CPU frequencies, so it isn't
> necessarily an ARM thing.
>

Sorry, forgot to mention this in the earlier email. But yeah, this is a 
totally SW feature. You should be able to enable this flag in your driver.

-Saravana

-- 
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
hosted by The Linux Foundation

Re: [PATCH] cpufreq, store_scaling_governor requires policy->rwsem to be held for duration of changing governors [v2]

[Prarit Bhargava]

On 08/01/2014 05:25 PM, Saravana Kannan wrote:
> On 08/01/2014 12:54 PM, Stephen Boyd wrote:
>> On 08/01/14 12:43, Prarit Bhargava wrote:
>>>
>>> On 08/01/2014 03:36 PM, Stephen Boyd wrote:
>>>> On 08/01/14 12:15, Prarit Bhargava wrote:
>>>>> On 08/01/2014 01:18 PM, Stephen Boyd wrote:
>>>>>> On 08/01/14 03:27, Prarit Bhargava wrote:
>>>>>>> Can you send me the test and the trace of the deadlock?  I'm not creating
>>>>>>> it with:
>>>>>>>
>>>>>> This was with conservative as the default, and switching to ondemand
>>>>>>
>>>>>> # cd /sys/devices/system/cpu/cpu2/cpufreq
>>>>>> # ls
>>>>>> affected_cpus                  scaling_available_governors
>>>>>> conservative                   scaling_cur_freq
>>>>>> cpuinfo_cur_freq               scaling_driver
>>>>>> cpuinfo_max_freq               scaling_governor
>>>>>> cpuinfo_min_freq               scaling_max_freq
>>>>>> cpuinfo_transition_latency     scaling_min_freq
>>>>>> related_cpus                   scaling_setspeed
>>>>>> scaling_available_frequencies  stats
>>>>>> # cat conservative/down_threshold
>>>>>> 20
>>>>>> # echo ondemand > scaling_governor
>>>>> Thanks Stephen,
>>>>>
>>>>> There's obviously a difference in our .configs.  I have a global conservative
>>>>> directory, ie) /sys/devices/system/cpu/cpufreq/conservative instead of a
>>>>> per-cpu
>>>>> governor file.
>>>>>
>>>>> ie) what are your .config options for CPUFREQ?
>>>>>
>>>>> Mine are:
>>>>>
>>>>> #
>>>>> # CPU Frequency scaling
>>>>> #
>>>>> CONFIG_CPU_FREQ=y
>>>>> CONFIG_CPU_FREQ_GOV_COMMON=y
>>>>> CONFIG_CPU_FREQ_STAT=m
>>>>> CONFIG_CPU_FREQ_STAT_DETAILS=y
>>>>> # CONFIG_CPU_FREQ_DEFAULT_GOV_PERFORMANCE is not set
>>>>> # CONFIG_CPU_FREQ_DEFAULT_GOV_USERSPACE is not set
>>>>> # CONFIG_CPU_FREQ_DEFAULT_GOV_ONDEMAND is not set
>>>>> CONFIG_CPU_FREQ_DEFAULT_GOV_CONSERVATIVE=y
>>>>> CONFIG_CPU_FREQ_GOV_PERFORMANCE=y
>>>>> CONFIG_CPU_FREQ_GOV_POWERSAVE=y
>>>>> CONFIG_CPU_FREQ_GOV_USERSPACE=y
>>>>> CONFIG_CPU_FREQ_GOV_ONDEMAND=y
>>>>> CONFIG_CPU_FREQ_GOV_CONSERVATIVE=y
>>>>>
>>>>> Is there some other config option I have to set?
>>>> I have the same options. The difference is that my driver has a governor
>>>> per policy. That's set with the CPUFREQ_HAVE_GOVERNOR_PER_POLICY flag.
>>>> If I remove that flag I can't trigger the lockdep splat anymore with
>>>> this sequence and your patch.
>>> I see -- so you're seeing this on arm then?  If so, let me know so I can reserve
>>> one to work on :)
>>>
>>
>> I only have ARM to test on. You can set this flag in your cpufreq driver
>> if you have independently scaling CPU frequencies, so it isn't
>> necessarily an ARM thing.
>>
> 
> Sorry, forgot to mention this in the earlier email. But yeah, this is a totally
> SW feature. You should be able to enable this flag in your driver.

Thanks Saravana, I'll get back to this today.  I'm also going to try to track
down an ARM server as well.

I'll get back to everyone in a bit.

P.

> 
> -Saravana
> 

Re: [PATCH] cpufreq, store_scaling_governor requires policy->rwsem to be held for duration of changing governors [v2]

[Viresh Kumar]

Sorry for the delay guys, was away :(

Adding Robert as well, he reported something similar so better discuss here.

On 1 August 2014 22:48, Stephen Boyd <sboyd@codeaurora.org> wrote:
> This was with conservative as the default, and switching to ondemand
>
> # cd /sys/devices/system/cpu/cpu2/cpufreq
> # ls
> affected_cpus                  scaling_available_governors
> conservative                   scaling_cur_freq
> cpuinfo_cur_freq               scaling_driver
> cpuinfo_max_freq               scaling_governor
> cpuinfo_min_freq               scaling_max_freq
> cpuinfo_transition_latency     scaling_min_freq
> related_cpus                   scaling_setspeed
> scaling_available_frequencies  stats
> # cat conservative/down_threshold
> 20
> # echo ondemand > scaling_governor
>
>  ======================================================
>  [ INFO: possible circular locking dependency detected ]
>  3.16.0-rc3-00039-ge1e38f124d87 #47 Not tainted
>  -------------------------------------------------------
>  sh/75 is trying to acquire lock:
>   (s_active#9){++++..}, at: [<c0358a94>] kernfs_remove_by_name_ns+0x3c/0x84
>
>  but task is already holding lock:
>   (&policy->rwsem){+++++.}, at: [<c05ab1f0>] store+0x68/0xb8
>
>  which lock already depends on the new lock.
>
>
>  the existing dependency chain (in reverse order) is:
>
> -> #1 (&policy->rwsem){+++++.}:
>         [<c0359234>] kernfs_fop_open+0x138/0x298
>         [<c02fa3f4>] do_dentry_open.isra.12+0x1b0/0x2f0
>         [<c02fa604>] finish_open+0x20/0x38
>         [<c0308d34>] do_last.isra.37+0x5ac/0xb68
>         [<c03093a4>] path_openat+0xb4/0x5d8
>         [<c0309bcc>] do_filp_open+0x2c/0x80
>         [<c02fb558>] do_sys_open+0x10c/0x1c8
>         [<c020f0a0>] ret_fast_syscall+0x0/0x48
>
> -> #0 (s_active#9){++++..}:
>         [<c0357d18>] __kernfs_remove+0x250/0x300
>         [<c0358a94>] kernfs_remove_by_name_ns+0x3c/0x84
>         [<c035aa78>] remove_files+0x34/0x78
>         [<c035aee0>] sysfs_remove_group+0x40/0x98
>         [<c05b0560>] cpufreq_governor_dbs+0x4c0/0x6ec
>         [<c05abebc>] __cpufreq_governor+0x118/0x200
>         [<c05ac0fc>] cpufreq_set_policy+0x158/0x2ac
>         [<c05ad5e4>] store_scaling_governor+0x6c/0x94
>         [<c05ab210>] store+0x88/0xb8
>         [<c035a00c>] sysfs_kf_write+0x4c/0x50
>         [<c03594d4>] kernfs_fop_write+0xc0/0x180
>         [<c02fc5c8>] vfs_write+0xa0/0x1a8
>         [<c02fc9d4>] SyS_write+0x40/0x8c
>         [<c020f0a0>] ret_fast_syscall+0x0/0x48
>
>  other info that might help us debug this:
>
>   Possible unsafe locking scenario:
>
>         CPU0                    CPU1
>         ----                    ----
>    lock(&policy->rwsem);
>                                 lock(s_active#9);
>                                 lock(&policy->rwsem);
>    lock(s_active#9);
>
>   *** DEADLOCK ***
>
>  6 locks held by sh/75:
>   #0:  (sb_writers#4){.+.+..}, at: [<c02fc6a8>] vfs_write+0x180/0x1a8
>   #1:  (&of->mutex){+.+...}, at: [<c0359498>] kernfs_fop_write+0x84/0x180
>   #2:  (s_active#10){.+.+..}, at: [<c03594a0>] kernfs_fop_write+0x8c/0x180
>   #3:  (cpu_hotplug.lock){++++++}, at: [<c0221ef8>] get_online_cpus+0x38/0x9c
>   #4:  (cpufreq_rwsem){.+.+.+}, at: [<c05ab1d8>] store+0x50/0xb8
>   #5:  (&policy->rwsem){+++++.}, at: [<c05ab1f0>] store+0x68/0xb8
>
>  stack backtrace:
>  CPU: 0 PID: 75 Comm: sh Not tainted 3.16.0-rc3-00039-ge1e38f124d87 #47
>  [<c0214de8>] (unwind_backtrace) from [<c02123f8>] (show_stack+0x10/0x14)
>  [<c02123f8>] (show_stack) from [<c0709e5c>] (dump_stack+0x70/0xbc)
>  [<c0709e5c>] (dump_stack) from [<c070722c>] (print_circular_bug+0x280/0x2d4)
>  [<c070722c>] (print_circular_bug) from [<c02629cc>] (__lock_acquire+0x18d0/0x1abc)
>  [<c02629cc>] (__lock_acquire) from [<c026310c>] (lock_acquire+0x9c/0x138)
>  [<c026310c>] (lock_acquire) from [<c0357d18>] (__kernfs_remove+0x250/0x300)
>  [<c0357d18>] (__kernfs_remove) from [<c0358a94>] (kernfs_remove_by_name_ns+0x3c/0x84)
>  [<c0358a94>] (kernfs_remove_by_name_ns) from [<c035aa78>] (remove_files+0x34/0x78)
>  [<c035aa78>] (remove_files) from [<c035aee0>] (sysfs_remove_group+0x40/0x98)
>  [<c035aee0>] (sysfs_remove_group) from [<c05b0560>] (cpufreq_governor_dbs+0x4c0/0x6ec)
>  [<c05b0560>] (cpufreq_governor_dbs) from [<c05abebc>] (__cpufreq_governor+0x118/0x200)
>  [<c05abebc>] (__cpufreq_governor) from [<c05ac0fc>] (cpufreq_set_policy+0x158/0x2ac)
>  [<c05ac0fc>] (cpufreq_set_policy) from [<c05ad5e4>] (store_scaling_governor+0x6c/0x94)
>  [<c05ad5e4>] (store_scaling_governor) from [<c05ab210>] (store+0x88/0xb8)
>  [<c05ab210>] (store) from [<c035a00c>] (sysfs_kf_write+0x4c/0x50)
>  [<c035a00c>] (sysfs_kf_write) from [<c03594d4>] (kernfs_fop_write+0xc0/0x180)
>  [<c03594d4>] (kernfs_fop_write) from [<c02fc5c8>] (vfs_write+0xa0/0x1a8)
>  [<c02fc5c8>] (vfs_write) from [<c02fc9d4>] (SyS_write+0x40/0x8c)
>  [<c02fc9d4>] (SyS_write) from [<c020f0a0>] (ret_fast_syscall+0x0/0x48)

Thanks for coming to my rescue Stephen :), I was quite sure I got this
with ondemand
as well..

I will be looking very closely at the code now to see what's going wrong.
And btw, does anybody here has the exact understanding of why this
lockdep does happen? I mean what was the real problem for which we
just dropped the rwsems.. I understood that earlier but couldn't get that
again :)

Thanks all for you work on getting this fixed.

--
viresh

Re: [PATCH] cpufreq, store_scaling_governor requires policy->rwsem to be held for duration of changing governors [v2]

[Prarit Bhargava]

On 08/04/2014 06:36 AM, Viresh Kumar wrote:
> Sorry for the delay guys, was away :(
> 
>>
>>   Possible unsafe locking scenario:
>>
>>         CPU0                    CPU1
>>         ----                    ----
>>    lock(&policy->rwsem);
>>                                 lock(s_active#9);
>>                                 lock(&policy->rwsem);
>>    lock(s_active#9);
>>
> 
> Thanks for coming to my rescue Stephen :), I was quite sure I got this
> with ondemand
> as well..

Yeah, I'm confused by it a bit because I wasn't able to reproduce it.  But I've
got some pretty clear instructions on how to do it.

> 
> I will be looking very closely at the code now to see what's going wrong.
> And btw, does anybody here has the exact understanding of why this
> lockdep does happen? I mean what was the real problem for which we

The issue is the collision between the setup & teardown of the policy's governor
sysfs files.

On creation the kernel does:

down_write(&policy->rwsem)
mutex_lock(kernfs_mutex) <- note this is similar to the "old" sysfs_mutex.

The opposite happens on a governor switch, specifically the existing governor's
exit, and then we get a lockdep warning.

I tried to reproduce with the instructions but was unable to ... ut that was on
Friday ;) and I'm going to try again this morning.  I've also ping'd some of the
engineers here in the office who are working on ARM to get access to a system to
do further analysis and testing.

I'll ping back later in the day with some results.

Sorry I don't have a better answer or solution yet, Viresh.

P.

Re: [PATCH] cpufreq, store_scaling_governor requires policy->rwsem to be held for duration of changing governors [v2]

[Viresh Kumar]

On 4 August 2014 17:55, Prarit Bhargava <prarit@redhat.com> wrote:
> The issue is the collision between the setup & teardown of the policy's governor
> sysfs files.
>
> On creation the kernel does:
>
> down_write(&policy->rwsem)
> mutex_lock(kernfs_mutex) <- note this is similar to the "old" sysfs_mutex.
>
> The opposite happens on a governor switch, specifically the existing governor's
> exit, and then we get a lockdep warning.

Okay, probably a bit more clarity is what I was looking for. Suppose we try
to change governor, now tell me what will happen.

> I tried to reproduce with the instructions but was unable to ... ut that was on
> Friday ;) and I'm going to try again this morning.  I've also ping'd some of the
> engineers here in the office who are working on ARM to get access to a system to
> do further analysis and testing.

You DON'T need an ARM for that, just try that on any x86 machine which has
multiple groups of CPUs sharing clock line. Or in other terms there are multiple
policy structures there..

You just need to enable the flag we were discussing about, it just decided the
location where governor's directory will get created. Nothing else.

Re: [PATCH] cpufreq, store_scaling_governor requires policy->rwsem to be held for duration of changing governors [v2]

[Prarit Bhargava]

On 08/04/2014 09:38 AM, Viresh Kumar wrote:
> On 4 August 2014 17:55, Prarit Bhargava <prarit@redhat.com> wrote:
>> The issue is the collision between the setup & teardown of the policy's governor
>> sysfs files.
>>
>> On creation the kernel does:
>>
>> down_write(&policy->rwsem)
>> mutex_lock(kernfs_mutex) <- note this is similar to the "old" sysfs_mutex.
>>
>> The opposite happens on a governor switch, specifically the existing governor's
>> exit, and then we get a lockdep warning.
> 
> Okay, probably a bit more clarity is what I was looking for. Suppose we try
> to change governor, now tell me what will happen.
> 
>> I tried to reproduce with the instructions but was unable to ... ut that was on
>> Friday ;) and I'm going to try again this morning.  I've also ping'd some of the
>> engineers here in the office who are working on ARM to get access to a system to
>> do further analysis and testing.
> 
> You DON'T need an ARM for that, just try that on any x86 machine which has
> multiple groups of CPUs sharing clock line. Or in other terms there are multiple
> policy structures there..

I do ... I really think I do.  Because this is all working on x86 AFAICT.

> 
> You just need to enable the flag we were discussing about, it just decided the
> location where governor's directory will get created. Nothing else.
> 

That doesn't appear to be correct.  I'm testing with the patch that removes the
locking workaround and:

diff --git a/drivers/cpufreq/acpi-cpufreq.c b/drivers/cpufreq/acpi-cpufreq.c
index b0c18ed..d86b421 100644
--- a/drivers/cpufreq/acpi-cpufreq.c
+++ b/drivers/cpufreq/acpi-cpufreq.c
@@ -884,6 +884,8 @@ static struct freq_attr *acpi_cpufreq_attr[] = {
 };

 static struct cpufreq_driver acpi_cpufreq_driver = {
+       .name           = "acpi_cpufreq",
+       .flags                  = CPUFREQ_HAVE_GOVERNOR_PER_POLICY,
        .verify         = cpufreq_generic_frequency_table_verify,
        .target_index   = acpi_cpufreq_target,
        .bios_limit     = acpi_processor_get_bios_limit,

as well as few printk statement sprinkled in the code.  I'm doing the following
and on *15* different x86 systems I do not see a problem:

My cpufreq related config is

#
# CPU Frequency scaling
#
CONFIG_CPU_FREQ=y
CONFIG_CPU_FREQ_GOV_COMMON=y
CONFIG_CPU_FREQ_STAT=m
CONFIG_CPU_FREQ_STAT_DETAILS=y
# CONFIG_CPU_FREQ_DEFAULT_GOV_PERFORMANCE is not set
# CONFIG_CPU_FREQ_DEFAULT_GOV_USERSPACE is not set
# CONFIG_CPU_FREQ_DEFAULT_GOV_ONDEMAND is not set
CONFIG_CPU_FREQ_DEFAULT_GOV_CONSERVATIVE=y
CONFIG_CPU_FREQ_GOV_PERFORMANCE=y
CONFIG_CPU_FREQ_GOV_POWERSAVE=y
CONFIG_CPU_FREQ_GOV_USERSPACE=y
CONFIG_CPU_FREQ_GOV_ONDEMAND=y
CONFIG_CPU_FREQ_GOV_CONSERVATIVE=y

I am doing (from boot)

[root@intel-canoepass-05 cpufreq]# cd /sys/devices/system/cpu/cpu2/cpufreq/
[root@intel-canoepass-05 cpufreq]# ls
affected_cpus     cpuinfo_transition_latency     scaling_driver
bios_limit        freqdomain_cpus                scaling_governor
conservative      related_cpus                   scaling_max_freq
cpuinfo_cur_freq  scaling_available_frequencies  scaling_min_freq
cpuinfo_max_freq  scaling_available_governors    scaling_setspeed
cpuinfo_min_freq  scaling_cur_freq
[root@intel-canoepass-05 cpufreq]# cat conservative/
down_threshold        sampling_down_factor  up_threshold
freq_step             sampling_rate
ignore_nice_load      sampling_rate_min
[root@intel-canoepass-05 cpufreq]# cat conservative/down_threshold
20
[root@intel-canoepass-05 cpufreq]# echo ondemand > scaling_governor
[root@intel-canoepass-05 cpufreq]# cat ondemand/up_threshold
95
[root@intel-canoepass-05 cpufreq]# echo conservative > scaling_governor
[root@intel-canoepass-05 cpufreq]#

without any issue.  My dmesg (with the printk's) shows


[ 55.331058] cpufreq_set_policy: stopping governor conservative
[ 55.337652] cpufreq_governor_dbs: removing sysfs files for governor conservative
[ 55.346028] cpufreq_set_policy: starting governor ondemand
[ 55.352167] cpufreq_governor_dbs: creating sysfs files for governor ondemand
[ 76.818989] cpufreq_set_policy: stopping governor ondemand
[ 76.825202] cpufreq_governor_dbs: removing sysfs files for governor ondemand
[ 76.833131] cpufreq_set_policy: starting governor conservative
[ 76.839667] cpufreq_governor_dbs: creating sysfs files for governor conservative

There is an already reported LOCKDEP warning in the xfs code that fires at login
so I know LOCKDEP is functional.

Stephen's report as well as the lockup report implies that I should open a file,

-> #1 (&policy->rwsem){+++++.}:
        [<c0359234>] kernfs_fop_open+0x138/0x298
        [<c02fa3f4>] do_dentry_open.isra.12+0x1b0/0x2f0
        [<c02fa604>] finish_open+0x20/0x38
        [<c0308d34>] do_last.isra.37+0x5ac/0xb68
        [<c03093a4>] path_openat+0xb4/0x5d8
        [<c0309bcc>] do_filp_open+0x2c/0x80
        [<c02fb558>] do_sys_open+0x10c/0x1c8
        [<c020f0a0>] ret_fast_syscall+0x0/0x48

and then switch the governor ...

-> #0 (s_active#9){++++..}:
        [<c0357d18>] __kernfs_remove+0x250/0x300
        [<c0358a94>] kernfs_remove_by_name_ns+0x3c/0x84
        [<c035aa78>] remove_files+0x34/0x78
        [<c035aee0>] sysfs_remove_group+0x40/0x98
        [<c05b0560>] cpufreq_governor_dbs+0x4c0/0x6ec
        [<c05abebc>] __cpufreq_governor+0x118/0x200
        [<c05ac0fc>] cpufreq_set_policy+0x158/0x2ac
        [<c05ad5e4>] store_scaling_governor+0x6c/0x94
        [<c05ab210>] store+0x88/0xb8
        [<c035a00c>] sysfs_kf_write+0x4c/0x50
        [<c03594d4>] kernfs_fop_write+0xc0/0x180
        [<c02fc5c8>] vfs_write+0xa0/0x1a8
        [<c02fc9d4>] SyS_write+0x40/0x8c
        [<c020f0a0>] ret_fast_syscall+0x0/0x48

... right?

P.

Re: [PATCH] cpufreq, store_scaling_governor requires policy->rwsem to be held for duration of changing governors [v2]

[Prarit Bhargava]

On 08/04/2014 10:00 AM, Prarit Bhargava wrote:

> There is an already reported LOCKDEP warning in the xfs code that fires at login
> so I know LOCKDEP is functional.
> 

It turns out that the above was the problem.  I didn't realize that LOCKDEP will
only report a single warning :(

[Aside: Is there a way to "re-arm" LOCKDEP?]

Reproduced on x86 and taking a further look:


[root@intel-canoepass-05 ~]# cd /sys/devices/system/cpu/cpu2/cpufreq/
[root@intel-canoepass-05 cpufreq]# cat conservative/*
20
5
0
1
20000
20000
80
[root@intel-canoepass-05 cpufreq]# echo ondemand > scaling_governor
[   75.163583] cpufreq_set_policy: stopping governor conservative
[   75.170348] cpufreq_governor_dbs: removing sysfs files for governor conservative
[   75.178698]
[   75.180364] ======================================================
[   75.187271] [ INFO: possible circular locking dependency detected ]
[   75.194278] 3.16.0-rc7+ #22 Not tainted
[   75.198561] -------------------------------------------------------
[   75.205564] bash/3131 is trying to acquire lock:
[   75.210726]  (s_active#219){++++.+}, at: [<ffffffff8126ce55>]
kernfs_remove_by_name_ns+0x45/0xa0
[   75.220623]
[   75.220623] but task is already holding lock:
[   75.227141]  (&policy->rwsem){+++++.}, at: [<ffffffff814ecab0>] store+0x60/0xc0
[   75.235370]
[   75.235370] which lock already depends on the new lock.
[   75.235370]
[   75.244509]
[   75.244509] the existing dependency chain (in reverse order) is:
[   75.252874]
-> #1 (&policy->rwsem){+++++.}:
[   75.257785]        [<ffffffff810cc892>] lock_acquire+0xa2/0x120
[   75.264429]        [<ffffffff8166ed40>] mutex_lock_nested+0x60/0x4d0
[   75.271558]        [<ffffffff8126de2e>] kernfs_fop_open+0x16e/0x340
[   75.278579]        [<ffffffff811efaef>] do_dentry_open+0x1ff/0x350
[   75.285510]        [<ffffffff811efe11>] finish_open+0x31/0x40
[   75.291948]        [<ffffffff81202e26>] do_last+0xc16/0x1350
[   75.298298]        [<ffffffff8120362d>] path_openat+0xcd/0x640
[   75.304821]        [<ffffffff8120442d>] do_filp_open+0x4d/0xb0
[   75.311353]        [<ffffffff811f18a7>] do_sys_open+0x137/0x240
[   75.317989]        [<ffffffff811f19ce>] SyS_open+0x1e/0x20
[   75.324122]        [<ffffffff81672ca9>] system_call_fastpath+0x16/0x1b
[   75.331437]
-> #0 (s_active#219){++++.+}:
[   75.336174]        [<ffffffff810cba76>] __lock_acquire+0x14e6/0x1b40
[   75.343293]        [<ffffffff810cc892>] lock_acquire+0xa2/0x120
[   75.349923]        [<ffffffff8126bc48>] __kernfs_remove+0x248/0x330
[   75.356950]        [<ffffffff8126ce55>] kernfs_remove_by_name_ns+0x45/0xa0
[   75.364638]        [<ffffffff8126f739>] remove_files.isra.1+0x39/0x70
[   75.371850]        [<ffffffff8126fa44>] sysfs_remove_group+0x44/0xa0
[   75.378965]        [<ffffffff814f280a>] cpufreq_governor_dbs+0x6da/0x770
[   75.386478]        [<ffffffff814f1277>] cs_cpufreq_governor_dbs+0x17/0x20
[   75.394088]        [<ffffffff814ed1bd>] __cpufreq_governor+0x11d/0x230
[   75.401395]        [<ffffffff814ed435>] cpufreq_set_policy+0x165/0x2c0
[   75.408706]        [<ffffffff814ee16d>] store_scaling_governor+0xad/0xf0
[   75.416211]        [<ffffffff814ecacc>] store+0x7c/0xc0
[   75.422062]        [<ffffffff8126e7e4>] sysfs_kf_write+0x44/0x60
[   75.428790]        [<ffffffff8126e0e7>] kernfs_fop_write+0xe7/0x170
[   75.435806]        [<ffffffff811f2857>] vfs_write+0xb7/0x1f0
[   75.442146]        [<ffffffff811f3488>] SyS_write+0x58/0xd0
[   75.448390]        [<ffffffff81672ca9>] system_call_fastpath+0x16/0x1b
[   75.455706]
[   75.455706] other info that might help us debug this:
[   75.455706]
[   75.464649]  Possible unsafe locking scenario:
[   75.464649]
[   75.471265]        CPU0                    CPU1
[   75.476327]        ----                    ----
[   75.481385]   lock(&policy->rwsem);
[   75.485307]                                lock(s_active#219);
[   75.491857]                                lock(&policy->rwsem);
[   75.498592]   lock(s_active#219);
[   75.502331]
[   75.502331]  *** DEADLOCK ***
[   75.502331]
[   75.508948] 6 locks held by bash/3131:
[   75.513136]  #0:  (sb_writers#3){.+.+.+}, at: [<ffffffff811f2953>]
vfs_write+0x1b3/0x1f0
[   75.522259]  #1:  (&of->mutex){+.+.+.}, at: [<ffffffff8126e0bb>]
kernfs_fop_write+0xbb/0x170
[   75.531751]  #2:  (s_active#221){.+.+.+}, at: [<ffffffff8126e0c3>]
kernfs_fop_write+0xc3/0x170
[   75.541448]  #3:  (cpu_hotplug.lock){++++++}, at: [<ffffffff810752d4>]
get_online_cpus+0x24/0x70
[   75.551339]  #4:  (cpufreq_rwsem){.+.+.+}, at: [<ffffffff814eca96>]
store+0x46/0xc0
[   75.559957]  #5:  (&policy->rwsem){+++++.}, at: [<ffffffff814ecab0>]
store+0x60/0xc0
[   75.568671]
[   75.568671] stack backtrace:
[   75.573544] CPU: 14 PID: 3131 Comm: bash Not tainted 3.16.0-rc7+ #22
[   75.580647] Hardware name: Intel Corporation S2600CP/S2600CP, BIOS
RMLSDP.86I.R2.21.D636.1301031557 01/03/2013
[   75.591824]  0000000000000000 000000000264b511 ffff881001837868 ffffffff816696a7
[   75.600137]  ffffffff8254dd40 ffff8810018378a8 ffffffff81662be3 ffff881001837900
[   75.608451]  ffff88100a6fba68 0000000000000005 ffff88100a6fba68 ffff88100a6fadc0
[   75.616764] Call Trace:
[   75.619505]  [<ffffffff816696a7>] dump_stack+0x45/0x56
[   75.625253]  [<ffffffff81662be3>] print_circular_bug+0x1f9/0x207
[   75.631966]  [<ffffffff810cba76>] __lock_acquire+0x14e6/0x1b40
[   75.638478]  [<ffffffff810cb284>] ? __lock_acquire+0xcf4/0x1b40
[   75.645095]  [<ffffffff810c9f7a>] ? mark_held_locks+0x6a/0x90
[   75.651522]  [<ffffffff810cc892>] lock_acquire+0xa2/0x120
[   75.657557]  [<ffffffff8126ce55>] ? kernfs_remove_by_name_ns+0x45/0xa0
[   75.664854]  [<ffffffff8126bc48>] __kernfs_remove+0x248/0x330
[   75.671279]  [<ffffffff8126ce55>] ? kernfs_remove_by_name_ns+0x45/0xa0
[   75.678577]  [<ffffffff8126b117>] ? kernfs_name_hash+0x17/0xd0
[   75.685097]  [<ffffffff8126bdec>] ? kernfs_find_ns+0xbc/0x160
[   75.691523]  [<ffffffff8126ce55>] kernfs_remove_by_name_ns+0x45/0xa0
[   75.698627]  [<ffffffff8126f739>] remove_files.isra.1+0x39/0x70
[   75.705242]  [<ffffffff8126fa44>] sysfs_remove_group+0x44/0xa0
[   75.711753]  [<ffffffff814f280a>] cpufreq_governor_dbs+0x6da/0x770
[   75.718661]  [<ffffffff810ca0a5>] ? trace_hardirqs_on_caller+0x105/0x1d0
[   75.726152]  [<ffffffff810ca17d>] ? trace_hardirqs_on+0xd/0x10
[   75.732672]  [<ffffffff814f1277>] cs_cpufreq_governor_dbs+0x17/0x20
[   75.739675]  [<ffffffff814ed1bd>] __cpufreq_governor+0x11d/0x230
[   75.746387]  [<ffffffff814ed435>] cpufreq_set_policy+0x165/0x2c0
[   75.753099]  [<ffffffff814ee16d>] store_scaling_governor+0xad/0xf0
[   75.760007]  [<ffffffff814ed780>] ? cpufreq_update_policy+0x1f0/0x1f0
[   75.767207]  [<ffffffff814ecacc>] store+0x7c/0xc0
[   75.772464]  [<ffffffff8126e7e4>] sysfs_kf_write+0x44/0x60
[   75.778586]  [<ffffffff8126e0e7>] kernfs_fop_write+0xe7/0x170
[   75.785000]  [<ffffffff811f2857>] vfs_write+0xb7/0x1f0
[   75.790741]  [<ffffffff811f3488>] SyS_write+0x58/0xd0
[   75.796386]  [<ffffffff81672ca9>] system_call_fastpath+0x16/0x1b
[   75.803172] cpufreq_set_policy: starting governor ondemand
[   75.809341] cpufreq_governor_dbs: creating sysfs files for governor ondemand

P.

Re: [PATCH] cpufreq, store_scaling_governor requires policy->rwsem to be held for duration of changing governors [v2]

[Saravana Kannan]

On 08/04/2014 06:38 AM, Viresh Kumar wrote:
> On 4 August 2014 17:55, Prarit Bhargava <prarit@redhat.com> wrote:
>> The issue is the collision between the setup & teardown of the policy's governor
>> sysfs files.
>>
>> On creation the kernel does:
>>
>> down_write(&policy->rwsem)
>> mutex_lock(kernfs_mutex) <- note this is similar to the "old" sysfs_mutex.
>>
>> The opposite happens on a governor switch, specifically the existing governor's
>> exit, and then we get a lockdep warning.
>
> Okay, probably a bit more clarity is what I was looking for. Suppose we try
> to change governor, now tell me what will happen.
>
Viresh, I have a good understanding of what's going on. I was planning 
to fix this after the rest of my patches are done. Didn't want to keep 
too many outstanding unaccepted patches.

The problem is when between one thread trying to cat a governor's file 
(say, sampling_rate) vs the governor getting a POLICY_EXIT.

In the read thread, the sysfs lock is grabbed first and then the policy 
lock is grabbed next. When the governor gets the POLICY_EXIT, we call it 
with the policy lock held and the governor tries to do sysfs remove 
which tries to grab the sysfs lock. Classic deadlock.

Could you please look at my policy free/remove patches? If you can do 
that, I can work on a fix for this. It might also be simpler to fix 
after my patch series (not sure, might be).

Thanks,
Saravana
-- 
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
hosted by The Linux Foundation

Re: [PATCH] cpufreq, store_scaling_governor requires policy->rwsem to be held for duration of changing governors [v2]

[Viresh Kumar]

On 5 August 2014 01:46, Saravana Kannan <skannan@codeaurora.org> wrote:
> The problem is when between one thread trying to cat a governor's file (say,
> sampling_rate) vs the governor getting a POLICY_EXIT.

I don't see two threads racing against each other here. Simply changing
the governor from conservative->ondemand creates this.

Or is it that the kernel is detecting two different orders of taking lock?

But during governor change, isn't the sysfs lock taken first as we are
storing a value to "scaling_governor"? So, isn't this a sysfs lock first
in all cases?

In short, I am still unclear about the *exact* problem here.

> Could you please look at my policy free/remove patches? If you can do that,
> I can work on a fix for this. It might also be simpler to fix after my patch
> series (not sure, might be).

I had an overall look of those on the day you posted them, but haven't
commented yet as was going away..

There is no way those can land in 3.17-rc1 atleast and so we still have
some time to get them pushed..

Anyway, they are my number two priority and the number one is this bug,
which we have to fix in stable kernels as well. So, a dependency on your
series wouldn't work..

Let me see if I can get some solution to this problem first.

Re: [PATCH] cpufreq, store_scaling_governor requires policy->rwsem to be held for duration of changing governors [v2]

[skannan]

Viresh Kumar wrote:
> On 5 August 2014 01:46, Saravana Kannan <skannan@codeaurora.org> wrote:
>> The problem is when between one thread trying to cat a governor's file
>> (say,
>> sampling_rate) vs the governor getting a POLICY_EXIT.
>
> I don't see two threads racing against each other here. Simply changing
> the governor from conservative->ondemand creates this.
>
> Or is it that the kernel is detecting two different orders of taking lock?
>
> But during governor change, isn't the sysfs lock taken first as we are
> storing a value to "scaling_governor"? So, isn't this a sysfs lock first
> in all cases?
>
> In short, I am still unclear about the *exact* problem here.
>
>> Could you please look at my policy free/remove patches? If you can do
>> that,
>> I can work on a fix for this. It might also be simpler to fix after my
>> patch
>> series (not sure, might be).
>
> I had an overall look of those on the day you posted them, but haven't
> commented yet as was going away..
>
> There is no way those can land in 3.17-rc1 atleast and so we still have
> some time to get them pushed..
>
> Anyway, they are my number two priority and the number one is this bug,
> which we have to fix in stable kernels as well. So, a dependency on your
> series wouldn't work..

Sigh... ok. I too will try to fix this one. I already have something in
mind for this.

Looks like Srivatsa has gone off the grid too. I'm hoping at least one of
you can do a review of my series. Come on guys, not everyone has to work
on the same patch/issue. :-(

-Saravana

-- 
The Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
hosted by The Linux Foundation

Re: [PATCH] cpufreq, store_scaling_governor requires policy->rwsem to be held for duration of changing governors [v2]

[Viresh Kumar]

On 5 August 2014 11:59,  <skannan@codeaurora.org> wrote:
> Looks like Srivatsa has gone off the grid too. I'm hoping at least one of
> you can do a review of my series. Come on guys, not everyone has to work
> on the same patch/issue. :-(

Srivatsa is currently going through some personal stuff. He is joining MIT
for his Masters/Phd and so would be moving to US soon :)

Re: [PATCH] cpufreq, store_scaling_governor requires policy->rwsem to be held for duration of changing governors [v2]

[Viresh Kumar]

On 2 August 2014 01:06, Stephen Boyd <sboyd@codeaurora.org> wrote:
> I have the same options. The difference is that my driver has a governor
> per policy. That's set with the CPUFREQ_HAVE_GOVERNOR_PER_POLICY flag.

You may call me stupid but I got a bit confused after looking into the code
again. Why does the crash dump depends on this flag?

We *always* remove the governor specific directory while switching governors
(Ofcourse only if its updated for All CPUs). And so on a dual core platform,
where both CPU 0 & 1 share a clock line, switching of governors should result
in this crash dump?

I may know the answer to the stupid question I had, but not sure why that is a
problem. The only (and quite significant) difference that this flag makes
is the location of governor-specific directory:
- w/o this flag: /sys/devices/system/cpu/cpufreq/<here>
- w/ this flag: /sys/devices/system/cpu/cpu*/cpufreq/<here>

So, is there some issue with the sysfs lock for <cpu*/cpufreq/> node as while
switching governor we change  <cpu*/cpufreq/scaling_governor> at the same
location?

--
viresh

--
viresh

Re: [PATCH] cpufreq, store_scaling_governor requires policy->rwsem to be held for duration of changing governors [v2]

[Prarit Bhargava]

On 08/05/2014 03:46 AM, Viresh Kumar wrote:
> On 2 August 2014 01:06, Stephen Boyd <sboyd@codeaurora.org> wrote:
>> I have the same options. The difference is that my driver has a governor
>> per policy. That's set with the CPUFREQ_HAVE_GOVERNOR_PER_POLICY flag.
> 
> You may call me stupid but I got a bit confused after looking into the code
> again. Why does the crash dump depends on this flag?

Nope, not a stupid question.  After reproducing (finally!) yesterday I've been
wondering the same thing.

> 
> We *always* remove the governor specific directory while switching governors
> (Ofcourse only if its updated for All CPUs). And so on a dual core platform,
> where both CPU 0 & 1 share a clock line, switching of governors should result
> in this crash dump?

I've been looking into *exactly* this.  On any platform where
cpu_weight(affected_cpus) == 1 for a particular cpu this lockdep trace should
happen.


> 
> I may know the answer to the stupid question I had, but not sure why that is a
> problem. The only (and quite significant) difference that this flag makes
> is the location of governor-specific directory:
> - w/o this flag: /sys/devices/system/cpu/cpufreq/<here>
> - w/ this flag: /sys/devices/system/cpu/cpu*/cpufreq/<here>
> 

cpufreq_global_kobject vs the policy's kobject.

> So, is there some issue with the sysfs lock for <cpu*/cpufreq/> node as while
> switching governor we change  <cpu*/cpufreq/scaling_governor> at the same
> location?

That's what I'm wondering too.  I'm going to instrument the code to find out
this morning.  I'm wondering if this comes down to a lockdep class issue
(perhaps lockdep puts globally defined locks like cpufreq_global_kobject in a
different class?).

P.

Re: [PATCH] cpufreq, store_scaling_governor requires policy->rwsem to be held for duration of changing governors [v2]

[Viresh Kumar]

On 5 August 2014 16:17, Prarit Bhargava <prarit@redhat.com> wrote:
> Nope, not a stupid question.  After reproducing (finally!) yesterday I've been
> wondering the same thing.

Good to know that :)

> I've been looking into *exactly* this.  On any platform where
> cpu_weight(affected_cpus) == 1 for a particular cpu this lockdep trace should
> happen.

> That's what I'm wondering too.  I'm going to instrument the code to find out
> this morning.  I'm wondering if this comes down to a lockdep class issue
> (perhaps lockdep puts globally defined locks like cpufreq_global_kobject in a
> different class?).

Maybe, I tried this Hack to make this somewhat similar to the other case
on my platform with just two CPUs:

diff --git a/drivers/cpufreq/cpufreq.c b/drivers/cpufreq/cpufreq.c
index 6f02485..6b4abac 100644
--- a/drivers/cpufreq/cpufreq.c
+++ b/drivers/cpufreq/cpufreq.c
@@ -98,7 +98,7 @@ static DEFINE_MUTEX(cpufreq_governor_mutex);

 bool have_governor_per_policy(void)
 {
-       return !!(cpufreq_driver->flags & CPUFREQ_HAVE_GOVERNOR_PER_POLICY);
+       return !(cpufreq_driver->flags & CPUFREQ_HAVE_GOVERNOR_PER_POLICY);
 }
 EXPORT_SYMBOL_GPL(have_governor_per_policy);


This should result in something similar to setting that per-policy-governor
flag (Actually I could have done that too :)), and I couldn't see that crash :(

That needs more investigation now, probably we can get some champ of
sysfs stuff like Tejun/Greg into discussion now..

--
viresh

Re: [PATCH] cpufreq, store_scaling_governor requires policy->rwsem to be held for duration of changing governors [v2]

[Saravana Kannan]

On 08/05/2014 03:53 AM, Viresh Kumar wrote:
> On 5 August 2014 16:17, Prarit Bhargava <prarit@redhat.com> wrote:
>> Nope, not a stupid question.  After reproducing (finally!) yesterday I've been
>> wondering the same thing.
>
> Good to know that :)
>
>> I've been looking into *exactly* this.  On any platform where
>> cpu_weight(affected_cpus) == 1 for a particular cpu this lockdep trace should
>> happen.
>
>> That's what I'm wondering too.  I'm going to instrument the code to find out
>> this morning.  I'm wondering if this comes down to a lockdep class issue
>> (perhaps lockdep puts globally defined locks like cpufreq_global_kobject in a
>> different class?).
>
> Maybe, I tried this Hack to make this somewhat similar to the other case
> on my platform with just two CPUs:
>
> diff --git a/drivers/cpufreq/cpufreq.c b/drivers/cpufreq/cpufreq.c
> index 6f02485..6b4abac 100644
> --- a/drivers/cpufreq/cpufreq.c
> +++ b/drivers/cpufreq/cpufreq.c
> @@ -98,7 +98,7 @@ static DEFINE_MUTEX(cpufreq_governor_mutex);
>
>   bool have_governor_per_policy(void)
>   {
> -       return !!(cpufreq_driver->flags & CPUFREQ_HAVE_GOVERNOR_PER_POLICY);
> +       return !(cpufreq_driver->flags & CPUFREQ_HAVE_GOVERNOR_PER_POLICY);
>   }
>   EXPORT_SYMBOL_GPL(have_governor_per_policy);
>
>
> This should result in something similar to setting that per-policy-governor
> flag (Actually I could have done that too :)), and I couldn't see that crash :(
>
> That needs more investigation now, probably we can get some champ of
> sysfs stuff like Tejun/Greg into discussion now..

Stephen and I looked into this. This is not a sysfs framework 
difference. The reason we don't have this issue when we use global 
tunables is because we add the attribute group to the 
cpufreq_global_kobject and that kobject doesn't have a kobj_type ops 
similar to the per policy kobject. So, read/write to those attributes do 
NOT go through the generic show/store ops that wrap every other cpufreq 
framework attribute read/writes.

So, none of those read/write do any kind of locking. They don't race 
with POLICY_EXIT (because we remove the sysfs group first thing in 
POLICY_EXIT) but might still race with START/STOPs (not sure, haven't 
looked closely yet).

For example, writing to sampling_rate of ondemand governor might cause a 
race in update_sampling_rate(). It could race and happen between a STOP 
and POLICY_EXIT (triggered by hotplug, gov change, etc).

So, this might be a completely separate bug that needs fixing when we 
don't use per policy govs.

-Saravana

-- 
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
hosted by The Linux Foundation

Re: [PATCH] cpufreq, store_scaling_governor requires policy->rwsem to be held for duration of changing governors [v2]

[Prarit Bhargava]

On 08/05/2014 06:06 PM, Saravana Kannan wrote:
> On 08/05/2014 03:53 AM, Viresh Kumar wrote:
>> On 5 August 2014 16:17, Prarit Bhargava <prarit@redhat.com> wrote:
>>> Nope, not a stupid question.  After reproducing (finally!) yesterday I've been
>>> wondering the same thing.
>>
>> Good to know that :)
>>
>>> I've been looking into *exactly* this.  On any platform where
>>> cpu_weight(affected_cpus) == 1 for a particular cpu this lockdep trace should
>>> happen.
>>
>>> That's what I'm wondering too.  I'm going to instrument the code to find out
>>> this morning.  I'm wondering if this comes down to a lockdep class issue
>>> (perhaps lockdep puts globally defined locks like cpufreq_global_kobject in a
>>> different class?).
>>
>> Maybe, I tried this Hack to make this somewhat similar to the other case
>> on my platform with just two CPUs:
>>
>> diff --git a/drivers/cpufreq/cpufreq.c b/drivers/cpufreq/cpufreq.c
>> index 6f02485..6b4abac 100644
>> --- a/drivers/cpufreq/cpufreq.c
>> +++ b/drivers/cpufreq/cpufreq.c
>> @@ -98,7 +98,7 @@ static DEFINE_MUTEX(cpufreq_governor_mutex);
>>
>>   bool have_governor_per_policy(void)
>>   {
>> -       return !!(cpufreq_driver->flags & CPUFREQ_HAVE_GOVERNOR_PER_POLICY);
>> +       return !(cpufreq_driver->flags & CPUFREQ_HAVE_GOVERNOR_PER_POLICY);
>>   }
>>   EXPORT_SYMBOL_GPL(have_governor_per_policy);
>>
>>
>> This should result in something similar to setting that per-policy-governor
>> flag (Actually I could have done that too :)), and I couldn't see that crash :(
>>
>> That needs more investigation now, probably we can get some champ of
>> sysfs stuff like Tejun/Greg into discussion now..
> 
> Stephen and I looked into this. This is not a sysfs framework difference. The
> reason we don't have this issue when we use global tunables is because we add
> the attribute group to the cpufreq_global_kobject and that kobject doesn't have
> a kobj_type ops similar to the per policy kobject. So, read/write to those
> attributes do NOT go through the generic show/store ops that wrap every other
> cpufreq framework attribute read/writes.
> 
> So, none of those read/write do any kind of locking. They don't race with
> POLICY_EXIT (because we remove the sysfs group first thing in POLICY_EXIT) but
> might still race with START/STOPs (not sure, haven't looked closely yet).
> 
> For example, writing to sampling_rate of ondemand governor might cause a race in
> update_sampling_rate(). It could race and happen between a STOP and POLICY_EXIT
> (triggered by hotplug, gov change, etc).
> 
> So, this might be a completely separate bug that needs fixing when we don't use
> per policy govs.

Yeah, the show_one & store_one macros don't have any locking in them :/.

Okay ... at least that isn't the issue.  I spent 1/2 the day trying to figure
out why

diff --git a/drivers/cpufreq/cpufreq.c b/drivers/cpufreq/cpufreq.c
index fa11a7d..6297c76 100644
--- a/drivers/cpufreq/cpufreq.c
+++ b/drivers/cpufreq/cpufreq.c
@@ -745,12 +745,14 @@ static struct attribute *default_attrs[] = {
 #define to_policy(k) container_of(k, struct cpufreq_policy, kobj)
 #define to_attr(a) container_of(a, struct freq_attr, attr)

+/* PRARIT - in the CPUFREQ_HAVE_GOVERNOR_PER_POLICY, this is used */
 static ssize_t show(struct kobject *kobj, struct attribute *attr, char *buf)
 {
        struct cpufreq_policy *policy = to_policy(kobj);
        struct freq_attr *fattr = to_attr(attr);
        ssize_t ret;

+       printk("%s: kobject %p\n", __FUNCTION__, kobj);
        if (!down_read_trylock(&cpufreq_rwsem))
                return -EINVAL;

wasn't printing the kobject line when acpi-cpufreq didn't have the
CPUFREQ_HAVE_GOVERNOR_PER_POLICY flag.  And I agree ... it is a bug.

P.

Re: [PATCH] cpufreq, store_scaling_governor requires policy->rwsem to be held for duration of changing governors [v2]

[Saravana Kannan]

On 08/05/2014 03:20 PM, Prarit Bhargava wrote:
>
>
> On 08/05/2014 06:06 PM, Saravana Kannan wrote:
>> On 08/05/2014 03:53 AM, Viresh Kumar wrote:
>>> On 5 August 2014 16:17, Prarit Bhargava <prarit@redhat.com> wrote:
>>>> Nope, not a stupid question.  After reproducing (finally!) yesterday I've been
>>>> wondering the same thing.
>>>
>>> Good to know that :)
>>>
>>>> I've been looking into *exactly* this.  On any platform where
>>>> cpu_weight(affected_cpus) == 1 for a particular cpu this lockdep trace should
>>>> happen.
>>>
>>>> That's what I'm wondering too.  I'm going to instrument the code to find out
>>>> this morning.  I'm wondering if this comes down to a lockdep class issue
>>>> (perhaps lockdep puts globally defined locks like cpufreq_global_kobject in a
>>>> different class?).
>>>
>>> Maybe, I tried this Hack to make this somewhat similar to the other case
>>> on my platform with just two CPUs:
>>>
>>> diff --git a/drivers/cpufreq/cpufreq.c b/drivers/cpufreq/cpufreq.c
>>> index 6f02485..6b4abac 100644
>>> --- a/drivers/cpufreq/cpufreq.c
>>> +++ b/drivers/cpufreq/cpufreq.c
>>> @@ -98,7 +98,7 @@ static DEFINE_MUTEX(cpufreq_governor_mutex);
>>>
>>>    bool have_governor_per_policy(void)
>>>    {
>>> -       return !!(cpufreq_driver->flags & CPUFREQ_HAVE_GOVERNOR_PER_POLICY);
>>> +       return !(cpufreq_driver->flags & CPUFREQ_HAVE_GOVERNOR_PER_POLICY);
>>>    }
>>>    EXPORT_SYMBOL_GPL(have_governor_per_policy);
>>>
>>>
>>> This should result in something similar to setting that per-policy-governor
>>> flag (Actually I could have done that too :)), and I couldn't see that crash :(
>>>
>>> That needs more investigation now, probably we can get some champ of
>>> sysfs stuff like Tejun/Greg into discussion now..
>>
>> Stephen and I looked into this. This is not a sysfs framework difference. The
>> reason we don't have this issue when we use global tunables is because we add
>> the attribute group to the cpufreq_global_kobject and that kobject doesn't have
>> a kobj_type ops similar to the per policy kobject. So, read/write to those
>> attributes do NOT go through the generic show/store ops that wrap every other
>> cpufreq framework attribute read/writes.
>>
>> So, none of those read/write do any kind of locking. They don't race with
>> POLICY_EXIT (because we remove the sysfs group first thing in POLICY_EXIT) but
>> might still race with START/STOPs (not sure, haven't looked closely yet).
>>
>> For example, writing to sampling_rate of ondemand governor might cause a race in
>> update_sampling_rate(). It could race and happen between a STOP and POLICY_EXIT
>> (triggered by hotplug, gov change, etc).
>>
>> So, this might be a completely separate bug that needs fixing when we don't use
>> per policy govs.
>
> Yeah, the show_one & store_one macros don't have any locking in them :/.
>
> Okay ... at least that isn't the issue.  I spent 1/2 the day trying to figure
> out why
>
> diff --git a/drivers/cpufreq/cpufreq.c b/drivers/cpufreq/cpufreq.c
> index fa11a7d..6297c76 100644
> --- a/drivers/cpufreq/cpufreq.c
> +++ b/drivers/cpufreq/cpufreq.c
> @@ -745,12 +745,14 @@ static struct attribute *default_attrs[] = {
>   #define to_policy(k) container_of(k, struct cpufreq_policy, kobj)
>   #define to_attr(a) container_of(a, struct freq_attr, attr)
>
> +/* PRARIT - in the CPUFREQ_HAVE_GOVERNOR_PER_POLICY, this is used */
>   static ssize_t show(struct kobject *kobj, struct attribute *attr, char *buf)
>   {
>          struct cpufreq_policy *policy = to_policy(kobj);
>          struct freq_attr *fattr = to_attr(attr);
>          ssize_t ret;
>
> +       printk("%s: kobject %p\n", __FUNCTION__, kobj);
>          if (!down_read_trylock(&cpufreq_rwsem))
>                  return -EINVAL;
>
> wasn't printing the kobject line when acpi-cpufreq didn't have the
> CPUFREQ_HAVE_GOVERNOR_PER_POLICY flag.  And I agree ... it is a bug.
>

Wait, should I stop reporting bugs so that my patch series gets reviewed? :P

-Saravana


-- 
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
hosted by The Linux Foundation

Re: [PATCH] cpufreq, store_scaling_governor requires policy->rwsem to be held for duration of changing governors [v2]

[Prarit Bhargava]

So back to the original problem, which in short, revolves around these two
functions (with comments included by me):

/* called with kernfs rwsem for this kobj held */
static ssize_t show(struct kobject *kobj, struct attribute *attr, char *buf)
{
        struct cpufreq_policy *policy = to_policy(kobj);
        struct freq_attr *fattr = to_attr(attr);
        ssize_t ret;

        if (!down_read_trylock(&cpufreq_rwsem))
                return -EINVAL;

        down_read(&policy->rwsem);


and

static ssize_t store(struct kobject *kobj, struct attribute *attr,
                     const char *buf, size_t count)
{
        struct cpufreq_policy *policy = to_policy(kobj);
        struct freq_attr *fattr = to_attr(attr);
        ssize_t ret = -EINVAL;

        get_online_cpus();

        if (!cpu_online(policy->cpu))
                goto unlock;

        if (!down_read_trylock(&cpufreq_rwsem))
                goto unlock;

        down_write(&policy->rwsem);

        /* if governor switch, calls sysfs_remove_group
         * and acquires kernfs rwsem for this kobj */

There's another bug here which I haven't had a chance to discuss.  There's the
possibility that we have multiple threads waiting at the store's
down_write(&policy->rwsem) when another thread does a governor switch.  When
the policy->rwsem is released by the governor switch thread, all the other
threads will enter this code path and race through while looking at stale data.

We hit some NULL pointers (very similar to the ones originally reported in this
thread) and, of course, the system dies.

I wonder if the show() down_read(&policy->rwsem) needs to be a
down_read_trylock(), and similarily in the store the down_write(&policy->rwsem)
needs to be a down_write_trylock() ?

We would also have to do a check on policy->governor_enabled to verify that
the data was still valid after taking the lock.

*If* we were to make this change, does that also happen to fix the risk of a
deadlock in this code?

That might be too hacky ... gotta be a better way :/ ...

Anyway, just a thought.

P.

Re: [PATCH] cpufreq, store_scaling_governor requires policy->rwsem to be held for duration of changing governors [v2]

[Saravana Kannan]

On 08/05/2014 03:42 PM, Prarit Bhargava wrote:
> So back to the original problem, which in short, revolves around these two
> functions (with comments included by me):
>
> /* called with kernfs rwsem for this kobj held */
> static ssize_t show(struct kobject *kobj, struct attribute *attr, char *buf)
> {
>          struct cpufreq_policy *policy = to_policy(kobj);
>          struct freq_attr *fattr = to_attr(attr);
>          ssize_t ret;
>
>          if (!down_read_trylock(&cpufreq_rwsem))
>                  return -EINVAL;
>
>          down_read(&policy->rwsem);
>
>
> and
>
> static ssize_t store(struct kobject *kobj, struct attribute *attr,
>                       const char *buf, size_t count)
> {
>          struct cpufreq_policy *policy = to_policy(kobj);
>          struct freq_attr *fattr = to_attr(attr);
>          ssize_t ret = -EINVAL;
>
>          get_online_cpus();
>
>          if (!cpu_online(policy->cpu))
>                  goto unlock;
>
>          if (!down_read_trylock(&cpufreq_rwsem))
>                  goto unlock;
>
>          down_write(&policy->rwsem);
>
>          /* if governor switch, calls sysfs_remove_group
>           * and acquires kernfs rwsem for this kobj */
>
> There's another bug here which I haven't had a chance to discuss.  There's the
> possibility that we have multiple threads waiting at the store's
> down_write(&policy->rwsem) when another thread does a governor switch.  When
> the policy->rwsem is released by the governor switch thread, all the other
> threads will enter this code path and race through while looking at stale data.
>
> We hit some NULL pointers (very similar to the ones originally reported in this
> thread) and, of course, the system dies.
>
> I wonder if the show() down_read(&policy->rwsem) needs to be a
> down_read_trylock(), and similarily in the store the down_write(&policy->rwsem)
> needs to be a down_write_trylock() ?

This will create bigger issues if you make this change to the generic 
show/store. The writes would no longer be reliable even if the race 
could have been handled properly in the kernel (say, a race between a 
call to cpufreq_update_policy() and user space reading/writing 
something). This would be a serious userspace ABI change.

> We would also have to do a check on policy->governor_enabled to verify that
> the data was still valid after taking the lock.
>
> *If* we were to make this change, does that also happen to fix the risk of a
> deadlock in this code?

We should not do the change you are referring to.

>
> That might be too hacky ... gotta be a better way :/ ...
>
> Anyway, just a thought.
>

I definitely have a fix for this and the original race you reported. 
It's basically reverting that commit you reverted + a fix for the 
deadlock. That's the only way to fix the scaling_governor issue.

You fix the deadlock by moving the governor attribute group removing to 
the framework code and doing it before STOP+EXIT to governor without 
holding the policy lock. And the reverse for INIT+STOP.

I'll try to get to it if no one else does.

-Saravana

-- 
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
hosted by The Linux Foundation

Re: [PATCH] cpufreq, store_scaling_governor requires policy->rwsem to be held for duration of changing governors [v2]

[Viresh Kumar]

On 6 August 2014 03:36, Saravana Kannan <skannan@codeaurora.org> wrote:
> Stephen and I looked into this. This is not a sysfs framework difference.
> The reason we don't have this issue when we use global tunables is because
> we add the attribute group to the cpufreq_global_kobject and that kobject
> doesn't have a kobj_type ops similar to the per policy kobject. So,
> read/write to those attributes do NOT go through the generic show/store ops
> that wrap every other cpufreq framework attribute read/writes.
>
> So, none of those read/write do any kind of locking. They don't race with
> POLICY_EXIT (because we remove the sysfs group first thing in POLICY_EXIT)
> but might still race with START/STOPs (not sure, haven't looked closely
> yet).
>
> For example, writing to sampling_rate of ondemand governor might cause a
> race in update_sampling_rate(). It could race and happen between a STOP and
> POLICY_EXIT (triggered by hotplug, gov change, etc).

This sounds good but I couldn't prove it. Doing this on my dual core exynos
doesn't give me that crash report and it should?

diff --git a/drivers/cpufreq/exynos-cpufreq.c b/drivers/cpufreq/exynos-cpufreq.c
index 1e0ec57..027b6f7 100644
--- a/drivers/cpufreq/exynos-cpufreq.c
+++ b/drivers/cpufreq/exynos-cpufreq.c
@@ -139,7 +139,7 @@ static int exynos_cpufreq_cpu_init(struct
cpufreq_policy *policy)
 }

 static struct cpufreq_driver exynos_driver = {
-       .flags          = CPUFREQ_STICKY | CPUFREQ_NEED_INITIAL_FREQ_CHECK,
+       .flags          = CPUFREQ_STICKY |
CPUFREQ_NEED_INITIAL_FREQ_CHECK | CPUFREQ_HAVE_GOVERNOR_PER_POLICY,
        .verify         = cpufreq_generic_frequency_table_verify,
        .target_index   = exynos_target,
        .get            = cpufreq_generic_get,

Re: [PATCH] cpufreq, store_scaling_governor requires policy->rwsem to be held for duration of changing governors [v2]

[Prarit Bhargava]

On 08/06/2014 04:10 AM, Viresh Kumar wrote:
> On 6 August 2014 03:36, Saravana Kannan <skannan@codeaurora.org> wrote:
>> Stephen and I looked into this. This is not a sysfs framework difference.
>> The reason we don't have this issue when we use global tunables is because
>> we add the attribute group to the cpufreq_global_kobject and that kobject
>> doesn't have a kobj_type ops similar to the per policy kobject. So,
>> read/write to those attributes do NOT go through the generic show/store ops
>> that wrap every other cpufreq framework attribute read/writes.
>>
>> So, none of those read/write do any kind of locking. They don't race with
>> POLICY_EXIT (because we remove the sysfs group first thing in POLICY_EXIT)
>> but might still race with START/STOPs (not sure, haven't looked closely
>> yet).
>>
>> For example, writing to sampling_rate of ondemand governor might cause a
>> race in update_sampling_rate(). It could race and happen between a STOP and
>> POLICY_EXIT (triggered by hotplug, gov change, etc).
> 
> This sounds good but I couldn't prove it. Doing this on my dual core exynos
> doesn't give me that crash report and it should?

Are you sure you're not seeing another lockdep warning?  That was my problem --
there was an xfs related lockdep warning which then resulted in lockdep being
disabled from that point on.

P.

> 
> diff --git a/drivers/cpufreq/exynos-cpufreq.c b/drivers/cpufreq/exynos-cpufreq.c
> index 1e0ec57..027b6f7 100644
> --- a/drivers/cpufreq/exynos-cpufreq.c
> +++ b/drivers/cpufreq/exynos-cpufreq.c
> @@ -139,7 +139,7 @@ static int exynos_cpufreq_cpu_init(struct
> cpufreq_policy *policy)
>  }
> 
>  static struct cpufreq_driver exynos_driver = {
> -       .flags          = CPUFREQ_STICKY | CPUFREQ_NEED_INITIAL_FREQ_CHECK,
> +       .flags          = CPUFREQ_STICKY |
> CPUFREQ_NEED_INITIAL_FREQ_CHECK | CPUFREQ_HAVE_GOVERNOR_PER_POLICY,
>         .verify         = cpufreq_generic_frequency_table_verify,
>         .target_index   = exynos_target,
>         .get            = cpufreq_generic_get,
> 

Re: [PATCH] cpufreq, store_scaling_governor requires policy->rwsem to be held for duration of changing governors [v2]

[Prarit Bhargava]

On 08/06/2014 04:10 AM, Viresh Kumar wrote:
> On 6 August 2014 03:36, Saravana Kannan <skannan@codeaurora.org> wrote:
>> Stephen and I looked into this. This is not a sysfs framework difference.
>> The reason we don't have this issue when we use global tunables is because
>> we add the attribute group to the cpufreq_global_kobject and that kobject
>> doesn't have a kobj_type ops similar to the per policy kobject. So,
>> read/write to those attributes do NOT go through the generic show/store ops
>> that wrap every other cpufreq framework attribute read/writes.
>>
>> So, none of those read/write do any kind of locking. They don't race with
>> POLICY_EXIT (because we remove the sysfs group first thing in POLICY_EXIT)
>> but might still race with START/STOPs (not sure, haven't looked closely
>> yet).
>>
>> For example, writing to sampling_rate of ondemand governor might cause a
>> race in update_sampling_rate(). It could race and happen between a STOP and
>> POLICY_EXIT (triggered by hotplug, gov change, etc).
> 
> This sounds good but I couldn't prove it. Doing this on my dual core exynos
> doesn't give me that crash report and it should?

Are you sure you're not seeing another lockdep warning?  That was my problem --
there was an xfs related lockdep warning which then resulted in lockdep being
disabled from that point on.

P.

> 
> diff --git a/drivers/cpufreq/exynos-cpufreq.c b/drivers/cpufreq/exynos-cpufreq.c
> index 1e0ec57..027b6f7 100644
> --- a/drivers/cpufreq/exynos-cpufreq.c
> +++ b/drivers/cpufreq/exynos-cpufreq.c
> @@ -139,7 +139,7 @@ static int exynos_cpufreq_cpu_init(struct
> cpufreq_policy *policy)
>  }
> 
>  static struct cpufreq_driver exynos_driver = {
> -       .flags          = CPUFREQ_STICKY | CPUFREQ_NEED_INITIAL_FREQ_CHECK,
> +       .flags          = CPUFREQ_STICKY |
> CPUFREQ_NEED_INITIAL_FREQ_CHECK | CPUFREQ_HAVE_GOVERNOR_PER_POLICY,
>         .verify         = cpufreq_generic_frequency_table_verify,
>         .target_index   = exynos_target,
>         .get            = cpufreq_generic_get,
> 

Re: [PATCH] cpufreq, store_scaling_governor requires policy->rwsem to be held for duration of changing governors [v2]

[Stephen Boyd]

On 08/06, Prarit Bhargava wrote:
> 
> 
> On 08/06/2014 04:10 AM, Viresh Kumar wrote:
> > On 6 August 2014 03:36, Saravana Kannan <skannan@codeaurora.org> wrote:
> >> Stephen and I looked into this. This is not a sysfs framework difference.
> >> The reason we don't have this issue when we use global tunables is because
> >> we add the attribute group to the cpufreq_global_kobject and that kobject
> >> doesn't have a kobj_type ops similar to the per policy kobject. So,
> >> read/write to those attributes do NOT go through the generic show/store ops
> >> that wrap every other cpufreq framework attribute read/writes.
> >>
> >> So, none of those read/write do any kind of locking. They don't race with
> >> POLICY_EXIT (because we remove the sysfs group first thing in POLICY_EXIT)
> >> but might still race with START/STOPs (not sure, haven't looked closely
> >> yet).
> >>
> >> For example, writing to sampling_rate of ondemand governor might cause a
> >> race in update_sampling_rate(). It could race and happen between a STOP and
> >> POLICY_EXIT (triggered by hotplug, gov change, etc).
> > 
> > This sounds good but I couldn't prove it. Doing this on my dual core exynos
> > doesn't give me that crash report and it should?
> 
> Are you sure you're not seeing another lockdep warning?  That was my problem --
> there was an xfs related lockdep warning which then resulted in lockdep being
> disabled from that point on.
> 

Are we talking about the lockdep splat or the crash that started
this thread or something else? For the lockdep splat you need the
corrected patch in this thread and the per policy governor flag.
I'm not sure how to recreate the crash that started this thread.

-- 
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
hosted by The Linux Foundation

Re: [PATCH] cpufreq, store_scaling_governor requires policy->rwsem to be held for duration of changing governors [v2]

[Viresh Kumar]

[-- Attachment #1: Type: text/plain, Size: 2530 bytes --]

On 6 August 2014 20:38, Stephen Boyd <sboyd@codeaurora.org> wrote:
> On 08/06, Prarit Bhargava wrote:
> > Are you sure you're not seeing another lockdep warning?  That was my problem --
> > there was an xfs related lockdep warning which then resulted in lockdep being
> > disabled from that point on.

There is a fair chance that I might be doing something really really stupid,
but I couldn't get the lockdep warning..

> Are we talking about the lockdep splat or the crash that started
> this thread or something else? For the lockdep splat you need the
> corrected patch in this thread and the per policy governor flag.
> I'm not sure how to recreate the crash that started this thread.

We are talking about the lockdep splat that would happen if we don't
drop locking around EXIT..

This is my full diff over mainline and my .config is attached.
Please enlighten me on what am I missing :)

diff --git a/drivers/cpufreq/cpufreq.c b/drivers/cpufreq/cpufreq.c
index 6f02485..fa11a7d 100644
--- a/drivers/cpufreq/cpufreq.c
+++ b/drivers/cpufreq/cpufreq.c
@@ -2200,9 +2200,7 @@ static int cpufreq_set_policy(struct
cpufreq_policy *policy,
        /* end old governor */
        if (old_gov) {
                __cpufreq_governor(policy, CPUFREQ_GOV_STOP);
-               up_write(&policy->rwsem);
                __cpufreq_governor(policy, CPUFREQ_GOV_POLICY_EXIT);
-               down_write(&policy->rwsem);
        }

        /* start new governor */
@@ -2211,9 +2209,7 @@ static int cpufreq_set_policy(struct
cpufreq_policy *policy,
                if (!__cpufreq_governor(policy, CPUFREQ_GOV_START))
                        goto out;

-               up_write(&policy->rwsem);
                __cpufreq_governor(policy, CPUFREQ_GOV_POLICY_EXIT);
-               down_write(&policy->rwsem);
        }

        /* new governor failed, so re-start old one */
diff --git a/drivers/cpufreq/exynos-cpufreq.c b/drivers/cpufreq/exynos-cpufreq.c
index 1e0ec57..027b6f7 100644
--- a/drivers/cpufreq/exynos-cpufreq.c
+++ b/drivers/cpufreq/exynos-cpufreq.c
@@ -139,7 +139,7 @@ static int exynos_cpufreq_cpu_init(struct
cpufreq_policy *policy)
 }

 static struct cpufreq_driver exynos_driver = {
-       .flags          = CPUFREQ_STICKY | CPUFREQ_NEED_INITIAL_FREQ_CHECK,
+       .flags          = CPUFREQ_STICKY |
CPUFREQ_NEED_INITIAL_FREQ_CHECK | CPUFREQ_HAVE_GOVERNOR_PER_POLICY,
        .verify         = cpufreq_generic_frequency_table_verify,
        .target_index   = exynos_target,
        .get            = cpufreq_generic_get,

[-- Attachment #2: exynos-lockdep-config --]
[-- Type: application/octet-stream, Size: 62787 bytes --]

#
# Automatically generated file; DO NOT EDIT.
# Linux/arm 3.16.0 Kernel Configuration
#
CONFIG_ARM=y
CONFIG_ARM_HAS_SG_CHAIN=y
CONFIG_MIGHT_HAVE_PCI=y
CONFIG_SYS_SUPPORTS_APM_EMULATION=y
CONFIG_HAVE_PROC_CPU=y
CONFIG_NO_IOPORT_MAP=y
CONFIG_STACKTRACE_SUPPORT=y
CONFIG_LOCKDEP_SUPPORT=y
CONFIG_TRACE_IRQFLAGS_SUPPORT=y
CONFIG_RWSEM_XCHGADD_ALGORITHM=y
CONFIG_ARCH_HAS_BANDGAP=y
CONFIG_GENERIC_HWEIGHT=y
CONFIG_GENERIC_CALIBRATE_DELAY=y
CONFIG_NEED_DMA_MAP_STATE=y
CONFIG_ARCH_SUPPORTS_UPROBES=y
CONFIG_VECTORS_BASE=0xffff0000
CONFIG_ARM_PATCH_PHYS_VIRT=y
CONFIG_GENERIC_BUG=y
CONFIG_DEFCONFIG_LIST="/lib/modules/$UNAME_RELEASE/.config"
CONFIG_IRQ_WORK=y
CONFIG_BUILDTIME_EXTABLE_SORT=y

#
# General setup
#
CONFIG_INIT_ENV_ARG_LIMIT=32
CONFIG_CROSS_COMPILE=""
# CONFIG_COMPILE_TEST is not set
CONFIG_LOCALVERSION=""
CONFIG_LOCALVERSION_AUTO=y
CONFIG_HAVE_KERNEL_GZIP=y
CONFIG_HAVE_KERNEL_LZMA=y
CONFIG_HAVE_KERNEL_XZ=y
CONFIG_HAVE_KERNEL_LZO=y
CONFIG_HAVE_KERNEL_LZ4=y
CONFIG_KERNEL_GZIP=y
# CONFIG_KERNEL_LZMA is not set
# CONFIG_KERNEL_XZ is not set
# CONFIG_KERNEL_LZO is not set
# CONFIG_KERNEL_LZ4 is not set
CONFIG_DEFAULT_HOSTNAME="(none)"
CONFIG_SWAP=y
CONFIG_SYSVIPC=y
CONFIG_SYSVIPC_SYSCTL=y
# CONFIG_POSIX_MQUEUE is not set
CONFIG_CROSS_MEMORY_ATTACH=y
# CONFIG_FHANDLE is not set
CONFIG_USELIB=y
# CONFIG_AUDIT is not set
CONFIG_HAVE_ARCH_AUDITSYSCALL=y

#
# IRQ subsystem
#
CONFIG_GENERIC_IRQ_PROBE=y
CONFIG_GENERIC_IRQ_SHOW=y
CONFIG_HARDIRQS_SW_RESEND=y
CONFIG_GENERIC_IRQ_CHIP=y
CONFIG_IRQ_DOMAIN=y
CONFIG_IRQ_FORCED_THREADING=y
CONFIG_SPARSE_IRQ=y
CONFIG_KTIME_SCALAR=y
CONFIG_GENERIC_CLOCKEVENTS=y
CONFIG_GENERIC_CLOCKEVENTS_BUILD=y
CONFIG_ARCH_HAS_TICK_BROADCAST=y
CONFIG_GENERIC_CLOCKEVENTS_BROADCAST=y

#
# Timers subsystem
#
CONFIG_TICK_ONESHOT=y
CONFIG_NO_HZ_COMMON=y
# CONFIG_HZ_PERIODIC is not set
CONFIG_NO_HZ_IDLE=y
# CONFIG_NO_HZ_FULL is not set
CONFIG_NO_HZ=y
CONFIG_HIGH_RES_TIMERS=y

#
# CPU/Task time and stats accounting
#
CONFIG_TICK_CPU_ACCOUNTING=y
# CONFIG_VIRT_CPU_ACCOUNTING_GEN is not set
# CONFIG_IRQ_TIME_ACCOUNTING is not set
# CONFIG_BSD_PROCESS_ACCT is not set
# CONFIG_TASKSTATS is not set

#
# RCU Subsystem
#
CONFIG_TREE_PREEMPT_RCU=y
CONFIG_PREEMPT_RCU=y
CONFIG_RCU_STALL_COMMON=y
# CONFIG_RCU_USER_QS is not set
CONFIG_RCU_FANOUT=32
CONFIG_RCU_FANOUT_LEAF=16
# CONFIG_RCU_FANOUT_EXACT is not set
# CONFIG_RCU_FAST_NO_HZ is not set
# CONFIG_TREE_RCU_TRACE is not set
# CONFIG_RCU_BOOST is not set
# CONFIG_RCU_NOCB_CPU is not set
# CONFIG_IKCONFIG is not set
CONFIG_LOG_BUF_SHIFT=17
CONFIG_GENERIC_SCHED_CLOCK=y
# CONFIG_CGROUPS is not set
# CONFIG_CHECKPOINT_RESTORE is not set
CONFIG_NAMESPACES=y
CONFIG_UTS_NS=y
CONFIG_IPC_NS=y
# CONFIG_USER_NS is not set
CONFIG_PID_NS=y
CONFIG_NET_NS=y
# CONFIG_SCHED_AUTOGROUP is not set
# CONFIG_SYSFS_DEPRECATED is not set
# CONFIG_RELAY is not set
CONFIG_BLK_DEV_INITRD=y
CONFIG_INITRAMFS_SOURCE=""
CONFIG_RD_GZIP=y
CONFIG_RD_BZIP2=y
CONFIG_RD_LZMA=y
CONFIG_RD_XZ=y
CONFIG_RD_LZO=y
CONFIG_RD_LZ4=y
# CONFIG_CC_OPTIMIZE_FOR_SIZE is not set
CONFIG_SYSCTL=y
CONFIG_ANON_INODES=y
CONFIG_HAVE_UID16=y
# CONFIG_EXPERT is not set
CONFIG_UID16=y
# CONFIG_SGETMASK_SYSCALL is not set
CONFIG_SYSFS_SYSCALL=y
# CONFIG_SYSCTL_SYSCALL is not set
CONFIG_KALLSYMS=y
CONFIG_KALLSYMS_ALL=y
CONFIG_PRINTK=y
CONFIG_BUG=y
CONFIG_ELF_CORE=y
CONFIG_BASE_FULL=y
CONFIG_FUTEX=y
CONFIG_EPOLL=y
CONFIG_SIGNALFD=y
CONFIG_TIMERFD=y
CONFIG_EVENTFD=y
CONFIG_SHMEM=y
CONFIG_AIO=y
# CONFIG_EMBEDDED is not set
CONFIG_HAVE_PERF_EVENTS=y
CONFIG_PERF_USE_VMALLOC=y

#
# Kernel Performance Events And Counters
#
# CONFIG_PERF_EVENTS is not set
CONFIG_VM_EVENT_COUNTERS=y
CONFIG_SLUB_DEBUG=y
CONFIG_COMPAT_BRK=y
# CONFIG_SLAB is not set
CONFIG_SLUB=y
CONFIG_SLUB_CPU_PARTIAL=y
# CONFIG_PROFILING is not set
CONFIG_HAVE_OPROFILE=y
# CONFIG_KPROBES is not set
# CONFIG_JUMP_LABEL is not set
# CONFIG_UPROBES is not set
# CONFIG_HAVE_64BIT_ALIGNED_ACCESS is not set
CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS=y
CONFIG_ARCH_USE_BUILTIN_BSWAP=y
CONFIG_HAVE_KPROBES=y
CONFIG_HAVE_KRETPROBES=y
CONFIG_HAVE_ARCH_TRACEHOOK=y
CONFIG_HAVE_DMA_ATTRS=y
CONFIG_HAVE_DMA_CONTIGUOUS=y
CONFIG_GENERIC_SMP_IDLE_THREAD=y
CONFIG_GENERIC_IDLE_POLL_SETUP=y
CONFIG_HAVE_REGS_AND_STACK_ACCESS_API=y
CONFIG_HAVE_CLK=y
CONFIG_HAVE_DMA_API_DEBUG=y
CONFIG_HAVE_PERF_REGS=y
CONFIG_HAVE_PERF_USER_STACK_DUMP=y
CONFIG_HAVE_ARCH_JUMP_LABEL=y
CONFIG_ARCH_WANT_IPC_PARSE_VERSION=y
CONFIG_HAVE_ARCH_SECCOMP_FILTER=y
CONFIG_HAVE_CC_STACKPROTECTOR=y
# CONFIG_CC_STACKPROTECTOR is not set
CONFIG_CC_STACKPROTECTOR_NONE=y
# CONFIG_CC_STACKPROTECTOR_REGULAR is not set
# CONFIG_CC_STACKPROTECTOR_STRONG is not set
CONFIG_HAVE_CONTEXT_TRACKING=y
CONFIG_HAVE_VIRT_CPU_ACCOUNTING_GEN=y
CONFIG_HAVE_IRQ_TIME_ACCOUNTING=y
CONFIG_HAVE_MOD_ARCH_SPECIFIC=y
CONFIG_MODULES_USE_ELF_REL=y
CONFIG_CLONE_BACKWARDS=y
CONFIG_OLD_SIGSUSPEND3=y
CONFIG_OLD_SIGACTION=y

#
# GCOV-based kernel profiling
#
CONFIG_HAVE_GENERIC_DMA_COHERENT=y
CONFIG_SLABINFO=y
CONFIG_RT_MUTEXES=y
CONFIG_BASE_SMALL=0
CONFIG_MODULES=y
# CONFIG_MODULE_FORCE_LOAD is not set
CONFIG_MODULE_UNLOAD=y
# CONFIG_MODULE_FORCE_UNLOAD is not set
# CONFIG_MODVERSIONS is not set
# CONFIG_MODULE_SRCVERSION_ALL is not set
# CONFIG_MODULE_SIG is not set
CONFIG_STOP_MACHINE=y
CONFIG_BLOCK=y
CONFIG_LBDAF=y
# CONFIG_BLK_DEV_BSG is not set
# CONFIG_BLK_DEV_BSGLIB is not set
# CONFIG_BLK_DEV_INTEGRITY is not set
# CONFIG_BLK_CMDLINE_PARSER is not set

#
# Partition Types
#
CONFIG_PARTITION_ADVANCED=y
# CONFIG_ACORN_PARTITION is not set
# CONFIG_AIX_PARTITION is not set
# CONFIG_OSF_PARTITION is not set
# CONFIG_AMIGA_PARTITION is not set
# CONFIG_ATARI_PARTITION is not set
# CONFIG_MAC_PARTITION is not set
CONFIG_MSDOS_PARTITION=y
# CONFIG_BSD_DISKLABEL is not set
# CONFIG_MINIX_SUBPARTITION is not set
# CONFIG_SOLARIS_X86_PARTITION is not set
# CONFIG_UNIXWARE_DISKLABEL is not set
# CONFIG_LDM_PARTITION is not set
# CONFIG_SGI_PARTITION is not set
# CONFIG_ULTRIX_PARTITION is not set
# CONFIG_SUN_PARTITION is not set
# CONFIG_KARMA_PARTITION is not set
CONFIG_EFI_PARTITION=y
# CONFIG_SYSV68_PARTITION is not set
# CONFIG_CMDLINE_PARTITION is not set

#
# IO Schedulers
#
CONFIG_IOSCHED_NOOP=y
CONFIG_IOSCHED_DEADLINE=y
CONFIG_IOSCHED_CFQ=y
# CONFIG_DEFAULT_DEADLINE is not set
CONFIG_DEFAULT_CFQ=y
# CONFIG_DEFAULT_NOOP is not set
CONFIG_DEFAULT_IOSCHED="cfq"
CONFIG_UNINLINE_SPIN_UNLOCK=y
CONFIG_ARCH_SUPPORTS_ATOMIC_RMW=y
CONFIG_RWSEM_SPIN_ON_OWNER=y
CONFIG_FREEZER=y

#
# System Type
#
CONFIG_MMU=y
CONFIG_ARCH_MULTIPLATFORM=y
# CONFIG_ARCH_INTEGRATOR is not set
# CONFIG_ARCH_REALVIEW is not set
# CONFIG_ARCH_VERSATILE is not set
# CONFIG_ARCH_AT91 is not set
# CONFIG_ARCH_CLPS711X is not set
# CONFIG_ARCH_GEMINI is not set
# CONFIG_ARCH_EBSA110 is not set
# CONFIG_ARCH_EP93XX is not set
# CONFIG_ARCH_FOOTBRIDGE is not set
# CONFIG_ARCH_NETX is not set
# CONFIG_ARCH_IOP13XX is not set
# CONFIG_ARCH_IOP32X is not set
# CONFIG_ARCH_IOP33X is not set
# CONFIG_ARCH_IXP4XX is not set
# CONFIG_ARCH_DOVE is not set
# CONFIG_ARCH_KIRKWOOD is not set
# CONFIG_ARCH_MV78XX0 is not set
# CONFIG_ARCH_ORION5X is not set
# CONFIG_ARCH_MMP is not set
# CONFIG_ARCH_KS8695 is not set
# CONFIG_ARCH_W90X900 is not set
# CONFIG_ARCH_LPC32XX is not set
# CONFIG_ARCH_PXA is not set
# CONFIG_ARCH_MSM is not set
# CONFIG_ARCH_SHMOBILE_LEGACY is not set
# CONFIG_ARCH_RPC is not set
# CONFIG_ARCH_SA1100 is not set
# CONFIG_ARCH_S3C24XX is not set
# CONFIG_ARCH_S3C64XX is not set
# CONFIG_ARCH_S5P64X0 is not set
# CONFIG_ARCH_S5PC100 is not set
# CONFIG_ARCH_S5PV210 is not set
# CONFIG_ARCH_DAVINCI is not set
# CONFIG_ARCH_OMAP1 is not set

#
# Multiple platform selection
#

#
# CPU Core family selection
#
# CONFIG_ARCH_MULTI_V6 is not set
CONFIG_ARCH_MULTI_V7=y
CONFIG_ARCH_MULTI_V6_V7=y
# CONFIG_ARCH_MULTI_CPU_AUTO is not set
# CONFIG_ARCH_VIRT is not set
# CONFIG_ARCH_MVEBU is not set
# CONFIG_ARCH_BCM is not set
# CONFIG_ARCH_BERLIN is not set
# CONFIG_ARCH_HIGHBANK is not set
# CONFIG_ARCH_HI3xxx is not set
# CONFIG_ARCH_KEYSTONE is not set
# CONFIG_ARCH_MXC is not set

#
# TI OMAP/AM/DM/DRA Family
#
# CONFIG_ARCH_OMAP3 is not set
# CONFIG_ARCH_OMAP4 is not set
# CONFIG_SOC_OMAP5 is not set
# CONFIG_SOC_AM33XX is not set
# CONFIG_SOC_AM43XX is not set
# CONFIG_SOC_DRA7XX is not set
# CONFIG_ARCH_QCOM is not set
# CONFIG_ARCH_ROCKCHIP is not set
# CONFIG_ARCH_SOCFPGA is not set
# CONFIG_PLAT_SPEAR is not set
# CONFIG_ARCH_STI is not set
CONFIG_ARCH_EXYNOS=y
# CONFIG_ARCH_EXYNOS3 is not set
CONFIG_ARCH_EXYNOS4=y
CONFIG_ARCH_EXYNOS5=y

#
# EXYNOS SoCs
#
CONFIG_CPU_EXYNOS4210=y
CONFIG_SOC_EXYNOS4212=y
CONFIG_SOC_EXYNOS4412=y
CONFIG_SOC_EXYNOS5250=y
CONFIG_SOC_EXYNOS5260=y
CONFIG_SOC_EXYNOS5410=y
CONFIG_SOC_EXYNOS5420=y
CONFIG_SOC_EXYNOS5440=y
CONFIG_SOC_EXYNOS5800=y
CONFIG_PLAT_SAMSUNG=y

#
# Samsung Common options
#

#
# Boot options
#
CONFIG_S5P_DEV_MFC=y

#
# Power management
#
# CONFIG_SAMSUNG_PM_CHECK is not set
# CONFIG_ARCH_SHMOBILE_MULTI is not set
# CONFIG_ARCH_SUNXI is not set
# CONFIG_ARCH_SIRF is not set
# CONFIG_ARCH_TEGRA is not set
# CONFIG_ARCH_U8500 is not set
# CONFIG_ARCH_VEXPRESS is not set
# CONFIG_ARCH_WM8850 is not set
# CONFIG_ARCH_ZYNQ is not set

#
# Processor Type
#
CONFIG_CPU_V7=y
CONFIG_CPU_32v6K=y
CONFIG_CPU_32v7=y
CONFIG_CPU_ABRT_EV7=y
CONFIG_CPU_PABRT_V7=y
CONFIG_CPU_CACHE_V7=y
CONFIG_CPU_CACHE_VIPT=y
CONFIG_CPU_COPY_V6=y
CONFIG_CPU_TLB_V7=y
CONFIG_CPU_HAS_ASID=y
CONFIG_CPU_CP15=y
CONFIG_CPU_CP15_MMU=y

#
# Processor Features
#
# CONFIG_ARM_LPAE is not set
# CONFIG_ARCH_PHYS_ADDR_T_64BIT is not set
CONFIG_ARM_THUMB=y
# CONFIG_ARM_THUMBEE is not set
CONFIG_ARM_VIRT_EXT=y
CONFIG_SWP_EMULATE=y
# CONFIG_CPU_ICACHE_DISABLE is not set
# CONFIG_CPU_DCACHE_DISABLE is not set
# CONFIG_CPU_BPREDICT_DISABLE is not set
CONFIG_KUSER_HELPERS=y
CONFIG_OUTER_CACHE=y
CONFIG_OUTER_CACHE_SYNC=y
CONFIG_MIGHT_HAVE_CACHE_L2X0=y
CONFIG_CACHE_L2X0=y
CONFIG_CACHE_PL310=y
# CONFIG_PL310_ERRATA_588369 is not set
# CONFIG_PL310_ERRATA_727915 is not set
# CONFIG_PL310_ERRATA_753970 is not set
# CONFIG_PL310_ERRATA_769419 is not set
CONFIG_ARM_L1_CACHE_SHIFT_6=y
CONFIG_ARM_L1_CACHE_SHIFT=6
CONFIG_ARM_DMA_MEM_BUFFERABLE=y
CONFIG_MULTI_IRQ_HANDLER=y
# CONFIG_ARM_ERRATA_430973 is not set
# CONFIG_ARM_ERRATA_643719 is not set
# CONFIG_ARM_ERRATA_720789 is not set
# CONFIG_ARM_ERRATA_754322 is not set
# CONFIG_ARM_ERRATA_754327 is not set
# CONFIG_ARM_ERRATA_764369 is not set
# CONFIG_ARM_ERRATA_775420 is not set
# CONFIG_ARM_ERRATA_798181 is not set
# CONFIG_ARM_ERRATA_773022 is not set

#
# Bus support
#
CONFIG_ARM_AMBA=y
# CONFIG_PCI is not set
# CONFIG_PCI_SYSCALL is not set
# CONFIG_PCCARD is not set

#
# Kernel Features
#
CONFIG_HAVE_SMP=y
CONFIG_SMP=y
CONFIG_SMP_ON_UP=y
CONFIG_ARM_CPU_TOPOLOGY=y
# CONFIG_SCHED_MC is not set
# CONFIG_SCHED_SMT is not set
CONFIG_HAVE_ARM_SCU=y
CONFIG_HAVE_ARM_ARCH_TIMER=y
# CONFIG_MCPM is not set
# CONFIG_BIG_LITTLE is not set
CONFIG_VMSPLIT_3G=y
# CONFIG_VMSPLIT_2G is not set
# CONFIG_VMSPLIT_1G is not set
CONFIG_PAGE_OFFSET=0xC0000000
CONFIG_NR_CPUS=8
CONFIG_HOTPLUG_CPU=y
# CONFIG_ARM_PSCI is not set
CONFIG_ARCH_NR_GPIO=512
# CONFIG_PREEMPT_NONE is not set
# CONFIG_PREEMPT_VOLUNTARY is not set
CONFIG_PREEMPT=y
CONFIG_PREEMPT_COUNT=y
CONFIG_HZ_FIXED=200
CONFIG_HZ=200
CONFIG_SCHED_HRTICK=y
# CONFIG_THUMB2_KERNEL is not set
CONFIG_AEABI=y
# CONFIG_OABI_COMPAT is not set
CONFIG_ARCH_HAS_HOLES_MEMORYMODEL=y
# CONFIG_ARCH_SPARSEMEM_DEFAULT is not set
# CONFIG_ARCH_SELECT_MEMORY_MODEL is not set
CONFIG_HAVE_ARCH_PFN_VALID=y
CONFIG_HIGHMEM=y
# CONFIG_HIGHPTE is not set
CONFIG_ARCH_WANT_GENERAL_HUGETLB=y
CONFIG_FLATMEM=y
CONFIG_FLAT_NODE_MEM_MAP=y
CONFIG_HAVE_MEMBLOCK=y
CONFIG_NO_BOOTMEM=y
# CONFIG_HAVE_BOOTMEM_INFO_NODE is not set
CONFIG_PAGEFLAGS_EXTENDED=y
CONFIG_SPLIT_PTLOCK_CPUS=4
CONFIG_COMPACTION=y
CONFIG_MIGRATION=y
# CONFIG_PHYS_ADDR_T_64BIT is not set
CONFIG_ZONE_DMA_FLAG=0
CONFIG_BOUNCE=y
# CONFIG_KSM is not set
CONFIG_DEFAULT_MMAP_MIN_ADDR=4096
# CONFIG_CLEANCACHE is not set
# CONFIG_FRONTSWAP is not set
# CONFIG_CMA is not set
# CONFIG_ZBUD is not set
# CONFIG_ZSMALLOC is not set
CONFIG_FORCE_MAX_ZONEORDER=11
CONFIG_ALIGNMENT_TRAP=y
# CONFIG_UACCESS_WITH_MEMCPY is not set
# CONFIG_SECCOMP is not set
CONFIG_SWIOTLB=y
CONFIG_IOMMU_HELPER=y
# CONFIG_XEN is not set

#
# Boot options
#
CONFIG_USE_OF=y
CONFIG_ATAGS=y
# CONFIG_DEPRECATED_PARAM_STRUCT is not set
CONFIG_ZBOOT_ROM_TEXT=0x0
CONFIG_ZBOOT_ROM_BSS=0x0
CONFIG_ARM_APPENDED_DTB=y
CONFIG_ARM_ATAG_DTB_COMPAT=y
CONFIG_ARM_ATAG_DTB_COMPAT_CMDLINE_FROM_BOOTLOADER=y
# CONFIG_ARM_ATAG_DTB_COMPAT_CMDLINE_EXTEND is not set
CONFIG_CMDLINE="root=/dev/ram0 rw ramdisk=8192 initrd=0x41000000,8M console=ttySAC1,115200 init=/linuxrc mem=256M"
CONFIG_CMDLINE_FROM_BOOTLOADER=y
# CONFIG_CMDLINE_EXTEND is not set
# CONFIG_CMDLINE_FORCE is not set
# CONFIG_KEXEC is not set
# CONFIG_CRASH_DUMP is not set
CONFIG_AUTO_ZRELADDR=y

#
# CPU Power Management
#

#
# CPU Frequency scaling
#
CONFIG_CPU_FREQ=y
CONFIG_CPU_FREQ_GOV_COMMON=y
CONFIG_CPU_FREQ_STAT=y
CONFIG_CPU_FREQ_STAT_DETAILS=y
# CONFIG_CPU_FREQ_DEFAULT_GOV_PERFORMANCE is not set
# CONFIG_CPU_FREQ_DEFAULT_GOV_USERSPACE is not set
# CONFIG_CPU_FREQ_DEFAULT_GOV_ONDEMAND is not set
CONFIG_CPU_FREQ_DEFAULT_GOV_CONSERVATIVE=y
CONFIG_CPU_FREQ_GOV_PERFORMANCE=y
CONFIG_CPU_FREQ_GOV_POWERSAVE=y
CONFIG_CPU_FREQ_GOV_USERSPACE=y
CONFIG_CPU_FREQ_GOV_ONDEMAND=y
CONFIG_CPU_FREQ_GOV_CONSERVATIVE=y
# CONFIG_GENERIC_CPUFREQ_CPU0 is not set

#
# ARM CPU frequency scaling drivers
#
CONFIG_ARM_EXYNOS_CPUFREQ=y
CONFIG_ARM_EXYNOS4210_CPUFREQ=y
CONFIG_ARM_EXYNOS4X12_CPUFREQ=y
CONFIG_ARM_EXYNOS5250_CPUFREQ=y
CONFIG_ARM_EXYNOS5440_CPUFREQ=y
# CONFIG_ARM_EXYNOS_CPU_FREQ_BOOST_SW is not set
# CONFIG_ARM_KIRKWOOD_CPUFREQ is not set

#
# CPU Idle
#
# CONFIG_CPU_IDLE is not set
# CONFIG_ARCH_NEEDS_CPU_IDLE_COUPLED is not set

#
# Floating point emulation
#

#
# At least one emulation must be selected
#
CONFIG_VFP=y
CONFIG_VFPv3=y
CONFIG_NEON=y
# CONFIG_KERNEL_MODE_NEON is not set

#
# Userspace binary formats
#
CONFIG_BINFMT_ELF=y
CONFIG_ARCH_BINFMT_ELF_RANDOMIZE_PIE=y
CONFIG_CORE_DUMP_DEFAULT_ELF_HEADERS=y
CONFIG_BINFMT_SCRIPT=y
# CONFIG_HAVE_AOUT is not set
# CONFIG_BINFMT_MISC is not set
CONFIG_COREDUMP=y

#
# Power management options
#
CONFIG_SUSPEND=y
CONFIG_SUSPEND_FREEZER=y
# CONFIG_HIBERNATION is not set
CONFIG_PM_SLEEP=y
CONFIG_PM_SLEEP_SMP=y
# CONFIG_PM_AUTOSLEEP is not set
# CONFIG_PM_WAKELOCKS is not set
# CONFIG_PM_RUNTIME is not set
CONFIG_PM=y
# CONFIG_PM_DEBUG is not set
# CONFIG_APM_EMULATION is not set
CONFIG_ARCH_HAS_OPP=y
CONFIG_PM_OPP=y
CONFIG_PM_CLK=y
# CONFIG_WQ_POWER_EFFICIENT_DEFAULT is not set
CONFIG_CPU_PM=y
CONFIG_ARCH_SUSPEND_POSSIBLE=y
CONFIG_ARM_CPU_SUSPEND=y
CONFIG_ARCH_HIBERNATION_POSSIBLE=y
CONFIG_NET=y

#
# Networking options
#
CONFIG_PACKET=y
# CONFIG_PACKET_DIAG is not set
CONFIG_UNIX=y
# CONFIG_UNIX_DIAG is not set
CONFIG_XFRM=y
CONFIG_XFRM_ALGO=y
# CONFIG_XFRM_USER is not set
# CONFIG_XFRM_SUB_POLICY is not set
# CONFIG_XFRM_MIGRATE is not set
# CONFIG_XFRM_STATISTICS is not set
CONFIG_NET_KEY=y
# CONFIG_NET_KEY_MIGRATE is not set
CONFIG_INET=y
# CONFIG_IP_MULTICAST is not set
# CONFIG_IP_ADVANCED_ROUTER is not set
# CONFIG_IP_PNP is not set
# CONFIG_NET_IPIP is not set
# CONFIG_NET_IPGRE_DEMUX is not set
CONFIG_NET_IP_TUNNEL=m
# CONFIG_SYN_COOKIES is not set
# CONFIG_NET_IPVTI is not set
# CONFIG_INET_AH is not set
# CONFIG_INET_ESP is not set
# CONFIG_INET_IPCOMP is not set
# CONFIG_INET_XFRM_TUNNEL is not set
CONFIG_INET_TUNNEL=m
CONFIG_INET_XFRM_MODE_TRANSPORT=y
CONFIG_INET_XFRM_MODE_TUNNEL=y
CONFIG_INET_XFRM_MODE_BEET=y
CONFIG_INET_LRO=y
CONFIG_INET_DIAG=y
CONFIG_INET_TCP_DIAG=y
# CONFIG_INET_UDP_DIAG is not set
# CONFIG_TCP_CONG_ADVANCED is not set
CONFIG_TCP_CONG_CUBIC=y
CONFIG_DEFAULT_TCP_CONG="cubic"
# CONFIG_TCP_MD5SIG is not set
CONFIG_IPV6=m
# CONFIG_IPV6_ROUTER_PREF is not set
# CONFIG_IPV6_OPTIMISTIC_DAD is not set
# CONFIG_INET6_AH is not set
# CONFIG_INET6_ESP is not set
# CONFIG_INET6_IPCOMP is not set
# CONFIG_IPV6_MIP6 is not set
# CONFIG_INET6_XFRM_TUNNEL is not set
# CONFIG_INET6_TUNNEL is not set
CONFIG_INET6_XFRM_MODE_TRANSPORT=m
CONFIG_INET6_XFRM_MODE_TUNNEL=m
CONFIG_INET6_XFRM_MODE_BEET=m
# CONFIG_INET6_XFRM_MODE_ROUTEOPTIMIZATION is not set
# CONFIG_IPV6_VTI is not set
CONFIG_IPV6_SIT=m
# CONFIG_IPV6_SIT_6RD is not set
CONFIG_IPV6_NDISC_NODETYPE=y
# CONFIG_IPV6_TUNNEL is not set
# CONFIG_IPV6_GRE is not set
# CONFIG_IPV6_MULTIPLE_TABLES is not set
# CONFIG_IPV6_MROUTE is not set
# CONFIG_NETWORK_SECMARK is not set
# CONFIG_NET_PTP_CLASSIFY is not set
# CONFIG_NETWORK_PHY_TIMESTAMPING is not set
# CONFIG_NETFILTER is not set
# CONFIG_IP_DCCP is not set
# CONFIG_IP_SCTP is not set
# CONFIG_RDS is not set
# CONFIG_TIPC is not set
# CONFIG_ATM is not set
# CONFIG_L2TP is not set
# CONFIG_BRIDGE is not set
CONFIG_HAVE_NET_DSA=y
# CONFIG_VLAN_8021Q is not set
# CONFIG_DECNET is not set
# CONFIG_LLC2 is not set
# CONFIG_IPX is not set
# CONFIG_ATALK is not set
# CONFIG_X25 is not set
# CONFIG_LAPB is not set
# CONFIG_PHONET is not set
# CONFIG_IEEE802154 is not set
# CONFIG_NET_SCHED is not set
# CONFIG_DCB is not set
# CONFIG_BATMAN_ADV is not set
# CONFIG_OPENVSWITCH is not set
# CONFIG_VSOCKETS is not set
# CONFIG_NETLINK_MMAP is not set
# CONFIG_NETLINK_DIAG is not set
# CONFIG_NET_MPLS_GSO is not set
# CONFIG_HSR is not set
CONFIG_RPS=y
CONFIG_RFS_ACCEL=y
CONFIG_XPS=y
CONFIG_NET_RX_BUSY_POLL=y
CONFIG_BQL=y
# CONFIG_BPF_JIT is not set
CONFIG_NET_FLOW_LIMIT=y

#
# Network testing
#
# CONFIG_NET_PKTGEN is not set
# CONFIG_HAMRADIO is not set
# CONFIG_CAN is not set
# CONFIG_IRDA is not set
# CONFIG_BT is not set
# CONFIG_AF_RXRPC is not set
CONFIG_WIRELESS=y
# CONFIG_CFG80211 is not set
# CONFIG_LIB80211 is not set

#
# CFG80211 needs to be enabled for MAC80211
#
# CONFIG_WIMAX is not set
# CONFIG_RFKILL is not set
CONFIG_RFKILL_REGULATOR=y
# CONFIG_NET_9P is not set
# CONFIG_CAIF is not set
# CONFIG_CEPH_LIB is not set
# CONFIG_NFC is not set
CONFIG_HAVE_BPF_JIT=y

#
# Device Drivers
#

#
# Generic Driver Options
#
CONFIG_UEVENT_HELPER=y
CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
CONFIG_DEVTMPFS=y
CONFIG_DEVTMPFS_MOUNT=y
CONFIG_STANDALONE=y
CONFIG_PREVENT_FIRMWARE_BUILD=y
CONFIG_FW_LOADER=y
CONFIG_FIRMWARE_IN_KERNEL=y
CONFIG_EXTRA_FIRMWARE=""
CONFIG_FW_LOADER_USER_HELPER=y
# CONFIG_DEBUG_DRIVER is not set
# CONFIG_DEBUG_DEVRES is not set
# CONFIG_SYS_HYPERVISOR is not set
# CONFIG_GENERIC_CPU_DEVICES is not set
CONFIG_REGMAP=y
CONFIG_REGMAP_I2C=y
CONFIG_REGMAP_IRQ=y
# CONFIG_DMA_SHARED_BUFFER is not set

#
# Bus devices
#
# CONFIG_BRCMSTB_GISB_ARB is not set
# CONFIG_ARM_CCI is not set
# CONFIG_VEXPRESS_CONFIG is not set
# CONFIG_CONNECTOR is not set
# CONFIG_MTD is not set
CONFIG_DTC=y
CONFIG_OF=y

#
# Device Tree and Open Firmware support
#
# CONFIG_OF_SELFTEST is not set
CONFIG_OF_FLATTREE=y
CONFIG_OF_EARLY_FLATTREE=y
CONFIG_OF_ADDRESS=y
CONFIG_OF_IRQ=y
CONFIG_OF_NET=y
CONFIG_OF_MDIO=y
CONFIG_OF_RESERVED_MEM=y
CONFIG_ARCH_MIGHT_HAVE_PC_PARPORT=y
# CONFIG_PARPORT is not set
CONFIG_BLK_DEV=y
# CONFIG_BLK_DEV_NULL_BLK is not set
# CONFIG_BLK_DEV_COW_COMMON is not set
CONFIG_BLK_DEV_LOOP=y
CONFIG_BLK_DEV_LOOP_MIN_COUNT=8
CONFIG_BLK_DEV_CRYPTOLOOP=y
# CONFIG_BLK_DEV_DRBD is not set
# CONFIG_BLK_DEV_NBD is not set
CONFIG_BLK_DEV_RAM=y
CONFIG_BLK_DEV_RAM_COUNT=16
CONFIG_BLK_DEV_RAM_SIZE=8192
# CONFIG_BLK_DEV_XIP is not set
# CONFIG_CDROM_PKTCDVD is not set
# CONFIG_ATA_OVER_ETH is not set
# CONFIG_MG_DISK is not set
# CONFIG_BLK_DEV_RBD is not set

#
# Misc devices
#
# CONFIG_SENSORS_LIS3LV02D is not set
# CONFIG_AD525X_DPOT is not set
# CONFIG_DUMMY_IRQ is not set
# CONFIG_ICS932S401 is not set
# CONFIG_ENCLOSURE_SERVICES is not set
# CONFIG_APDS9802ALS is not set
# CONFIG_ISL29003 is not set
# CONFIG_ISL29020 is not set
# CONFIG_SENSORS_TSL2550 is not set
# CONFIG_SENSORS_BH1780 is not set
# CONFIG_SENSORS_BH1770 is not set
# CONFIG_SENSORS_APDS990X is not set
# CONFIG_HMC6352 is not set
# CONFIG_DS1682 is not set
# CONFIG_BMP085_I2C is not set
# CONFIG_USB_SWITCH_FSA9480 is not set
CONFIG_SRAM=y
# CONFIG_C2PORT is not set

#
# EEPROM support
#
# CONFIG_EEPROM_AT24 is not set
# CONFIG_EEPROM_LEGACY is not set
# CONFIG_EEPROM_MAX6875 is not set
# CONFIG_EEPROM_93CX6 is not set

#
# Texas Instruments shared transport line discipline
#
# CONFIG_TI_ST is not set
# CONFIG_SENSORS_LIS3_I2C is not set

#
# Altera FPGA firmware download module
#
# CONFIG_ALTERA_STAPL is not set

#
# Intel MIC Host Driver
#

#
# Intel MIC Card Driver
#
# CONFIG_ECHO is not set

#
# SCSI device support
#
CONFIG_SCSI_MOD=y
# CONFIG_RAID_ATTRS is not set
CONFIG_SCSI=y
CONFIG_SCSI_DMA=y
# CONFIG_SCSI_TGT is not set
# CONFIG_SCSI_NETLINK is not set
CONFIG_SCSI_PROC_FS=y

#
# SCSI support type (disk, tape, CD-ROM)
#
CONFIG_BLK_DEV_SD=y
# CONFIG_CHR_DEV_ST is not set
# CONFIG_CHR_DEV_OSST is not set
# CONFIG_BLK_DEV_SR is not set
CONFIG_CHR_DEV_SG=y
# CONFIG_CHR_DEV_SCH is not set
# CONFIG_SCSI_MULTI_LUN is not set
# CONFIG_SCSI_CONSTANTS is not set
# CONFIG_SCSI_LOGGING is not set
# CONFIG_SCSI_SCAN_ASYNC is not set

#
# SCSI Transports
#
# CONFIG_SCSI_SPI_ATTRS is not set
# CONFIG_SCSI_FC_ATTRS is not set
# CONFIG_SCSI_ISCSI_ATTRS is not set
# CONFIG_SCSI_SAS_ATTRS is not set
# CONFIG_SCSI_SAS_LIBSAS is not set
# CONFIG_SCSI_SRP_ATTRS is not set
CONFIG_SCSI_LOWLEVEL=y
# CONFIG_ISCSI_TCP is not set
# CONFIG_ISCSI_BOOT_SYSFS is not set
# CONFIG_SCSI_UFSHCD is not set
# CONFIG_LIBFC is not set
# CONFIG_LIBFCOE is not set
# CONFIG_SCSI_DEBUG is not set
# CONFIG_SCSI_DH is not set
# CONFIG_SCSI_OSD_INITIATOR is not set
# CONFIG_ATA is not set
CONFIG_MD=y
# CONFIG_BLK_DEV_MD is not set
# CONFIG_BCACHE is not set
CONFIG_BLK_DEV_DM_BUILTIN=y
CONFIG_BLK_DEV_DM=y
# CONFIG_DM_DEBUG is not set
CONFIG_DM_CRYPT=m
# CONFIG_DM_SNAPSHOT is not set
# CONFIG_DM_THIN_PROVISIONING is not set
# CONFIG_DM_CACHE is not set
# CONFIG_DM_ERA is not set
# CONFIG_DM_MIRROR is not set
# CONFIG_DM_RAID is not set
# CONFIG_DM_ZERO is not set
# CONFIG_DM_MULTIPATH is not set
# CONFIG_DM_DELAY is not set
# CONFIG_DM_UEVENT is not set
# CONFIG_DM_FLAKEY is not set
# CONFIG_DM_VERITY is not set
# CONFIG_DM_SWITCH is not set
# CONFIG_TARGET_CORE is not set
CONFIG_NETDEVICES=y
CONFIG_MII=y
CONFIG_NET_CORE=y
# CONFIG_BONDING is not set
# CONFIG_DUMMY is not set
# CONFIG_EQUALIZER is not set
# CONFIG_NET_TEAM is not set
# CONFIG_MACVLAN is not set
# CONFIG_VXLAN is not set
# CONFIG_NETCONSOLE is not set
# CONFIG_NETPOLL is not set
# CONFIG_NET_POLL_CONTROLLER is not set
# CONFIG_TUN is not set
# CONFIG_VETH is not set
# CONFIG_NLMON is not set

#
# CAIF transport drivers
#

#
# Distributed Switch Architecture drivers
#
# CONFIG_NET_DSA_MV88E6XXX is not set
# CONFIG_NET_DSA_MV88E6060 is not set
# CONFIG_NET_DSA_MV88E6XXX_NEED_PPU is not set
# CONFIG_NET_DSA_MV88E6131 is not set
# CONFIG_NET_DSA_MV88E6123_61_65 is not set
CONFIG_ETHERNET=y
# CONFIG_ALTERA_TSE is not set
CONFIG_NET_VENDOR_ARC=y
# CONFIG_ARC_EMAC is not set
CONFIG_NET_CADENCE=y
CONFIG_NET_VENDOR_BROADCOM=y
# CONFIG_B44 is not set
# CONFIG_BCMGENET is not set
# CONFIG_SYSTEMPORT is not set
# CONFIG_NET_CALXEDA_XGMAC is not set
CONFIG_NET_VENDOR_CIRRUS=y
# CONFIG_CS89x0 is not set
# CONFIG_DM9000 is not set
# CONFIG_DNET is not set
CONFIG_NET_VENDOR_FARADAY=y
# CONFIG_FTMAC100 is not set
# CONFIG_FTGMAC100 is not set
CONFIG_NET_VENDOR_HISILICON=y
# CONFIG_HIX5HD2_GMAC is not set
CONFIG_NET_VENDOR_INTEL=y
CONFIG_NET_VENDOR_I825XX=y
CONFIG_NET_VENDOR_MARVELL=y
# CONFIG_MVMDIO is not set
CONFIG_NET_VENDOR_MICREL=y
# CONFIG_KS8851_MLL is not set
CONFIG_NET_VENDOR_NATSEMI=y
CONFIG_NET_VENDOR_8390=y
# CONFIG_AX88796 is not set
# CONFIG_ETHOC is not set
# CONFIG_SH_ETH is not set
CONFIG_NET_VENDOR_SAMSUNG=y
# CONFIG_SXGBE_ETH is not set
CONFIG_NET_VENDOR_SEEQ=y
CONFIG_NET_VENDOR_SMSC=y
# CONFIG_SMC91X is not set
# CONFIG_SMC911X is not set
CONFIG_SMSC911X=y
# CONFIG_SMSC911X_ARCH_HOOKS is not set
CONFIG_NET_VENDOR_STMICRO=y
# CONFIG_STMMAC_ETH is not set
CONFIG_NET_VENDOR_VIA=y
# CONFIG_VIA_RHINE is not set
# CONFIG_VIA_VELOCITY is not set
CONFIG_NET_VENDOR_WIZNET=y
# CONFIG_WIZNET_W5100 is not set
# CONFIG_WIZNET_W5300 is not set
CONFIG_PHYLIB=y

#
# MII PHY device drivers
#
# CONFIG_AT803X_PHY is not set
# CONFIG_AMD_PHY is not set
# CONFIG_AMD_XGBE_PHY is not set
# CONFIG_MARVELL_PHY is not set
# CONFIG_DAVICOM_PHY is not set
# CONFIG_QSEMI_PHY is not set
# CONFIG_LXT_PHY is not set
# CONFIG_CICADA_PHY is not set
# CONFIG_VITESSE_PHY is not set
# CONFIG_SMSC_PHY is not set
# CONFIG_BROADCOM_PHY is not set
# CONFIG_BCM7XXX_PHY is not set
# CONFIG_BCM87XX_PHY is not set
# CONFIG_ICPLUS_PHY is not set
# CONFIG_REALTEK_PHY is not set
# CONFIG_NATIONAL_PHY is not set
# CONFIG_STE10XP is not set
# CONFIG_LSI_ET1011C_PHY is not set
# CONFIG_MICREL_PHY is not set
# CONFIG_FIXED_PHY is not set
# CONFIG_MDIO_BITBANG is not set
# CONFIG_MDIO_BUS_MUX_GPIO is not set
# CONFIG_MDIO_BUS_MUX_MMIOREG is not set
# CONFIG_PPP is not set
# CONFIG_SLIP is not set

#
# USB Network Adapters
#
# CONFIG_USB_CATC is not set
# CONFIG_USB_KAWETH is not set
# CONFIG_USB_PEGASUS is not set
# CONFIG_USB_RTL8150 is not set
# CONFIG_USB_RTL8152 is not set
CONFIG_USB_USBNET=y
CONFIG_USB_NET_AX8817X=y
CONFIG_USB_NET_AX88179_178A=y
CONFIG_USB_NET_CDCETHER=y
# CONFIG_USB_NET_CDC_EEM is not set
CONFIG_USB_NET_CDC_NCM=y
# CONFIG_USB_NET_HUAWEI_CDC_NCM is not set
# CONFIG_USB_NET_CDC_MBIM is not set
# CONFIG_USB_NET_DM9601 is not set
# CONFIG_USB_NET_SR9700 is not set
# CONFIG_USB_NET_SR9800 is not set
CONFIG_USB_NET_SMSC75XX=y
CONFIG_USB_NET_SMSC95XX=y
# CONFIG_USB_NET_GL620A is not set
CONFIG_USB_NET_NET1080=y
# CONFIG_USB_NET_PLUSB is not set
# CONFIG_USB_NET_MCS7830 is not set
# CONFIG_USB_NET_RNDIS_HOST is not set
CONFIG_USB_NET_CDC_SUBSET=y
# CONFIG_USB_ALI_M5632 is not set
# CONFIG_USB_AN2720 is not set
CONFIG_USB_BELKIN=y
CONFIG_USB_ARMLINUX=y
# CONFIG_USB_EPSON2888 is not set
# CONFIG_USB_KC2190 is not set
CONFIG_USB_NET_ZAURUS=y
# CONFIG_USB_NET_CX82310_ETH is not set
# CONFIG_USB_NET_KALMIA is not set
# CONFIG_USB_NET_QMI_WWAN is not set
# CONFIG_USB_NET_INT51X1 is not set
# CONFIG_USB_IPHETH is not set
# CONFIG_USB_SIERRA_NET is not set
# CONFIG_USB_VL600 is not set
CONFIG_WLAN=y
# CONFIG_HOSTAP is not set
# CONFIG_WL_TI is not set

#
# Enable WiMAX (Networking options) to see the WiMAX drivers
#
# CONFIG_WAN is not set
# CONFIG_ISDN is not set

#
# Input device support
#
CONFIG_INPUT=y
# CONFIG_INPUT_FF_MEMLESS is not set
# CONFIG_INPUT_POLLDEV is not set
# CONFIG_INPUT_SPARSEKMAP is not set
CONFIG_INPUT_MATRIXKMAP=y

#
# Userland interfaces
#
CONFIG_INPUT_MOUSEDEV=y
CONFIG_INPUT_MOUSEDEV_PSAUX=y
CONFIG_INPUT_MOUSEDEV_SCREEN_X=1024
CONFIG_INPUT_MOUSEDEV_SCREEN_Y=768
# CONFIG_INPUT_JOYDEV is not set
CONFIG_INPUT_EVDEV=y
# CONFIG_INPUT_EVBUG is not set

#
# Input Device Drivers
#
CONFIG_INPUT_KEYBOARD=y
# CONFIG_KEYBOARD_ADP5588 is not set
# CONFIG_KEYBOARD_ADP5589 is not set
CONFIG_KEYBOARD_ATKBD=y
# CONFIG_KEYBOARD_QT1070 is not set
# CONFIG_KEYBOARD_QT2160 is not set
# CONFIG_KEYBOARD_LKKBD is not set
CONFIG_KEYBOARD_GPIO=y
# CONFIG_KEYBOARD_GPIO_POLLED is not set
# CONFIG_KEYBOARD_TCA6416 is not set
# CONFIG_KEYBOARD_TCA8418 is not set
# CONFIG_KEYBOARD_MATRIX is not set
# CONFIG_KEYBOARD_LM8333 is not set
# CONFIG_KEYBOARD_MAX7359 is not set
# CONFIG_KEYBOARD_MCS is not set
# CONFIG_KEYBOARD_MPR121 is not set
# CONFIG_KEYBOARD_NEWTON is not set
# CONFIG_KEYBOARD_OPENCORES is not set
CONFIG_KEYBOARD_SAMSUNG=y
# CONFIG_KEYBOARD_STOWAWAY is not set
# CONFIG_KEYBOARD_SUNKBD is not set
# CONFIG_KEYBOARD_OMAP4 is not set
# CONFIG_KEYBOARD_XTKBD is not set
CONFIG_KEYBOARD_CROS_EC=y
CONFIG_INPUT_MOUSE=y
# CONFIG_MOUSE_PS2 is not set
# CONFIG_MOUSE_SERIAL is not set
# CONFIG_MOUSE_APPLETOUCH is not set
# CONFIG_MOUSE_BCM5974 is not set
CONFIG_MOUSE_CYAPA=y
# CONFIG_MOUSE_VSXXXAA is not set
# CONFIG_MOUSE_GPIO is not set
# CONFIG_MOUSE_SYNAPTICS_I2C is not set
# CONFIG_MOUSE_SYNAPTICS_USB is not set
# CONFIG_INPUT_JOYSTICK is not set
# CONFIG_INPUT_TABLET is not set
CONFIG_INPUT_TOUCHSCREEN=y
CONFIG_OF_TOUCHSCREEN=y
# CONFIG_TOUCHSCREEN_AD7879 is not set
# CONFIG_TOUCHSCREEN_ATMEL_MXT is not set
# CONFIG_TOUCHSCREEN_AUO_PIXCIR is not set
# CONFIG_TOUCHSCREEN_BU21013 is not set
# CONFIG_TOUCHSCREEN_CY8CTMG110 is not set
# CONFIG_TOUCHSCREEN_CYTTSP_CORE is not set
# CONFIG_TOUCHSCREEN_CYTTSP4_CORE is not set
# CONFIG_TOUCHSCREEN_DYNAPRO is not set
# CONFIG_TOUCHSCREEN_HAMPSHIRE is not set
# CONFIG_TOUCHSCREEN_EETI is not set
# CONFIG_TOUCHSCREEN_EGALAX is not set
# CONFIG_TOUCHSCREEN_FUJITSU is not set
# CONFIG_TOUCHSCREEN_ILI210X is not set
# CONFIG_TOUCHSCREEN_GUNZE is not set
# CONFIG_TOUCHSCREEN_ELO is not set
# CONFIG_TOUCHSCREEN_WACOM_W8001 is not set
# CONFIG_TOUCHSCREEN_WACOM_I2C is not set
# CONFIG_TOUCHSCREEN_MAX11801 is not set
# CONFIG_TOUCHSCREEN_MCS5000 is not set
# CONFIG_TOUCHSCREEN_MMS114 is not set
# CONFIG_TOUCHSCREEN_MTOUCH is not set
# CONFIG_TOUCHSCREEN_INEXIO is not set
# CONFIG_TOUCHSCREEN_MK712 is not set
# CONFIG_TOUCHSCREEN_PENMOUNT is not set
# CONFIG_TOUCHSCREEN_EDT_FT5X06 is not set
# CONFIG_TOUCHSCREEN_TOUCHRIGHT is not set
# CONFIG_TOUCHSCREEN_TOUCHWIN is not set
# CONFIG_TOUCHSCREEN_PIXCIR is not set
# CONFIG_TOUCHSCREEN_USB_COMPOSITE is not set
# CONFIG_TOUCHSCREEN_TOUCHIT213 is not set
# CONFIG_TOUCHSCREEN_TSC_SERIO is not set
# CONFIG_TOUCHSCREEN_TSC2007 is not set
# CONFIG_TOUCHSCREEN_ST1232 is not set
# CONFIG_TOUCHSCREEN_SUR40 is not set
# CONFIG_TOUCHSCREEN_TPS6507X is not set
# CONFIG_TOUCHSCREEN_ZFORCE is not set
# CONFIG_INPUT_MISC is not set

#
# Hardware I/O ports
#
CONFIG_SERIO=y
CONFIG_SERIO_SERPORT=y
# CONFIG_SERIO_AMBAKMI is not set
CONFIG_SERIO_LIBPS2=y
# CONFIG_SERIO_RAW is not set
# CONFIG_SERIO_ALTERA_PS2 is not set
# CONFIG_SERIO_PS2MULT is not set
# CONFIG_SERIO_ARC_PS2 is not set
# CONFIG_SERIO_APBPS2 is not set
# CONFIG_GAMEPORT is not set

#
# Character devices
#
CONFIG_TTY=y
CONFIG_VT=y
CONFIG_CONSOLE_TRANSLATIONS=y
CONFIG_VT_CONSOLE=y
CONFIG_VT_CONSOLE_SLEEP=y
CONFIG_HW_CONSOLE=y
CONFIG_VT_HW_CONSOLE_BINDING=y
CONFIG_UNIX98_PTYS=y
# CONFIG_DEVPTS_MULTIPLE_INSTANCES is not set
CONFIG_LEGACY_PTYS=y
CONFIG_LEGACY_PTY_COUNT=256
# CONFIG_SERIAL_NONSTANDARD is not set
# CONFIG_N_GSM is not set
# CONFIG_TRACE_SINK is not set
CONFIG_DEVKMEM=y

#
# Serial drivers
#
CONFIG_SERIAL_8250=y
CONFIG_SERIAL_8250_DEPRECATED_OPTIONS=y
# CONFIG_SERIAL_8250_CONSOLE is not set
CONFIG_SERIAL_8250_NR_UARTS=4
CONFIG_SERIAL_8250_RUNTIME_UARTS=4
# CONFIG_SERIAL_8250_EXTENDED is not set
# CONFIG_SERIAL_8250_DW is not set
# CONFIG_SERIAL_8250_EM is not set

#
# Non-8250 serial port support
#
# CONFIG_SERIAL_AMBA_PL010 is not set
# CONFIG_SERIAL_AMBA_PL011 is not set
# CONFIG_SERIAL_EARLYCON_ARM_SEMIHOST is not set
CONFIG_SERIAL_SAMSUNG=y
CONFIG_SERIAL_SAMSUNG_UARTS_4=y
CONFIG_SERIAL_SAMSUNG_UARTS=4
CONFIG_SERIAL_SAMSUNG_CONSOLE=y
CONFIG_SERIAL_CORE=y
CONFIG_SERIAL_CORE_CONSOLE=y
CONFIG_SERIAL_OF_PLATFORM=y
# CONFIG_SERIAL_SCCNXP is not set
# CONFIG_SERIAL_SC16IS7XX is not set
# CONFIG_SERIAL_ALTERA_JTAGUART is not set
# CONFIG_SERIAL_ALTERA_UART is not set
# CONFIG_SERIAL_XILINX_PS_UART is not set
# CONFIG_SERIAL_ARC is not set
# CONFIG_SERIAL_FSL_LPUART is not set
# CONFIG_SERIAL_ST_ASC is not set
# CONFIG_HVC_DCC is not set
# CONFIG_IPMI_HANDLER is not set
CONFIG_HW_RANDOM=y
# CONFIG_HW_RANDOM_TIMERIOMEM is not set
CONFIG_HW_RANDOM_EXYNOS=y
CONFIG_HW_RANDOM_TPM=y
# CONFIG_R3964 is not set
# CONFIG_RAW_DRIVER is not set
CONFIG_TCG_TPM=y
# CONFIG_TCG_TIS_I2C_ATMEL is not set
CONFIG_TCG_TIS_I2C_INFINEON=y
# CONFIG_TCG_TIS_I2C_NUVOTON is not set
# CONFIG_TCG_ST33_I2C is not set
CONFIG_I2C=y
CONFIG_I2C_BOARDINFO=y
CONFIG_I2C_COMPAT=y
# CONFIG_I2C_CHARDEV is not set
CONFIG_I2C_MUX=y

#
# Multiplexer I2C Chip support
#
CONFIG_I2C_ARB_GPIO_CHALLENGE=y
# CONFIG_I2C_MUX_GPIO is not set
# CONFIG_I2C_MUX_PCA9541 is not set
# CONFIG_I2C_MUX_PCA954x is not set
# CONFIG_I2C_MUX_PINCTRL is not set
CONFIG_I2C_HELPER_AUTO=y

#
# I2C Hardware Bus support
#

#
# I2C system bus drivers (mostly embedded / system-on-chip)
#
# CONFIG_I2C_CBUS_GPIO is not set
# CONFIG_I2C_DESIGNWARE_PLATFORM is not set
CONFIG_I2C_EXYNOS5=y
# CONFIG_I2C_GPIO is not set
# CONFIG_I2C_NOMADIK is not set
# CONFIG_I2C_OCORES is not set
# CONFIG_I2C_PCA_PLATFORM is not set
# CONFIG_I2C_PXA_PCI is not set
# CONFIG_I2C_RK3X is not set
CONFIG_HAVE_S3C2410_I2C=y
CONFIG_I2C_S3C2410=y
# CONFIG_I2C_SIMTEC is not set
# CONFIG_I2C_XILINX is not set

#
# External I2C/SMBus adapter drivers
#
# CONFIG_I2C_DIOLAN_U2C is not set
# CONFIG_I2C_PARPORT_LIGHT is not set
# CONFIG_I2C_ROBOTFUZZ_OSIF is not set
# CONFIG_I2C_TAOS_EVM is not set
# CONFIG_I2C_TINY_USB is not set

#
# Other I2C/SMBus bus drivers
#
# CONFIG_I2C_CROS_EC_TUNNEL is not set
# CONFIG_I2C_STUB is not set
# CONFIG_I2C_DEBUG_CORE is not set
# CONFIG_I2C_DEBUG_ALGO is not set
# CONFIG_I2C_DEBUG_BUS is not set
# CONFIG_SPI is not set
# CONFIG_SPMI is not set
# CONFIG_HSI is not set

#
# PPS support
#
# CONFIG_PPS is not set

#
# PPS generators support
#

#
# PTP clock support
#
# CONFIG_PTP_1588_CLOCK is not set

#
# Enable PHYLIB and NETWORK_PHY_TIMESTAMPING to see the additional clocks.
#
CONFIG_PINCTRL=y

#
# Pin controllers
#
CONFIG_PINMUX=y
CONFIG_PINCONF=y
# CONFIG_DEBUG_PINCTRL is not set
# CONFIG_PINCTRL_BCM281XX is not set
# CONFIG_PINCTRL_APQ8064 is not set
# CONFIG_PINCTRL_IPQ8064 is not set
# CONFIG_PINCTRL_SINGLE is not set
CONFIG_PINCTRL_SAMSUNG=y
CONFIG_PINCTRL_EXYNOS=y
CONFIG_PINCTRL_EXYNOS5440=y
CONFIG_ARCH_HAVE_CUSTOM_GPIO_H=y
CONFIG_ARCH_WANT_OPTIONAL_GPIOLIB=y
CONFIG_ARCH_REQUIRE_GPIOLIB=y
CONFIG_GPIOLIB=y
CONFIG_GPIO_DEVRES=y
CONFIG_OF_GPIO=y
CONFIG_DEBUG_GPIO=y
# CONFIG_GPIO_SYSFS is not set

#
# Memory mapped GPIO drivers:
#
# CONFIG_GPIO_GENERIC_PLATFORM is not set
# CONFIG_GPIO_DWAPB is not set
# CONFIG_GPIO_EM is not set
# CONFIG_GPIO_ZEVIO is not set
# CONFIG_GPIO_PL061 is not set
# CONFIG_GPIO_SCH311X is not set
# CONFIG_GPIO_GRGPIO is not set

#
# I2C GPIO expanders:
#
# CONFIG_GPIO_MAX7300 is not set
# CONFIG_GPIO_MAX732X is not set
# CONFIG_GPIO_PCA953X is not set
# CONFIG_GPIO_PCF857X is not set
# CONFIG_GPIO_SX150X is not set
# CONFIG_GPIO_ADP5588 is not set
# CONFIG_GPIO_ADNP is not set

#
# PCI GPIO expanders:
#

#
# SPI GPIO expanders:
#
# CONFIG_GPIO_MCP23S08 is not set

#
# AC97 GPIO expanders:
#

#
# LPC GPIO expanders:
#

#
# MODULbus GPIO expanders:
#
# CONFIG_GPIO_BCM_KONA is not set

#
# USB GPIO expanders:
#
# CONFIG_W1 is not set
# CONFIG_POWER_SUPPLY is not set
# CONFIG_POWER_AVS is not set
# CONFIG_HWMON is not set
# CONFIG_THERMAL is not set
# CONFIG_WATCHDOG is not set
CONFIG_SSB_POSSIBLE=y

#
# Sonics Silicon Backplane
#
# CONFIG_SSB is not set
CONFIG_BCMA_POSSIBLE=y

#
# Broadcom specific AMBA
#
# CONFIG_BCMA is not set

#
# Multifunction device drivers
#
CONFIG_MFD_CORE=y
# CONFIG_MFD_AS3711 is not set
# CONFIG_MFD_AS3722 is not set
# CONFIG_PMIC_ADP5520 is not set
# CONFIG_MFD_AAT2870_CORE is not set
# CONFIG_MFD_BCM590XX is not set
# CONFIG_MFD_AXP20X is not set
CONFIG_MFD_CROS_EC=y
CONFIG_MFD_CROS_EC_I2C=y
# CONFIG_MFD_ASIC3 is not set
# CONFIG_PMIC_DA903X is not set
# CONFIG_MFD_DA9052_I2C is not set
# CONFIG_MFD_DA9055 is not set
# CONFIG_MFD_DA9063 is not set
# CONFIG_MFD_MC13XXX_I2C is not set
# CONFIG_HTC_EGPIO is not set
# CONFIG_HTC_PASIC3 is not set
# CONFIG_HTC_I2CPLD is not set
# CONFIG_MFD_KEMPLD is not set
# CONFIG_MFD_88PM800 is not set
# CONFIG_MFD_88PM805 is not set
# CONFIG_MFD_88PM860X is not set
# CONFIG_MFD_MAX14577 is not set
CONFIG_MFD_MAX77686=y
# CONFIG_MFD_MAX77693 is not set
# CONFIG_MFD_MAX8907 is not set
# CONFIG_MFD_MAX8925 is not set
CONFIG_MFD_MAX8997=y
# CONFIG_MFD_MAX8998 is not set
# CONFIG_MFD_VIPERBOARD is not set
# CONFIG_MFD_RETU is not set
# CONFIG_MFD_PCF50633 is not set
# CONFIG_MFD_PM8921_CORE is not set
# CONFIG_MFD_RTSX_USB is not set
# CONFIG_MFD_RC5T583 is not set
CONFIG_MFD_SEC_CORE=y
# CONFIG_MFD_SI476X_CORE is not set
# CONFIG_MFD_SM501 is not set
# CONFIG_MFD_SMSC is not set
# CONFIG_ABX500_CORE is not set
# CONFIG_MFD_STMPE is not set
# CONFIG_MFD_SYSCON is not set
# CONFIG_MFD_TI_AM335X_TSCADC is not set
# CONFIG_MFD_LP3943 is not set
# CONFIG_MFD_LP8788 is not set
# CONFIG_MFD_PALMAS is not set
# CONFIG_TPS6105X is not set
# CONFIG_TPS65010 is not set
# CONFIG_TPS6507X is not set
CONFIG_MFD_TPS65090=y
# CONFIG_MFD_TPS65217 is not set
# CONFIG_MFD_TPS65218 is not set
# CONFIG_MFD_TPS6586X is not set
# CONFIG_MFD_TPS65910 is not set
# CONFIG_MFD_TPS65912 is not set
# CONFIG_MFD_TPS65912_I2C is not set
# CONFIG_MFD_TPS80031 is not set
# CONFIG_TWL4030_CORE is not set
# CONFIG_TWL6040_CORE is not set
# CONFIG_MFD_WL1273_CORE is not set
# CONFIG_MFD_LM3533 is not set
# CONFIG_MFD_TC3589X is not set
# CONFIG_MFD_TMIO is not set
# CONFIG_MFD_T7L66XB is not set
# CONFIG_MFD_TC6387XB is not set
# CONFIG_MFD_TC6393XB is not set
# CONFIG_MFD_ARIZONA_I2C is not set
# CONFIG_MFD_WM8400 is not set
# CONFIG_MFD_WM831X_I2C is not set
# CONFIG_MFD_WM8350_I2C is not set
# CONFIG_MFD_WM8994 is not set
CONFIG_REGULATOR=y
# CONFIG_REGULATOR_DEBUG is not set
CONFIG_REGULATOR_FIXED_VOLTAGE=y
# CONFIG_REGULATOR_VIRTUAL_CONSUMER is not set
# CONFIG_REGULATOR_USERSPACE_CONSUMER is not set
# CONFIG_REGULATOR_ACT8865 is not set
# CONFIG_REGULATOR_AD5398 is not set
# CONFIG_REGULATOR_DA9210 is not set
# CONFIG_REGULATOR_FAN53555 is not set
CONFIG_REGULATOR_GPIO=y
# CONFIG_REGULATOR_ISL6271A is not set
# CONFIG_REGULATOR_LP3971 is not set
# CONFIG_REGULATOR_LP3972 is not set
# CONFIG_REGULATOR_LP872X is not set
# CONFIG_REGULATOR_LP8755 is not set
# CONFIG_REGULATOR_LTC3589 is not set
# CONFIG_REGULATOR_MAX1586 is not set
# CONFIG_REGULATOR_MAX8649 is not set
# CONFIG_REGULATOR_MAX8660 is not set
# CONFIG_REGULATOR_MAX8952 is not set
# CONFIG_REGULATOR_MAX8973 is not set
CONFIG_REGULATOR_MAX8997=y
CONFIG_REGULATOR_MAX77686=y
# CONFIG_REGULATOR_PFUZE100 is not set
# CONFIG_REGULATOR_S2MPA01 is not set
CONFIG_REGULATOR_S2MPS11=y
CONFIG_REGULATOR_S5M8767=y
# CONFIG_REGULATOR_TPS51632 is not set
# CONFIG_REGULATOR_TPS62360 is not set
# CONFIG_REGULATOR_TPS65023 is not set
# CONFIG_REGULATOR_TPS6507X is not set
CONFIG_REGULATOR_TPS65090=y
# CONFIG_MEDIA_SUPPORT is not set

#
# Graphics support
#

#
# Direct Rendering Manager
#
# CONFIG_DRM is not set

#
# Frame buffer Devices
#
CONFIG_FB=y
# CONFIG_FIRMWARE_EDID is not set
# CONFIG_FB_DDC is not set
# CONFIG_FB_BOOT_VESA_SUPPORT is not set
CONFIG_FB_CFB_FILLRECT=y
CONFIG_FB_CFB_COPYAREA=y
CONFIG_FB_CFB_IMAGEBLIT=y
# CONFIG_FB_CFB_REV_PIXELS_IN_BYTE is not set
# CONFIG_FB_SYS_FILLRECT is not set
# CONFIG_FB_SYS_COPYAREA is not set
# CONFIG_FB_SYS_IMAGEBLIT is not set
# CONFIG_FB_FOREIGN_ENDIAN is not set
# CONFIG_FB_SYS_FOPS is not set
# CONFIG_FB_SVGALIB is not set
# CONFIG_FB_MACMODES is not set
# CONFIG_FB_BACKLIGHT is not set
CONFIG_FB_MODE_HELPERS=y
# CONFIG_FB_TILEBLITTING is not set

#
# Frame buffer hardware drivers
#
# CONFIG_FB_ARMCLCD is not set
# CONFIG_FB_OPENCORES is not set
# CONFIG_FB_S1D13XXX is not set
# CONFIG_FB_S3C is not set
# CONFIG_FB_SMSCUFX is not set
# CONFIG_FB_UDL is not set
# CONFIG_FB_VIRTUAL is not set
# CONFIG_FB_METRONOME is not set
# CONFIG_FB_BROADSHEET is not set
# CONFIG_FB_AUO_K190X is not set
CONFIG_FB_SIMPLE=y
CONFIG_EXYNOS_VIDEO=y
CONFIG_EXYNOS_MIPI_DSI=y
# CONFIG_FB_SSD1307 is not set
# CONFIG_BACKLIGHT_LCD_SUPPORT is not set
# CONFIG_VGASTATE is not set

#
# Console display driver support
#
CONFIG_DUMMY_CONSOLE=y
CONFIG_FRAMEBUFFER_CONSOLE=y
# CONFIG_FRAMEBUFFER_CONSOLE_DETECT_PRIMARY is not set
# CONFIG_FRAMEBUFFER_CONSOLE_ROTATION is not set
CONFIG_LOGO=y
CONFIG_LOGO_LINUX_MONO=y
CONFIG_LOGO_LINUX_VGA16=y
CONFIG_LOGO_LINUX_CLUT224=y
# CONFIG_SOUND is not set

#
# HID support
#
CONFIG_HID=y
# CONFIG_HIDRAW is not set
# CONFIG_UHID is not set
CONFIG_HID_GENERIC=y

#
# Special HID drivers
#
CONFIG_HID_A4TECH=y
# CONFIG_HID_ACRUX is not set
CONFIG_HID_APPLE=y
# CONFIG_HID_APPLEIR is not set
# CONFIG_HID_AUREAL is not set
CONFIG_HID_BELKIN=y
CONFIG_HID_CHERRY=y
CONFIG_HID_CHICONY=y
# CONFIG_HID_CP2112 is not set
CONFIG_HID_CYPRESS=y
# CONFIG_HID_DRAGONRISE is not set
# CONFIG_HID_EMS_FF is not set
# CONFIG_HID_ELECOM is not set
# CONFIG_HID_ELO is not set
CONFIG_HID_EZKEY=y
# CONFIG_HID_HOLTEK is not set
# CONFIG_HID_HUION is not set
# CONFIG_HID_KEYTOUCH is not set
# CONFIG_HID_KYE is not set
# CONFIG_HID_UCLOGIC is not set
# CONFIG_HID_WALTOP is not set
# CONFIG_HID_GYRATION is not set
# CONFIG_HID_ICADE is not set
# CONFIG_HID_TWINHAN is not set
CONFIG_HID_KENSINGTON=y
# CONFIG_HID_LCPOWER is not set
# CONFIG_HID_LENOVO_TPKBD is not set
CONFIG_HID_LOGITECH=y
# CONFIG_LOGITECH_FF is not set
# CONFIG_LOGIRUMBLEPAD2_FF is not set
# CONFIG_LOGIG940_FF is not set
# CONFIG_LOGIWHEELS_FF is not set
# CONFIG_HID_MAGICMOUSE is not set
CONFIG_HID_MICROSOFT=y
CONFIG_HID_MONTEREY=y
# CONFIG_HID_MULTITOUCH is not set
# CONFIG_HID_NTRIG is not set
# CONFIG_HID_ORTEK is not set
# CONFIG_HID_PANTHERLORD is not set
# CONFIG_HID_PETALYNX is not set
# CONFIG_HID_PICOLCD is not set
# CONFIG_HID_PRIMAX is not set
# CONFIG_HID_ROCCAT is not set
# CONFIG_HID_SAITEK is not set
# CONFIG_HID_SAMSUNG is not set
# CONFIG_HID_SPEEDLINK is not set
# CONFIG_HID_STEELSERIES is not set
# CONFIG_HID_SUNPLUS is not set
# CONFIG_HID_RMI is not set
# CONFIG_HID_GREENASIA is not set
# CONFIG_HID_SMARTJOYPLUS is not set
# CONFIG_HID_TIVO is not set
# CONFIG_HID_TOPSEED is not set
# CONFIG_HID_THRUSTMASTER is not set
# CONFIG_HID_XINMO is not set
# CONFIG_HID_ZEROPLUS is not set
# CONFIG_HID_ZYDACRON is not set
# CONFIG_HID_SENSOR_HUB is not set

#
# USB HID support
#
CONFIG_USB_HID=y
# CONFIG_HID_PID is not set
# CONFIG_USB_HIDDEV is not set

#
# I2C HID support
#
# CONFIG_I2C_HID is not set
CONFIG_USB_OHCI_LITTLE_ENDIAN=y
CONFIG_USB_SUPPORT=y
CONFIG_USB_COMMON=y
CONFIG_USB_ARCH_HAS_HCD=y
CONFIG_USB=y
# CONFIG_USB_ANNOUNCE_NEW_DEVICES is not set

#
# Miscellaneous USB options
#
CONFIG_USB_DEFAULT_PERSIST=y
# CONFIG_USB_DYNAMIC_MINORS is not set
# CONFIG_USB_OTG_FSM is not set
# CONFIG_USB_MON is not set
# CONFIG_USB_WUSB_CBAF is not set

#
# USB Host Controller Drivers
#
# CONFIG_USB_C67X00_HCD is not set
# CONFIG_USB_XHCI_HCD is not set
CONFIG_USB_EHCI_HCD=y
# CONFIG_USB_EHCI_ROOT_HUB_TT is not set
CONFIG_USB_EHCI_TT_NEWSCHED=y
CONFIG_USB_EHCI_EXYNOS=y
# CONFIG_USB_EHCI_HCD_PLATFORM is not set
# CONFIG_USB_OXU210HP_HCD is not set
# CONFIG_USB_ISP116X_HCD is not set
# CONFIG_USB_ISP1760_HCD is not set
# CONFIG_USB_ISP1362_HCD is not set
# CONFIG_USB_FUSBH200_HCD is not set
# CONFIG_USB_FOTG210_HCD is not set
# CONFIG_USB_OHCI_HCD is not set
# CONFIG_USB_SL811_HCD is not set
# CONFIG_USB_R8A66597_HCD is not set
# CONFIG_USB_HCD_TEST_MODE is not set

#
# USB Device Class drivers
#
# CONFIG_USB_ACM is not set
# CONFIG_USB_PRINTER is not set
# CONFIG_USB_WDM is not set
# CONFIG_USB_TMC is not set

#
# NOTE: USB_STORAGE depends on SCSI but BLK_DEV_SD may
#

#
# also be needed; see USB_STORAGE Help for more info
#
CONFIG_USB_STORAGE=y
# CONFIG_USB_STORAGE_DEBUG is not set
# CONFIG_USB_STORAGE_REALTEK is not set
# CONFIG_USB_STORAGE_DATAFAB is not set
# CONFIG_USB_STORAGE_FREECOM is not set
# CONFIG_USB_STORAGE_ISD200 is not set
# CONFIG_USB_STORAGE_USBAT is not set
# CONFIG_USB_STORAGE_SDDR09 is not set
# CONFIG_USB_STORAGE_SDDR55 is not set
# CONFIG_USB_STORAGE_JUMPSHOT is not set
# CONFIG_USB_STORAGE_ALAUDA is not set
# CONFIG_USB_STORAGE_ONETOUCH is not set
# CONFIG_USB_STORAGE_KARMA is not set
# CONFIG_USB_STORAGE_CYPRESS_ATACB is not set
# CONFIG_USB_STORAGE_ENE_UB6250 is not set
# CONFIG_USB_UAS is not set

#
# USB Imaging devices
#
# CONFIG_USB_MDC800 is not set
# CONFIG_USB_MICROTEK is not set
# CONFIG_USB_MUSB_HDRC is not set
CONFIG_USB_DWC3=y
CONFIG_USB_DWC3_HOST=y

#
# Platform Glue Driver Support
#
CONFIG_USB_DWC3_EXYNOS=y

#
# Debugging features
#
# CONFIG_USB_DWC3_DEBUG is not set
# CONFIG_USB_DWC2 is not set
# CONFIG_USB_CHIPIDEA is not set

#
# USB port drivers
#
# CONFIG_USB_SERIAL is not set

#
# USB Miscellaneous drivers
#
# CONFIG_USB_EMI62 is not set
# CONFIG_USB_EMI26 is not set
# CONFIG_USB_ADUTUX is not set
# CONFIG_USB_SEVSEG is not set
# CONFIG_USB_RIO500 is not set
# CONFIG_USB_LEGOTOWER is not set
# CONFIG_USB_LCD is not set
# CONFIG_USB_LED is not set
# CONFIG_USB_CYPRESS_CY7C63 is not set
# CONFIG_USB_CYTHERM is not set
# CONFIG_USB_IDMOUSE is not set
# CONFIG_USB_FTDI_ELAN is not set
# CONFIG_USB_APPLEDISPLAY is not set
# CONFIG_USB_SISUSBVGA is not set
# CONFIG_USB_LD is not set
# CONFIG_USB_TRANCEVIBRATOR is not set
# CONFIG_USB_IOWARRIOR is not set
# CONFIG_USB_TEST is not set
# CONFIG_USB_EHSET_TEST_FIXTURE is not set
# CONFIG_USB_ISIGHTFW is not set
# CONFIG_USB_YUREX is not set
# CONFIG_USB_EZUSB_FX2 is not set
# CONFIG_USB_HSIC_USB3503 is not set

#
# USB Physical Layer drivers
#
CONFIG_USB_PHY=y
# CONFIG_NOP_USB_XCEIV is not set
# CONFIG_AM335X_PHY_USB is not set
CONFIG_SAMSUNG_USBPHY=y
CONFIG_SAMSUNG_USB2PHY=y
CONFIG_SAMSUNG_USB3PHY=y
# CONFIG_USB_GPIO_VBUS is not set
# CONFIG_USB_ISP1301 is not set
# CONFIG_USB_ULPI is not set
# CONFIG_USB_GADGET is not set
CONFIG_MMC=y
# CONFIG_MMC_DEBUG is not set
# CONFIG_MMC_CLKGATE is not set

#
# MMC/SD/SDIO Card Drivers
#
CONFIG_MMC_BLOCK=y
CONFIG_MMC_BLOCK_MINORS=8
CONFIG_MMC_BLOCK_BOUNCE=y
# CONFIG_SDIO_UART is not set
# CONFIG_MMC_TEST is not set

#
# MMC/SD/SDIO Host Controller Drivers
#
# CONFIG_MMC_ARMMMCI is not set
CONFIG_MMC_SDHCI=y
# CONFIG_MMC_SDHCI_PLTFM is not set
CONFIG_MMC_SDHCI_S3C=y
# CONFIG_MMC_SDHCI_S3C_DMA is not set
CONFIG_MMC_DW=y
CONFIG_MMC_DW_IDMAC=y
CONFIG_MMC_DW_PLTFM=y
CONFIG_MMC_DW_EXYNOS=y
# CONFIG_MMC_DW_K3 is not set
# CONFIG_MMC_VUB300 is not set
# CONFIG_MMC_USHC is not set
# CONFIG_MMC_USDHI6ROL0 is not set
# CONFIG_MEMSTICK is not set
# CONFIG_NEW_LEDS is not set
# CONFIG_ACCESSIBILITY is not set
# CONFIG_EDAC is not set
CONFIG_RTC_LIB=y
CONFIG_RTC_CLASS=y
CONFIG_RTC_HCTOSYS=y
CONFIG_RTC_SYSTOHC=y
CONFIG_RTC_HCTOSYS_DEVICE="rtc0"
# CONFIG_RTC_DEBUG is not set

#
# RTC interfaces
#
CONFIG_RTC_INTF_SYSFS=y
CONFIG_RTC_INTF_PROC=y
CONFIG_RTC_INTF_DEV=y
# CONFIG_RTC_INTF_DEV_UIE_EMUL is not set
# CONFIG_RTC_DRV_TEST is not set

#
# I2C RTC drivers
#
# CONFIG_RTC_DRV_DS1307 is not set
# CONFIG_RTC_DRV_DS1374 is not set
# CONFIG_RTC_DRV_DS1672 is not set
# CONFIG_RTC_DRV_DS3232 is not set
# CONFIG_RTC_DRV_HYM8563 is not set
# CONFIG_RTC_DRV_MAX6900 is not set
# CONFIG_RTC_DRV_MAX8997 is not set
# CONFIG_RTC_DRV_MAX77686 is not set
# CONFIG_RTC_DRV_RS5C372 is not set
# CONFIG_RTC_DRV_ISL1208 is not set
# CONFIG_RTC_DRV_ISL12022 is not set
# CONFIG_RTC_DRV_ISL12057 is not set
# CONFIG_RTC_DRV_X1205 is not set
# CONFIG_RTC_DRV_PCF2127 is not set
# CONFIG_RTC_DRV_PCF8523 is not set
# CONFIG_RTC_DRV_PCF8563 is not set
# CONFIG_RTC_DRV_PCF8583 is not set
# CONFIG_RTC_DRV_M41T80 is not set
# CONFIG_RTC_DRV_BQ32K is not set
# CONFIG_RTC_DRV_S35390A is not set
# CONFIG_RTC_DRV_FM3130 is not set
# CONFIG_RTC_DRV_RX8581 is not set
# CONFIG_RTC_DRV_RX8025 is not set
# CONFIG_RTC_DRV_EM3027 is not set
# CONFIG_RTC_DRV_RV3029C2 is not set
# CONFIG_RTC_DRV_S5M is not set

#
# SPI RTC drivers
#

#
# Platform RTC drivers
#
# CONFIG_RTC_DRV_CMOS is not set
# CONFIG_RTC_DRV_DS1286 is not set
# CONFIG_RTC_DRV_DS1511 is not set
# CONFIG_RTC_DRV_DS1553 is not set
# CONFIG_RTC_DRV_DS1742 is not set
# CONFIG_RTC_DRV_STK17TA8 is not set
# CONFIG_RTC_DRV_M48T86 is not set
# CONFIG_RTC_DRV_M48T35 is not set
# CONFIG_RTC_DRV_M48T59 is not set
# CONFIG_RTC_DRV_MSM6242 is not set
# CONFIG_RTC_DRV_BQ4802 is not set
# CONFIG_RTC_DRV_RP5C01 is not set
# CONFIG_RTC_DRV_V3020 is not set
# CONFIG_RTC_DRV_DS2404 is not set

#
# on-CPU RTC drivers
#
CONFIG_HAVE_S3C_RTC=y
CONFIG_RTC_DRV_S3C=y
# CONFIG_RTC_DRV_PL030 is not set
# CONFIG_RTC_DRV_PL031 is not set
# CONFIG_RTC_DRV_SNVS is not set
# CONFIG_RTC_DRV_MOXART is not set
# CONFIG_RTC_DRV_XGENE is not set

#
# HID Sensor RTC drivers
#
# CONFIG_RTC_DRV_HID_SENSOR_TIME is not set
# CONFIG_DMADEVICES is not set
# CONFIG_AUXDISPLAY is not set
# CONFIG_UIO is not set
# CONFIG_VIRT_DRIVERS is not set

#
# Virtio drivers
#
# CONFIG_VIRTIO_MMIO is not set

#
# Microsoft Hyper-V guest support
#
# CONFIG_STAGING is not set

#
# SOC (System On Chip) specific Drivers
#
CONFIG_CLKDEV_LOOKUP=y
CONFIG_HAVE_CLK_PREPARE=y
CONFIG_COMMON_CLK=y

#
# Common Clock Framework
#
CONFIG_COMMON_CLK_MAX77686=y
# CONFIG_COMMON_CLK_SI5351 is not set
# CONFIG_COMMON_CLK_SI570 is not set
# CONFIG_COMMON_CLK_S2MPS11 is not set
# CONFIG_COMMON_CLK_QCOM is not set
CONFIG_COMMON_CLK_SAMSUNG=y

#
# Hardware Spinlock drivers
#
CONFIG_CLKSRC_OF=y
CONFIG_ARM_ARCH_TIMER=y
CONFIG_ARM_ARCH_TIMER_EVTSTREAM=y
CONFIG_CLKSRC_EXYNOS_MCT=y
CONFIG_CLKSRC_SAMSUNG_PWM=y
# CONFIG_SH_TIMER_CMT is not set
# CONFIG_SH_TIMER_MTU2 is not set
# CONFIG_SH_TIMER_TMU is not set
# CONFIG_EM_TIMER_STI is not set
# CONFIG_CLKSRC_VERSATILE is not set
# CONFIG_MAILBOX is not set
CONFIG_IOMMU_SUPPORT=y
CONFIG_OF_IOMMU=y
# CONFIG_EXYNOS_IOMMU is not set

#
# Remoteproc drivers
#
# CONFIG_STE_MODEM_RPROC is not set

#
# Rpmsg drivers
#
# CONFIG_PM_DEVFREQ is not set
# CONFIG_EXTCON is not set
# CONFIG_MEMORY is not set
# CONFIG_IIO is not set
# CONFIG_PWM is not set
CONFIG_IRQCHIP=y
CONFIG_ARM_GIC=y
CONFIG_GIC_NON_BANKED=y
# CONFIG_IPACK_BUS is not set
# CONFIG_RESET_CONTROLLER is not set
# CONFIG_FMC is not set

#
# PHY Subsystem
#
CONFIG_GENERIC_PHY=y
CONFIG_PHY_EXYNOS_MIPI_VIDEO=y
CONFIG_PHY_EXYNOS_DP_VIDEO=y
# CONFIG_BCM_KONA_USB2_PHY is not set
# CONFIG_PHY_EXYNOS5250_SATA is not set
# CONFIG_PHY_SAMSUNG_USB2 is not set
# CONFIG_PHY_EXYNOS5_USBDRD is not set
# CONFIG_POWERCAP is not set
# CONFIG_MCB is not set

#
# File systems
#
CONFIG_DCACHE_WORD_ACCESS=y
CONFIG_EXT2_FS=y
# CONFIG_EXT2_FS_XATTR is not set
# CONFIG_EXT2_FS_XIP is not set
CONFIG_EXT3_FS=y
CONFIG_EXT3_DEFAULTS_TO_ORDERED=y
CONFIG_EXT3_FS_XATTR=y
# CONFIG_EXT3_FS_POSIX_ACL is not set
# CONFIG_EXT3_FS_SECURITY is not set
CONFIG_EXT4_FS=y
# CONFIG_EXT4_FS_POSIX_ACL is not set
# CONFIG_EXT4_FS_SECURITY is not set
# CONFIG_EXT4_DEBUG is not set
CONFIG_JBD=y
CONFIG_JBD2=y
# CONFIG_JBD2_DEBUG is not set
CONFIG_FS_MBCACHE=y
# CONFIG_REISERFS_FS is not set
# CONFIG_JFS_FS is not set
# CONFIG_XFS_FS is not set
# CONFIG_GFS2_FS is not set
# CONFIG_BTRFS_FS is not set
# CONFIG_NILFS2_FS is not set
CONFIG_FS_POSIX_ACL=y
CONFIG_FILE_LOCKING=y
CONFIG_FSNOTIFY=y
CONFIG_DNOTIFY=y
CONFIG_INOTIFY_USER=y
# CONFIG_FANOTIFY is not set
# CONFIG_QUOTA is not set
# CONFIG_QUOTACTL is not set
# CONFIG_AUTOFS4_FS is not set
# CONFIG_FUSE_FS is not set

#
# Caches
#
# CONFIG_FSCACHE is not set

#
# CD-ROM/DVD Filesystems
#
# CONFIG_ISO9660_FS is not set
# CONFIG_UDF_FS is not set

#
# DOS/FAT/NT Filesystems
#
CONFIG_FAT_FS=y
CONFIG_MSDOS_FS=y
CONFIG_VFAT_FS=y
CONFIG_FAT_DEFAULT_CODEPAGE=437
CONFIG_FAT_DEFAULT_IOCHARSET="iso8859-1"
# CONFIG_NTFS_FS is not set

#
# Pseudo filesystems
#
CONFIG_PROC_FS=y
CONFIG_PROC_SYSCTL=y
CONFIG_PROC_PAGE_MONITOR=y
CONFIG_KERNFS=y
CONFIG_SYSFS=y
CONFIG_TMPFS=y
CONFIG_TMPFS_POSIX_ACL=y
CONFIG_TMPFS_XATTR=y
# CONFIG_HUGETLB_PAGE is not set
# CONFIG_CONFIGFS_FS is not set
CONFIG_MISC_FILESYSTEMS=y
# CONFIG_ADFS_FS is not set
# CONFIG_AFFS_FS is not set
# CONFIG_HFS_FS is not set
# CONFIG_HFSPLUS_FS is not set
# CONFIG_BEFS_FS is not set
# CONFIG_BFS_FS is not set
# CONFIG_EFS_FS is not set
# CONFIG_LOGFS is not set
CONFIG_CRAMFS=y
# CONFIG_SQUASHFS is not set
# CONFIG_VXFS_FS is not set
# CONFIG_MINIX_FS is not set
# CONFIG_OMFS_FS is not set
# CONFIG_HPFS_FS is not set
# CONFIG_QNX4FS_FS is not set
# CONFIG_QNX6FS_FS is not set
CONFIG_ROMFS_FS=y
CONFIG_ROMFS_BACKED_BY_BLOCK=y
CONFIG_ROMFS_ON_BLOCK=y
# CONFIG_PSTORE is not set
# CONFIG_SYSV_FS is not set
# CONFIG_UFS_FS is not set
# CONFIG_F2FS_FS is not set
CONFIG_NETWORK_FILESYSTEMS=y
# CONFIG_NFS_FS is not set
# CONFIG_NFSD is not set
# CONFIG_CEPH_FS is not set
# CONFIG_CIFS is not set
# CONFIG_NCP_FS is not set
# CONFIG_CODA_FS is not set
# CONFIG_AFS_FS is not set
CONFIG_NLS=y
CONFIG_NLS_DEFAULT="iso8859-1"
CONFIG_NLS_CODEPAGE_437=y
# CONFIG_NLS_CODEPAGE_737 is not set
# CONFIG_NLS_CODEPAGE_775 is not set
# CONFIG_NLS_CODEPAGE_850 is not set
# CONFIG_NLS_CODEPAGE_852 is not set
# CONFIG_NLS_CODEPAGE_855 is not set
# CONFIG_NLS_CODEPAGE_857 is not set
# CONFIG_NLS_CODEPAGE_860 is not set
# CONFIG_NLS_CODEPAGE_861 is not set
# CONFIG_NLS_CODEPAGE_862 is not set
# CONFIG_NLS_CODEPAGE_863 is not set
# CONFIG_NLS_CODEPAGE_864 is not set
# CONFIG_NLS_CODEPAGE_865 is not set
# CONFIG_NLS_CODEPAGE_866 is not set
# CONFIG_NLS_CODEPAGE_869 is not set
# CONFIG_NLS_CODEPAGE_936 is not set
# CONFIG_NLS_CODEPAGE_950 is not set
# CONFIG_NLS_CODEPAGE_932 is not set
# CONFIG_NLS_CODEPAGE_949 is not set
# CONFIG_NLS_CODEPAGE_874 is not set
# CONFIG_NLS_ISO8859_8 is not set
# CONFIG_NLS_CODEPAGE_1250 is not set
# CONFIG_NLS_CODEPAGE_1251 is not set
CONFIG_NLS_ASCII=y
CONFIG_NLS_ISO8859_1=y
# CONFIG_NLS_ISO8859_2 is not set
# CONFIG_NLS_ISO8859_3 is not set
# CONFIG_NLS_ISO8859_4 is not set
# CONFIG_NLS_ISO8859_5 is not set
# CONFIG_NLS_ISO8859_6 is not set
# CONFIG_NLS_ISO8859_7 is not set
# CONFIG_NLS_ISO8859_9 is not set
# CONFIG_NLS_ISO8859_13 is not set
# CONFIG_NLS_ISO8859_14 is not set
# CONFIG_NLS_ISO8859_15 is not set
# CONFIG_NLS_KOI8_R is not set
# CONFIG_NLS_KOI8_U is not set
# CONFIG_NLS_MAC_ROMAN is not set
# CONFIG_NLS_MAC_CELTIC is not set
# CONFIG_NLS_MAC_CENTEURO is not set
# CONFIG_NLS_MAC_CROATIAN is not set
# CONFIG_NLS_MAC_CYRILLIC is not set
# CONFIG_NLS_MAC_GAELIC is not set
# CONFIG_NLS_MAC_GREEK is not set
# CONFIG_NLS_MAC_ICELAND is not set
# CONFIG_NLS_MAC_INUIT is not set
# CONFIG_NLS_MAC_ROMANIAN is not set
# CONFIG_NLS_MAC_TURKISH is not set
# CONFIG_NLS_UTF8 is not set

#
# Kernel hacking
#

#
# printk and dmesg options
#
CONFIG_PRINTK_TIME=y
CONFIG_DEFAULT_MESSAGE_LOGLEVEL=4
# CONFIG_BOOT_PRINTK_DELAY is not set

#
# Compile-time checks and compiler options
#
CONFIG_DEBUG_INFO=y
# CONFIG_DEBUG_INFO_REDUCED is not set
CONFIG_ENABLE_WARN_DEPRECATED=y
CONFIG_ENABLE_MUST_CHECK=y
CONFIG_FRAME_WARN=1024
# CONFIG_STRIP_ASM_SYMS is not set
# CONFIG_READABLE_ASM is not set
# CONFIG_UNUSED_SYMBOLS is not set
# CONFIG_DEBUG_FS is not set
# CONFIG_HEADERS_CHECK is not set
# CONFIG_DEBUG_SECTION_MISMATCH is not set
# CONFIG_DEBUG_FORCE_WEAK_PER_CPU is not set
CONFIG_MAGIC_SYSRQ=y
CONFIG_MAGIC_SYSRQ_DEFAULT_ENABLE=0x1
CONFIG_DEBUG_KERNEL=y

#
# Memory Debugging
#
# CONFIG_DEBUG_PAGEALLOC is not set
# CONFIG_DEBUG_OBJECTS is not set
# CONFIG_SLUB_DEBUG_ON is not set
# CONFIG_SLUB_STATS is not set
CONFIG_HAVE_DEBUG_KMEMLEAK=y
# CONFIG_DEBUG_KMEMLEAK is not set
# CONFIG_DEBUG_STACK_USAGE is not set
# CONFIG_DEBUG_VM is not set
CONFIG_DEBUG_MEMORY_INIT=y
# CONFIG_DEBUG_PER_CPU_MAPS is not set
# CONFIG_DEBUG_HIGHMEM is not set
# CONFIG_DEBUG_SHIRQ is not set

#
# Debug Lockups and Hangs
#
CONFIG_LOCKUP_DETECTOR=y
CONFIG_BOOTPARAM_SOFTLOCKUP_PANIC=y
CONFIG_BOOTPARAM_SOFTLOCKUP_PANIC_VALUE=1
CONFIG_DETECT_HUNG_TASK=y
CONFIG_DEFAULT_HUNG_TASK_TIMEOUT=120
# CONFIG_BOOTPARAM_HUNG_TASK_PANIC is not set
CONFIG_BOOTPARAM_HUNG_TASK_PANIC_VALUE=0
# CONFIG_PANIC_ON_OOPS is not set
CONFIG_PANIC_ON_OOPS_VALUE=0
CONFIG_PANIC_TIMEOUT=0
CONFIG_SCHED_DEBUG=y
# CONFIG_SCHEDSTATS is not set
# CONFIG_TIMER_STATS is not set
CONFIG_DEBUG_PREEMPT=y

#
# Lock Debugging (spinlocks, mutexes, etc...)
#
CONFIG_DEBUG_RT_MUTEXES=y
# CONFIG_RT_MUTEX_TESTER is not set
CONFIG_DEBUG_SPINLOCK=y
CONFIG_DEBUG_MUTEXES=y
# CONFIG_DEBUG_WW_MUTEX_SLOWPATH is not set
CONFIG_DEBUG_LOCK_ALLOC=y
CONFIG_PROVE_LOCKING=y
CONFIG_LOCKDEP=y
# CONFIG_LOCK_STAT is not set
# CONFIG_DEBUG_LOCKDEP is not set
CONFIG_DEBUG_ATOMIC_SLEEP=y
# CONFIG_DEBUG_LOCKING_API_SELFTESTS is not set
# CONFIG_LOCK_TORTURE_TEST is not set
CONFIG_TRACE_IRQFLAGS=y
CONFIG_STACKTRACE=y
# CONFIG_DEBUG_KOBJECT is not set
CONFIG_DEBUG_BUGVERBOSE=y
# CONFIG_DEBUG_LIST is not set
# CONFIG_DEBUG_PI_LIST is not set
# CONFIG_DEBUG_SG is not set
# CONFIG_DEBUG_NOTIFIERS is not set
# CONFIG_DEBUG_CREDENTIALS is not set

#
# RCU Debugging
#
# CONFIG_PROVE_RCU is not set
# CONFIG_PROVE_RCU_DELAY is not set
# CONFIG_SPARSE_RCU_POINTER is not set
# CONFIG_TORTURE_TEST is not set
# CONFIG_RCU_TORTURE_TEST is not set
CONFIG_RCU_CPU_STALL_TIMEOUT=21
CONFIG_RCU_CPU_STALL_VERBOSE=y
# CONFIG_RCU_CPU_STALL_INFO is not set
# CONFIG_RCU_TRACE is not set
# CONFIG_DEBUG_BLOCK_EXT_DEVT is not set
# CONFIG_NOTIFIER_ERROR_INJECTION is not set
# CONFIG_FAULT_INJECTION is not set
CONFIG_HAVE_FUNCTION_TRACER=y
CONFIG_HAVE_FUNCTION_GRAPH_TRACER=y
CONFIG_HAVE_DYNAMIC_FTRACE=y
CONFIG_HAVE_FTRACE_MCOUNT_RECORD=y
CONFIG_HAVE_SYSCALL_TRACEPOINTS=y
CONFIG_HAVE_C_RECORDMCOUNT=y
CONFIG_TRACING_SUPPORT=y
CONFIG_FTRACE=y
# CONFIG_FUNCTION_TRACER is not set
# CONFIG_IRQSOFF_TRACER is not set
# CONFIG_PREEMPT_TRACER is not set
# CONFIG_SCHED_TRACER is not set
# CONFIG_ENABLE_DEFAULT_TRACERS is not set
# CONFIG_FTRACE_SYSCALLS is not set
# CONFIG_TRACER_SNAPSHOT is not set
CONFIG_BRANCH_PROFILE_NONE=y
# CONFIG_PROFILE_ANNOTATED_BRANCHES is not set
# CONFIG_PROFILE_ALL_BRANCHES is not set
# CONFIG_STACK_TRACER is not set
# CONFIG_BLK_DEV_IO_TRACE is not set
# CONFIG_PROBE_EVENTS is not set
# CONFIG_TRACEPOINT_BENCHMARK is not set

#
# Runtime Testing
#
# CONFIG_TEST_LIST_SORT is not set
# CONFIG_BACKTRACE_SELF_TEST is not set
# CONFIG_RBTREE_TEST is not set
# CONFIG_INTERVAL_TREE_TEST is not set
# CONFIG_PERCPU_TEST is not set
# CONFIG_ATOMIC64_SELFTEST is not set
# CONFIG_TEST_STRING_HELPERS is not set
# CONFIG_TEST_KSTRTOX is not set
# CONFIG_DMA_API_DEBUG is not set
# CONFIG_TEST_MODULE is not set
# CONFIG_TEST_USER_COPY is not set
# CONFIG_TEST_BPF is not set
# CONFIG_SAMPLES is not set
CONFIG_HAVE_ARCH_KGDB=y
# CONFIG_KGDB is not set
# CONFIG_ARM_PTDUMP is not set
# CONFIG_STRICT_DEVMEM is not set
CONFIG_ARM_UNWIND=y
CONFIG_DEBUG_USER=y
# CONFIG_DEBUG_LL is not set
CONFIG_DEBUG_LL_INCLUDE="mach/debug-macro.S"
# CONFIG_DEBUG_UART_PL01X is not set
# CONFIG_DEBUG_UART_8250 is not set
CONFIG_UNCOMPRESS_INCLUDE="debug/uncompress.h"
# CONFIG_OC_ETM is not set
# CONFIG_PID_IN_CONTEXTIDR is not set
# CONFIG_DEBUG_SET_MODULE_RONX is not set

#
# Security options
#
# CONFIG_KEYS is not set
# CONFIG_SECURITY_DMESG_RESTRICT is not set
# CONFIG_SECURITY is not set
CONFIG_SECURITYFS=y
CONFIG_DEFAULT_SECURITY_DAC=y
CONFIG_DEFAULT_SECURITY=""
CONFIG_CRYPTO=y

#
# Crypto core or helper
#
CONFIG_CRYPTO_ALGAPI=y
CONFIG_CRYPTO_ALGAPI2=y
CONFIG_CRYPTO_AEAD2=y
CONFIG_CRYPTO_BLKCIPHER=y
CONFIG_CRYPTO_BLKCIPHER2=y
CONFIG_CRYPTO_HASH=y
CONFIG_CRYPTO_HASH2=y
CONFIG_CRYPTO_RNG=m
CONFIG_CRYPTO_RNG2=y
CONFIG_CRYPTO_PCOMP2=y
CONFIG_CRYPTO_MANAGER=y
CONFIG_CRYPTO_MANAGER2=y
# CONFIG_CRYPTO_USER is not set
CONFIG_CRYPTO_MANAGER_DISABLE_TESTS=y
# CONFIG_CRYPTO_GF128MUL is not set
# CONFIG_CRYPTO_NULL is not set
# CONFIG_CRYPTO_PCRYPT is not set
CONFIG_CRYPTO_WORKQUEUE=y
# CONFIG_CRYPTO_CRYPTD is not set
# CONFIG_CRYPTO_AUTHENC is not set
# CONFIG_CRYPTO_TEST is not set

#
# Authenticated Encryption with Associated Data
#
# CONFIG_CRYPTO_CCM is not set
# CONFIG_CRYPTO_GCM is not set
# CONFIG_CRYPTO_SEQIV is not set

#
# Block modes
#
CONFIG_CRYPTO_CBC=y
# CONFIG_CRYPTO_CTR is not set
# CONFIG_CRYPTO_CTS is not set
# CONFIG_CRYPTO_ECB is not set
# CONFIG_CRYPTO_LRW is not set
# CONFIG_CRYPTO_PCBC is not set
# CONFIG_CRYPTO_XTS is not set

#
# Hash modes
#
# CONFIG_CRYPTO_CMAC is not set
# CONFIG_CRYPTO_HMAC is not set
# CONFIG_CRYPTO_XCBC is not set
# CONFIG_CRYPTO_VMAC is not set

#
# Digest
#
CONFIG_CRYPTO_CRC32C=y
# CONFIG_CRYPTO_CRC32 is not set
# CONFIG_CRYPTO_CRCT10DIF is not set
# CONFIG_CRYPTO_GHASH is not set
# CONFIG_CRYPTO_MD4 is not set
# CONFIG_CRYPTO_MD5 is not set
# CONFIG_CRYPTO_MICHAEL_MIC is not set
# CONFIG_CRYPTO_RMD128 is not set
# CONFIG_CRYPTO_RMD160 is not set
# CONFIG_CRYPTO_RMD256 is not set
# CONFIG_CRYPTO_RMD320 is not set
# CONFIG_CRYPTO_SHA1 is not set
# CONFIG_CRYPTO_SHA1_ARM is not set
CONFIG_CRYPTO_SHA256=y
# CONFIG_CRYPTO_SHA512 is not set
# CONFIG_CRYPTO_TGR192 is not set
# CONFIG_CRYPTO_WP512 is not set

#
# Ciphers
#
CONFIG_CRYPTO_AES=y
# CONFIG_CRYPTO_AES_ARM is not set
# CONFIG_CRYPTO_ANUBIS is not set
# CONFIG_CRYPTO_ARC4 is not set
# CONFIG_CRYPTO_BLOWFISH is not set
# CONFIG_CRYPTO_CAMELLIA is not set
# CONFIG_CRYPTO_CAST5 is not set
# CONFIG_CRYPTO_CAST6 is not set
# CONFIG_CRYPTO_DES is not set
# CONFIG_CRYPTO_FCRYPT is not set
# CONFIG_CRYPTO_KHAZAD is not set
# CONFIG_CRYPTO_SALSA20 is not set
# CONFIG_CRYPTO_SEED is not set
# CONFIG_CRYPTO_SERPENT is not set
# CONFIG_CRYPTO_TEA is not set
# CONFIG_CRYPTO_TWOFISH is not set

#
# Compression
#
# CONFIG_CRYPTO_DEFLATE is not set
# CONFIG_CRYPTO_ZLIB is not set
# CONFIG_CRYPTO_LZO is not set
# CONFIG_CRYPTO_LZ4 is not set
# CONFIG_CRYPTO_LZ4HC is not set

#
# Random Number Generation
#
CONFIG_CRYPTO_ANSI_CPRNG=m
# CONFIG_CRYPTO_USER_API_HASH is not set
# CONFIG_CRYPTO_USER_API_SKCIPHER is not set
CONFIG_CRYPTO_HW=y
# CONFIG_CRYPTO_DEV_S5P is not set
# CONFIG_BINARY_PRINTF is not set

#
# Library routines
#
CONFIG_BITREVERSE=y
CONFIG_GENERIC_STRNCPY_FROM_USER=y
CONFIG_GENERIC_STRNLEN_USER=y
CONFIG_GENERIC_NET_UTILS=y
CONFIG_GENERIC_PCI_IOMAP=y
CONFIG_GENERIC_IO=y
CONFIG_ARCH_USE_CMPXCHG_LOCKREF=y
CONFIG_CRC_CCITT=y
CONFIG_CRC16=y
# CONFIG_CRC_T10DIF is not set
# CONFIG_CRC_ITU_T is not set
CONFIG_CRC32=y
# CONFIG_CRC32_SELFTEST is not set
CONFIG_CRC32_SLICEBY8=y
# CONFIG_CRC32_SLICEBY4 is not set
# CONFIG_CRC32_SARWATE is not set
# CONFIG_CRC32_BIT is not set
# CONFIG_CRC7 is not set
# CONFIG_LIBCRC32C is not set
# CONFIG_CRC8 is not set
# CONFIG_AUDIT_ARCH_COMPAT_GENERIC is not set
# CONFIG_RANDOM32_SELFTEST is not set
CONFIG_ZLIB_INFLATE=y
CONFIG_LZO_COMPRESS=y
CONFIG_LZO_DECOMPRESS=y
CONFIG_LZ4_DECOMPRESS=y
CONFIG_XZ_DEC=y
CONFIG_XZ_DEC_X86=y
CONFIG_XZ_DEC_POWERPC=y
CONFIG_XZ_DEC_IA64=y
CONFIG_XZ_DEC_ARM=y
CONFIG_XZ_DEC_ARMTHUMB=y
CONFIG_XZ_DEC_SPARC=y
CONFIG_XZ_DEC_BCJ=y
# CONFIG_XZ_DEC_TEST is not set
CONFIG_DECOMPRESS_GZIP=y
CONFIG_DECOMPRESS_BZIP2=y
CONFIG_DECOMPRESS_LZMA=y
CONFIG_DECOMPRESS_XZ=y
CONFIG_DECOMPRESS_LZO=y
CONFIG_DECOMPRESS_LZ4=y
CONFIG_GENERIC_ALLOCATOR=y
CONFIG_HAS_IOMEM=y
CONFIG_HAS_DMA=y
CONFIG_CPU_RMAP=y
CONFIG_DQL=y
CONFIG_NLATTR=y
CONFIG_ARCH_HAS_ATOMIC64_DEC_IF_POSITIVE=y
# CONFIG_AVERAGE is not set
# CONFIG_CORDIC is not set
# CONFIG_DDR is not set
CONFIG_LIBFDT=y
CONFIG_FONT_SUPPORT=y
CONFIG_FONTS=y
# CONFIG_FONT_8x8 is not set
# CONFIG_FONT_8x16 is not set
# CONFIG_FONT_6x11 is not set
CONFIG_FONT_7x14=y
# CONFIG_FONT_PEARL_8x8 is not set
# CONFIG_FONT_ACORN_8x8 is not set
# CONFIG_FONT_MINI_4x6 is not set
# CONFIG_FONT_SUN8x16 is not set
# CONFIG_FONT_SUN12x22 is not set
# CONFIG_FONT_10x18 is not set
# CONFIG_VIRTUALIZATION is not set

Re: [PATCH] cpufreq, store_scaling_governor requires policy->rwsem to be held for duration of changing governors [v2]

[Prarit Bhargava]

On 08/07/2014 02:36 AM, Viresh Kumar wrote:
> On 6 August 2014 20:38, Stephen Boyd <sboyd@codeaurora.org> wrote:
>> On 08/06, Prarit Bhargava wrote:
>>> Are you sure you're not seeing another lockdep warning?  That was my problem --
>>> there was an xfs related lockdep warning which then resulted in lockdep being
>>> disabled from that point on.
> 
> There is a fair chance that I might be doing something really really stupid,
> but I couldn't get the lockdep warning..
> 
>> Are we talking about the lockdep splat or the crash that started
>> this thread or something else? For the lockdep splat you need the
>> corrected patch in this thread and the per policy governor flag.
>> I'm not sure how to recreate the crash that started this thread.
> 
> We are talking about the lockdep splat that would happen if we don't
> drop locking around EXIT..
> 
> This is my full diff over mainline and my .config is attached.
> Please enlighten me on what am I missing :)

That should have done it.  What are your CPUFREQ configs?

P.

> 
> diff --git a/drivers/cpufreq/cpufreq.c b/drivers/cpufreq/cpufreq.c
> index 6f02485..fa11a7d 100644
> --- a/drivers/cpufreq/cpufreq.c
> +++ b/drivers/cpufreq/cpufreq.c
> @@ -2200,9 +2200,7 @@ static int cpufreq_set_policy(struct
> cpufreq_policy *policy,
>         /* end old governor */
>         if (old_gov) {
>                 __cpufreq_governor(policy, CPUFREQ_GOV_STOP);
> -               up_write(&policy->rwsem);
>                 __cpufreq_governor(policy, CPUFREQ_GOV_POLICY_EXIT);
> -               down_write(&policy->rwsem);
>         }
> 
>         /* start new governor */
> @@ -2211,9 +2209,7 @@ static int cpufreq_set_policy(struct
> cpufreq_policy *policy,
>                 if (!__cpufreq_governor(policy, CPUFREQ_GOV_START))
>                         goto out;
> 
> -               up_write(&policy->rwsem);
>                 __cpufreq_governor(policy, CPUFREQ_GOV_POLICY_EXIT);
> -               down_write(&policy->rwsem);
>         }
> 
>         /* new governor failed, so re-start old one */
> diff --git a/drivers/cpufreq/exynos-cpufreq.c b/drivers/cpufreq/exynos-cpufreq.c
> index 1e0ec57..027b6f7 100644
> --- a/drivers/cpufreq/exynos-cpufreq.c
> +++ b/drivers/cpufreq/exynos-cpufreq.c
> @@ -139,7 +139,7 @@ static int exynos_cpufreq_cpu_init(struct
> cpufreq_policy *policy)
>  }
> 
>  static struct cpufreq_driver exynos_driver = {
> -       .flags          = CPUFREQ_STICKY | CPUFREQ_NEED_INITIAL_FREQ_CHECK,
> +       .flags          = CPUFREQ_STICKY |
> CPUFREQ_NEED_INITIAL_FREQ_CHECK | CPUFREQ_HAVE_GOVERNOR_PER_POLICY,
>         .verify         = cpufreq_generic_frequency_table_verify,
>         .target_index   = exynos_target,
>         .get            = cpufreq_generic_get,
> 

Re: [PATCH] cpufreq, store_scaling_governor requires policy->rwsem to be held for duration of changing governors [v2]

[Viresh Kumar]

On 7 August 2014 15:42, Prarit Bhargava <prarit@redhat.com> wrote:
> That should have done it.  What are your CPUFREQ configs?

You can check the same .config I attached last time for that :)

CONFIG_CPU_FREQ=y
CONFIG_CPU_FREQ_GOV_COMMON=y
CONFIG_CPU_FREQ_STAT=y
CONFIG_CPU_FREQ_STAT_DETAILS=y
# CONFIG_CPU_FREQ_DEFAULT_GOV_PERFORMANCE is not set
# CONFIG_CPU_FREQ_DEFAULT_GOV_USERSPACE is not set
# CONFIG_CPU_FREQ_DEFAULT_GOV_ONDEMAND is not set
CONFIG_CPU_FREQ_DEFAULT_GOV_CONSERVATIVE=y
CONFIG_CPU_FREQ_GOV_PERFORMANCE=y
CONFIG_CPU_FREQ_GOV_POWERSAVE=y
CONFIG_CPU_FREQ_GOV_USERSPACE=y
CONFIG_CPU_FREQ_GOV_ONDEMAND=y
CONFIG_CPU_FREQ_GOV_CONSERVATIVE=y


Anyway, has anybody tried to test what I have been trying now?
@Prarit: You can try that on your x86 box as well, which has a
single cluster or group of CPUs sharing clock line.

Re: [PATCH] cpufreq, store_scaling_governor requires policy->rwsem to be held for duration of changing governors [v2]

[Viresh Kumar]

On 7 August 2014 15:45, Viresh Kumar <viresh.kumar@linaro.org> wrote:
> On 7 August 2014 15:42, Prarit Bhargava <prarit@redhat.com> wrote:
>> That should have done it.  What are your CPUFREQ configs?
>
> You can check the same .config I attached last time for that :)
>
> CONFIG_CPU_FREQ=y
> CONFIG_CPU_FREQ_GOV_COMMON=y
> CONFIG_CPU_FREQ_STAT=y
> CONFIG_CPU_FREQ_STAT_DETAILS=y
> # CONFIG_CPU_FREQ_DEFAULT_GOV_PERFORMANCE is not set
> # CONFIG_CPU_FREQ_DEFAULT_GOV_USERSPACE is not set
> # CONFIG_CPU_FREQ_DEFAULT_GOV_ONDEMAND is not set
> CONFIG_CPU_FREQ_DEFAULT_GOV_CONSERVATIVE=y
> CONFIG_CPU_FREQ_GOV_PERFORMANCE=y
> CONFIG_CPU_FREQ_GOV_POWERSAVE=y
> CONFIG_CPU_FREQ_GOV_USERSPACE=y
> CONFIG_CPU_FREQ_GOV_ONDEMAND=y
> CONFIG_CPU_FREQ_GOV_CONSERVATIVE=y
>
>
> Anyway, has anybody tried to test what I have been trying now?
> @Prarit: You can try that on your x86 box as well, which has a
> single cluster or group of CPUs sharing clock line.

Ping!!

Re: [PATCH] cpufreq, store_scaling_governor requires policy->rwsem to be held for duration of changing governors [v2]

[Prarit Bhargava]

On 08/12/2014 05:03 AM, Viresh Kumar wrote:
> On 7 August 2014 15:45, Viresh Kumar <viresh.kumar@linaro.org> wrote:
>> On 7 August 2014 15:42, Prarit Bhargava <prarit@redhat.com> wrote:
>>> That should have done it.  What are your CPUFREQ configs?
>>
>> You can check the same .config I attached last time for that :)
>>
>> CONFIG_CPU_FREQ=y
>> CONFIG_CPU_FREQ_GOV_COMMON=y
>> CONFIG_CPU_FREQ_STAT=y
>> CONFIG_CPU_FREQ_STAT_DETAILS=y
>> # CONFIG_CPU_FREQ_DEFAULT_GOV_PERFORMANCE is not set
>> # CONFIG_CPU_FREQ_DEFAULT_GOV_USERSPACE is not set
>> # CONFIG_CPU_FREQ_DEFAULT_GOV_ONDEMAND is not set
>> CONFIG_CPU_FREQ_DEFAULT_GOV_CONSERVATIVE=y
>> CONFIG_CPU_FREQ_GOV_PERFORMANCE=y
>> CONFIG_CPU_FREQ_GOV_POWERSAVE=y
>> CONFIG_CPU_FREQ_GOV_USERSPACE=y
>> CONFIG_CPU_FREQ_GOV_ONDEMAND=y
>> CONFIG_CPU_FREQ_GOV_CONSERVATIVE=y

Viresh, sorry for my late reply -- I got distracted by another issue.

>>
>>
>> Anyway, has anybody tried to test what I have been trying now?
>> @Prarit: You can try that on your x86 box as well, which has a
>> single cluster or group of CPUs sharing clock line.
> 

Okay, this is what I have and I can reproduce this *easily* 100% of the time.

I've used your above config options and have enabled LOCKDEP.

In order to restore the locking, I've applied the following patch to the cpufreq
core (sorry for the cut-and-paste):

diff --git a/drivers/cpufreq/cpufreq.c b/drivers/cpufreq/cpufreq.c
index d9fdedd..dfda238 100644
--- a/drivers/cpufreq/cpufreq.c
+++ b/drivers/cpufreq/cpufreq.c
@@ -2192,9 +2192,7 @@ static int cpufreq_set_policy(struct cpufreq_policy *polic
        /* end old governor */
        if (old_gov) {
                __cpufreq_governor(policy, CPUFREQ_GOV_STOP);
-               up_write(&policy->rwsem);
                __cpufreq_governor(policy, CPUFREQ_GOV_POLICY_EXIT);
-               down_write(&policy->rwsem);
        }

        /* start new governor */
@@ -2203,9 +2201,7 @@ static int cpufreq_set_policy(struct cpufreq_policy *polic
                if (!__cpufreq_governor(policy, CPUFREQ_GOV_START))
                        goto out;

-               up_write(&policy->rwsem);
                __cpufreq_governor(policy, CPUFREQ_GOV_POLICY_EXIT);
-               down_write(&policy->rwsem);
        }

        /* new governor failed, so re-start old one */


I've modified the acpi-cpufreq driver to include (sorry for the cut-and-paste)

diff --git a/drivers/cpufreq/acpi-cpufreq.c b/drivers/cpufreq/acpi-cpufreq.c
index b0c18ed..97653c3 100644
--- a/drivers/cpufreq/acpi-cpufreq.c
+++ b/drivers/cpufreq/acpi-cpufreq.c
@@ -884,6 +884,9 @@ static struct freq_attr *acpi_cpufreq_attr[] = {
 };

 static struct cpufreq_driver acpi_cpufreq_driver = {
+       .flags                  = CPUFREQ_STICKY |
+                                       CPUFREQ_HAVE_GOVERNOR_PER_POLICY |
+                                       CPUFREQ_NEED_INITIAL_FREQ_CHECK,
        .verify         = cpufreq_generic_frequency_table_verify,
        .target_index   = acpi_cpufreq_target,
        .bios_limit     = acpi_processor_get_bios_limit,

I do a

cat /sys/devices/system/cpu/cpu2/cpufreq/conservative/*
echo ondemand > /sys/devices/system/cpu/cpu2/cpufreq/scaling_governor

and then I immediately see the stack trace.

P.


> Ping!!
> 

Re: [PATCH] cpufreq, store_scaling_governor requires policy->rwsem to be held for duration of changing governors [v2]

[Viresh Kumar]

On 12 August 2014 17:03, Prarit Bhargava <prarit@redhat.com> wrote:
> Okay, this is what I have and I can reproduce this *easily* 100% of the time.
>
> I've used your above config options and have enabled LOCKDEP.
>
> In order to restore the locking, I've applied the following patch to the cpufreq
> core (sorry for the cut-and-paste):
>
> diff --git a/drivers/cpufreq/cpufreq.c b/drivers/cpufreq/cpufreq.c
> index d9fdedd..dfda238 100644
> --- a/drivers/cpufreq/cpufreq.c
> +++ b/drivers/cpufreq/cpufreq.c
> @@ -2192,9 +2192,7 @@ static int cpufreq_set_policy(struct cpufreq_policy *polic
>         /* end old governor */
>         if (old_gov) {
>                 __cpufreq_governor(policy, CPUFREQ_GOV_STOP);
> -               up_write(&policy->rwsem);
>                 __cpufreq_governor(policy, CPUFREQ_GOV_POLICY_EXIT);
> -               down_write(&policy->rwsem);
>         }
>
>         /* start new governor */
> @@ -2203,9 +2201,7 @@ static int cpufreq_set_policy(struct cpufreq_policy *polic
>                 if (!__cpufreq_governor(policy, CPUFREQ_GOV_START))
>                         goto out;
>
> -               up_write(&policy->rwsem);
>                 __cpufreq_governor(policy, CPUFREQ_GOV_POLICY_EXIT);
> -               down_write(&policy->rwsem);
>         }
>
>         /* new governor failed, so re-start old one */
>
>
> I've modified the acpi-cpufreq driver to include (sorry for the cut-and-paste)
>
> diff --git a/drivers/cpufreq/acpi-cpufreq.c b/drivers/cpufreq/acpi-cpufreq.c
> index b0c18ed..97653c3 100644
> --- a/drivers/cpufreq/acpi-cpufreq.c
> +++ b/drivers/cpufreq/acpi-cpufreq.c
> @@ -884,6 +884,9 @@ static struct freq_attr *acpi_cpufreq_attr[] = {
>  };
>
>  static struct cpufreq_driver acpi_cpufreq_driver = {
> +       .flags                  = CPUFREQ_STICKY |
> +                                       CPUFREQ_HAVE_GOVERNOR_PER_POLICY |
> +                                       CPUFREQ_NEED_INITIAL_FREQ_CHECK,
>         .verify         = cpufreq_generic_frequency_table_verify,
>         .target_index   = acpi_cpufreq_target,
>         .bios_limit     = acpi_processor_get_bios_limit,
>
> I do a
>
> cat /sys/devices/system/cpu/cpu2/cpufreq/conservative/*
> echo ondemand > /sys/devices/system/cpu/cpu2/cpufreq/scaling_governor
>
> and then I immediately see the stack trace.

What's your system configuration? How many clusters/cpus/etc..

Re: [PATCH] cpufreq, store_scaling_governor requires policy->rwsem to be held for duration of changing governors [v2]

[Prarit Bhargava]

On 08/13/2014 03:39 AM, Viresh Kumar wrote:
> On 12 August 2014 17:03, Prarit Bhargava <prarit@redhat.com> wrote:
>> Okay, this is what I have and I can reproduce this *easily* 100% of the time.
>>
>> I've used your above config options and have enabled LOCKDEP.
>>
>> In order to restore the locking, I've applied the following patch to the cpufreq
>> core (sorry for the cut-and-paste):
>>
>> diff --git a/drivers/cpufreq/cpufreq.c b/drivers/cpufreq/cpufreq.c
>> index d9fdedd..dfda238 100644
>> --- a/drivers/cpufreq/cpufreq.c
>> +++ b/drivers/cpufreq/cpufreq.c
>> @@ -2192,9 +2192,7 @@ static int cpufreq_set_policy(struct cpufreq_policy *polic
>>         /* end old governor */
>>         if (old_gov) {
>>                 __cpufreq_governor(policy, CPUFREQ_GOV_STOP);
>> -               up_write(&policy->rwsem);
>>                 __cpufreq_governor(policy, CPUFREQ_GOV_POLICY_EXIT);
>> -               down_write(&policy->rwsem);
>>         }
>>
>>         /* start new governor */
>> @@ -2203,9 +2201,7 @@ static int cpufreq_set_policy(struct cpufreq_policy *polic
>>                 if (!__cpufreq_governor(policy, CPUFREQ_GOV_START))
>>                         goto out;
>>
>> -               up_write(&policy->rwsem);
>>                 __cpufreq_governor(policy, CPUFREQ_GOV_POLICY_EXIT);
>> -               down_write(&policy->rwsem);
>>         }
>>
>>         /* new governor failed, so re-start old one */
>>
>>
>> I've modified the acpi-cpufreq driver to include (sorry for the cut-and-paste)
>>
>> diff --git a/drivers/cpufreq/acpi-cpufreq.c b/drivers/cpufreq/acpi-cpufreq.c
>> index b0c18ed..97653c3 100644
>> --- a/drivers/cpufreq/acpi-cpufreq.c
>> +++ b/drivers/cpufreq/acpi-cpufreq.c
>> @@ -884,6 +884,9 @@ static struct freq_attr *acpi_cpufreq_attr[] = {
>>  };
>>
>>  static struct cpufreq_driver acpi_cpufreq_driver = {
>> +       .flags                  = CPUFREQ_STICKY |
>> +                                       CPUFREQ_HAVE_GOVERNOR_PER_POLICY |
>> +                                       CPUFREQ_NEED_INITIAL_FREQ_CHECK,
>>         .verify         = cpufreq_generic_frequency_table_verify,
>>         .target_index   = acpi_cpufreq_target,
>>         .bios_limit     = acpi_processor_get_bios_limit,
>>
>> I do a
>>
>> cat /sys/devices/system/cpu/cpu2/cpufreq/conservative/*
>> echo ondemand > /sys/devices/system/cpu/cpu2/cpufreq/scaling_governor
>>
>> and then I immediately see the stack trace.
> 
> What's your system configuration? How many clusters/cpus/etc..

Anywhere from 2-4 sockets, 8 - 240 cpus (depending on # of sockets), x86 arch.

P.
> 

Re: [PATCH] cpufreq, store_scaling_governor requires policy->rwsem to be held for duration of changing governors [v2]

[Prarit Bhargava]

On 08/13/2014 03:39 AM, Viresh Kumar wrote:
> On 12 August 2014 17:03, Prarit Bhargava <prarit@redhat.com> wrote:
>> Okay, this is what I have and I can reproduce this *easily* 100% of the time.
>>
>> I've used your above config options and have enabled LOCKDEP.
>>
>> In order to restore the locking, I've applied the following patch to the cpufreq
>> core (sorry for the cut-and-paste):
>>
>> diff --git a/drivers/cpufreq/cpufreq.c b/drivers/cpufreq/cpufreq.c
>> index d9fdedd..dfda238 100644
>> --- a/drivers/cpufreq/cpufreq.c
>> +++ b/drivers/cpufreq/cpufreq.c
>> @@ -2192,9 +2192,7 @@ static int cpufreq_set_policy(struct cpufreq_policy *polic
>>         /* end old governor */
>>         if (old_gov) {
>>                 __cpufreq_governor(policy, CPUFREQ_GOV_STOP);
>> -               up_write(&policy->rwsem);
>>                 __cpufreq_governor(policy, CPUFREQ_GOV_POLICY_EXIT);
>> -               down_write(&policy->rwsem);
>>         }
>>
>>         /* start new governor */
>> @@ -2203,9 +2201,7 @@ static int cpufreq_set_policy(struct cpufreq_policy *polic
>>                 if (!__cpufreq_governor(policy, CPUFREQ_GOV_START))
>>                         goto out;
>>
>> -               up_write(&policy->rwsem);
>>                 __cpufreq_governor(policy, CPUFREQ_GOV_POLICY_EXIT);
>> -               down_write(&policy->rwsem);
>>         }
>>
>>         /* new governor failed, so re-start old one */
>>
>>
>> I've modified the acpi-cpufreq driver to include (sorry for the cut-and-paste)
>>
>> diff --git a/drivers/cpufreq/acpi-cpufreq.c b/drivers/cpufreq/acpi-cpufreq.c
>> index b0c18ed..97653c3 100644
>> --- a/drivers/cpufreq/acpi-cpufreq.c
>> +++ b/drivers/cpufreq/acpi-cpufreq.c
>> @@ -884,6 +884,9 @@ static struct freq_attr *acpi_cpufreq_attr[] = {
>>  };
>>
>>  static struct cpufreq_driver acpi_cpufreq_driver = {
>> +       .flags                  = CPUFREQ_STICKY |
>> +                                       CPUFREQ_HAVE_GOVERNOR_PER_POLICY |
>> +                                       CPUFREQ_NEED_INITIAL_FREQ_CHECK,
>>         .verify         = cpufreq_generic_frequency_table_verify,
>>         .target_index   = acpi_cpufreq_target,
>>         .bios_limit     = acpi_processor_get_bios_limit,
>>
>> I do a
>>
>> cat /sys/devices/system/cpu/cpu2/cpufreq/conservative/*
>> echo ondemand > /sys/devices/system/cpu/cpu2/cpufreq/scaling_governor
>>
>> and then I immediately see the stack trace.
> 
> What's your system configuration? How many clusters/cpus/etc..

Anywhere from 2-4 sockets, 8 - 240 cpus (depending on # of sockets), x86 arch.

P.
> 

Re: [PATCH] cpufreq, store_scaling_governor requires policy->rwsem to be held for duration of changing governors [v2]

[Prarit Bhargava]

On 08/05/2014 06:51 PM, Saravana Kannan wrote:

> 
> I definitely have a fix for this and the original race you reported. It's
> basically reverting that commit you reverted + a fix for the deadlock. That's
> the only way to fix the scaling_governor issue.
> 
> You fix the deadlock by moving the governor attribute group removing to the
> framework code and doing it before STOP+EXIT to governor without holding the
> policy lock. And the reverse for INIT+STOP.
> 

I'm still not convinced of the deadlock so I did a bit of additional research
and am pretty close to saying that this is a false positive from the lockdep
code in the kernfs area.

A few things that have caused me concern about the lockdep splat we're seeing:

1.  The splat occurs when we hit __kernfs_remove+0x25b/0x360 which resolves to

        if (kernfs_lockdep(kn)) {
                rwsem_acquire(&kn->dep_map, 0, 0, _RET_IP_);  <<< RIGHT HERE
                if (atomic_read(&kn->active) != KN_DEACTIVATED_BIAS)
                        lock_contended(&kn->dep_map, _RET_IP_);
        }

ie) the *ONLY* way we hit a "deadlock" in this code is if we have LOCKDEP
configured in the kernfs.

It should be noted, that having kernfs_lockdep() always return 0 [1], results in
NO additional lockdep warnings.

Additionally the splat contains

[  107.428421]        CPU0                    CPU1
[  107.433482]        ----                    ----
[  107.438544]   lock(&policy->rwsem);
[  107.442459]                                lock(s_active#98);
[  107.448916]                                lock(&policy->rwsem);
[  107.455650]   lock(s_active#98);

which also points to the situation above (s_active is the default naming used in
the kernfs lockdep code).

In short -- there is no deadlock here.  It only happens in the lockdep code
itself, not because lockdep has identified a real problem.

2.  I then started asking myself why this was occurring.  The reason appears to
be that the attribute for scaling_governor is deleting other sysfs attributes
and that got me to wondering if there were other areas where this occurred and I
remembered it does!  In the USB code writing and reading to the  bConfiguration
of a device may lead to the removal of "down stream" attributes.  The reading
and writing of bConfiguration occurs in
drivers/usb/core/sysfs.c:79


/* configuration value is always present, and r/w */
usb_actconfig_show(bConfigurationValue, "%u\n");

static ssize_t bConfigurationValue_store(struct device *dev,
                                         struct device_attribute *attr,
                                         const char *buf, size_t count)
{
        struct usb_device       *udev = to_usb_device(dev);
        int                     config, value;

        if (sscanf(buf, "%d", &config) != 1 || config < -1 || config > 255)
                return -EINVAL;
        usb_lock_device(udev);
        value = usb_set_configuration(udev, config);
        usb_unlock_device(udev);
        return (value < 0) ? value : count;
}

... and the next lines are IMO important here:

static DEVICE_ATTR_IGNORE_LOCKDEP(bConfigurationValue, S_IRUGO | S_IWUSR,
                bConfigurationValue_show, bConfigurationValue_store);

FWIW, it isn't *exactly* the same ... but commit
356c05d58af05d582e634b54b40050c73609617b explains a similarity between what is
happening with our lockdep splat and the lockdep issues seen in USB.

3.  I came across this from an earlier discussion ...

https://lkml.org/lkml/2010/1/29/306

"We get false positives when the code of a sysfs attribute
synchronously removes other sysfs attributes.  In general that is not
safe due to hotplug etc, but there are specific instances of static
sysfs entries like the pm_core where it appears to be safe."

...


So ... the question that I have is: is this lockdep splat "real"?  It seems to
only occur because we enable the lockdep code in kernfs, that is it occurs as a
side-effect and doesn't appear to be "real" to me.

I only offer this in an effort to keep work to a minimum ;)

P.

[1] It wasn't that simple.  There are some other changes that have to be made.
But you get the idea ...

Re: [PATCH] cpufreq, store_scaling_governor requires policy->rwsem to be held for duration of changing governors [v2]

[Prarit Bhargava]

On 08/05/2014 06:51 PM, Saravana Kannan wrote:

> 
> I definitely have a fix for this and the original race you reported. It's
> basically reverting that commit you reverted + a fix for the deadlock. That's
> the only way to fix the scaling_governor issue.
> 
> You fix the deadlock by moving the governor attribute group removing to the
> framework code and doing it before STOP+EXIT to governor without holding the
> policy lock. And the reverse for INIT+STOP.
> 

I'm still not convinced of the deadlock so I did a bit of additional research
and am pretty close to saying that this is a false positive from the lockdep
code in the kernfs area.

A few things that have caused me concern about the lockdep splat we're seeing:

1.  The splat occurs when we hit __kernfs_remove+0x25b/0x360 which resolves to

        if (kernfs_lockdep(kn)) {
                rwsem_acquire(&kn->dep_map, 0, 0, _RET_IP_);  <<< RIGHT HERE
                if (atomic_read(&kn->active) != KN_DEACTIVATED_BIAS)
                        lock_contended(&kn->dep_map, _RET_IP_);
        }

ie) the *ONLY* way we hit a "deadlock" in this code is if we have LOCKDEP
configured in the kernfs.

It should be noted, that having kernfs_lockdep() always return 0 [1], results in
NO additional lockdep warnings.

Additionally the splat contains

[  107.428421]        CPU0                    CPU1
[  107.433482]        ----                    ----
[  107.438544]   lock(&policy->rwsem);
[  107.442459]                                lock(s_active#98);
[  107.448916]                                lock(&policy->rwsem);
[  107.455650]   lock(s_active#98);

which also points to the situation above (s_active is the default naming used in
the kernfs lockdep code).

In short -- there is no deadlock here.  It only happens in the lockdep code
itself, not because lockdep has identified a real problem.

2.  I then started asking myself why this was occurring.  The reason appears to
be that the attribute for scaling_governor is deleting other sysfs attributes
and that got me to wondering if there were other areas where this occurred and I
remembered it does!  In the USB code writing and reading to the  bConfiguration
of a device may lead to the removal of "down stream" attributes.  The reading
and writing of bConfiguration occurs in
drivers/usb/core/sysfs.c:79


/* configuration value is always present, and r/w */
usb_actconfig_show(bConfigurationValue, "%u\n");

static ssize_t bConfigurationValue_store(struct device *dev,
                                         struct device_attribute *attr,
                                         const char *buf, size_t count)
{
        struct usb_device       *udev = to_usb_device(dev);
        int                     config, value;

        if (sscanf(buf, "%d", &config) != 1 || config < -1 || config > 255)
                return -EINVAL;
        usb_lock_device(udev);
        value = usb_set_configuration(udev, config);
        usb_unlock_device(udev);
        return (value < 0) ? value : count;
}

... and the next lines are IMO important here:

static DEVICE_ATTR_IGNORE_LOCKDEP(bConfigurationValue, S_IRUGO | S_IWUSR,
                bConfigurationValue_show, bConfigurationValue_store);

FWIW, it isn't *exactly* the same ... but commit
356c05d58af05d582e634b54b40050c73609617b explains a similarity between what is
happening with our lockdep splat and the lockdep issues seen in USB.

3.  I came across this from an earlier discussion ...

https://lkml.org/lkml/2010/1/29/306

"We get false positives when the code of a sysfs attribute
synchronously removes other sysfs attributes.  In general that is not
safe due to hotplug etc, but there are specific instances of static
sysfs entries like the pm_core where it appears to be safe."

...


So ... the question that I have is: is this lockdep splat "real"?  It seems to
only occur because we enable the lockdep code in kernfs, that is it occurs as a
side-effect and doesn't appear to be "real" to me.

I only offer this in an effort to keep work to a minimum ;)

P.

[1] It wasn't that simple.  There are some other changes that have to be made.
But you get the idea ...

Re: [PATCH] cpufreq, store_scaling_governor requires policy->rwsem to be held for duration of changing governors [v2]

[Viresh Kumar]

On 13 August 2014 15:28, Prarit Bhargava <prarit@redhat.com> wrote:
> Anywhere from 2-4 sockets, 8 - 240 cpus (depending on # of sockets), x86 arch.

That's what. We know that it does happen on multi cluster systems
and I was reproducing it on a single cluster one. i.e. all CPUs share
clock line.

Re: [PATCH] cpufreq, store_scaling_governor requires policy->rwsem to be held for duration of changing governors [v2]

[Saravana Kannan]

On 08/13/2014 12:57 PM, Prarit Bhargava wrote:
>
>
> On 08/05/2014 06:51 PM, Saravana Kannan wrote:
>
>>
>> I definitely have a fix for this and the original race you reported. It's
>> basically reverting that commit you reverted + a fix for the deadlock. That's
>> the only way to fix the scaling_governor issue.
>>
>> You fix the deadlock by moving the governor attribute group removing to the
>> framework code and doing it before STOP+EXIT to governor without holding the
>> policy lock. And the reverse for INIT+STOP.
>>
>
> I'm still not convinced of the deadlock so I did a bit of additional research
> and am pretty close to saying that this is a false positive from the lockdep
> code in the kernfs area.
>
> A few things that have caused me concern about the lockdep splat we're seeing:
>
> 1.  The splat occurs when we hit __kernfs_remove+0x25b/0x360 which resolves to
>
>          if (kernfs_lockdep(kn)) {
>                  rwsem_acquire(&kn->dep_map, 0, 0, _RET_IP_);  <<< RIGHT HERE
>                  if (atomic_read(&kn->active) != KN_DEACTIVATED_BIAS)
>                          lock_contended(&kn->dep_map, _RET_IP_);
>          }
>
> ie) the *ONLY* way we hit a "deadlock" in this code is if we have LOCKDEP
> configured in the kernfs.
>
> It should be noted, that having kernfs_lockdep() always return 0 [1], results in
> NO additional lockdep warnings.
>
> Additionally the splat contains
>
> [  107.428421]        CPU0                    CPU1
> [  107.433482]        ----                    ----
> [  107.438544]   lock(&policy->rwsem);
> [  107.442459]                                lock(s_active#98);
> [  107.448916]                                lock(&policy->rwsem);
> [  107.455650]   lock(s_active#98);
>
> which also points to the situation above (s_active is the default naming used in
> the kernfs lockdep code).
>
> In short -- there is no deadlock here.  It only happens in the lockdep code
> itself, not because lockdep has identified a real problem.
>
> 2.  I then started asking myself why this was occurring.  The reason appears to
> be that the attribute for scaling_governor is deleting other sysfs attributes
> and that got me to wondering if there were other areas where this occurred and I
> remembered it does!  In the USB code writing and reading to the  bConfiguration
> of a device may lead to the removal of "down stream" attributes.  The reading
> and writing of bConfiguration occurs in
> drivers/usb/core/sysfs.c:79
>
>
> /* configuration value is always present, and r/w */
> usb_actconfig_show(bConfigurationValue, "%u\n");
>
> static ssize_t bConfigurationValue_store(struct device *dev,
>                                           struct device_attribute *attr,
>                                           const char *buf, size_t count)
> {
>          struct usb_device       *udev = to_usb_device(dev);
>          int                     config, value;
>
>          if (sscanf(buf, "%d", &config) != 1 || config < -1 || config > 255)
>                  return -EINVAL;
>          usb_lock_device(udev);
>          value = usb_set_configuration(udev, config);
>          usb_unlock_device(udev);
>          return (value < 0) ? value : count;
> }
>
> ... and the next lines are IMO important here:
>
> static DEVICE_ATTR_IGNORE_LOCKDEP(bConfigurationValue, S_IRUGO | S_IWUSR,
>                  bConfigurationValue_show, bConfigurationValue_store);
>
> FWIW, it isn't *exactly* the same ... but commit
> 356c05d58af05d582e634b54b40050c73609617b explains a similarity between what is
> happening with our lockdep splat and the lockdep issues seen in USB.

This seems VERY different from our situation. I don't see an equivalent 
of a policy lock that's grabbed from both threads, but in opposite order.

If I'm not mistaken, the sysfs entry here uses some wait/complete pair 
to wait for something. But that's an equivalent of a semaphore with max 
count of 1. Lockdep just seems to be making it obvious by adding 
semaphore calls.

So, a semaphore equivalent deadlock with another semaphore. I believe 
this is a read deadlock.

-Saravana


-- 
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
hosted by The Linux Foundation

Re: [PATCH] cpufreq, store_scaling_governor requires policy->rwsem to be held for duration of changing governors [v2]

[Saravana Kannan]

On 08/13/2014 12:57 PM, Prarit Bhargava wrote:
>
>
> On 08/05/2014 06:51 PM, Saravana Kannan wrote:
>
>>
>> I definitely have a fix for this and the original race you reported. It's
>> basically reverting that commit you reverted + a fix for the deadlock. That's
>> the only way to fix the scaling_governor issue.
>>
>> You fix the deadlock by moving the governor attribute group removing to the
>> framework code and doing it before STOP+EXIT to governor without holding the
>> policy lock. And the reverse for INIT+STOP.
>>
>
> I'm still not convinced of the deadlock so I did a bit of additional research
> and am pretty close to saying that this is a false positive from the lockdep
> code in the kernfs area.
>
> A few things that have caused me concern about the lockdep splat we're seeing:
>
> 1.  The splat occurs when we hit __kernfs_remove+0x25b/0x360 which resolves to
>
>          if (kernfs_lockdep(kn)) {
>                  rwsem_acquire(&kn->dep_map, 0, 0, _RET_IP_);  <<< RIGHT HERE
>                  if (atomic_read(&kn->active) != KN_DEACTIVATED_BIAS)
>                          lock_contended(&kn->dep_map, _RET_IP_);
>          }
>
> ie) the *ONLY* way we hit a "deadlock" in this code is if we have LOCKDEP
> configured in the kernfs.
>
> It should be noted, that having kernfs_lockdep() always return 0 [1], results in
> NO additional lockdep warnings.
>
> Additionally the splat contains
>
> [  107.428421]        CPU0                    CPU1
> [  107.433482]        ----                    ----
> [  107.438544]   lock(&policy->rwsem);
> [  107.442459]                                lock(s_active#98);
> [  107.448916]                                lock(&policy->rwsem);
> [  107.455650]   lock(s_active#98);
>
> which also points to the situation above (s_active is the default naming used in
> the kernfs lockdep code).
>
> In short -- there is no deadlock here.  It only happens in the lockdep code
> itself, not because lockdep has identified a real problem.
>
> 2.  I then started asking myself why this was occurring.  The reason appears to
> be that the attribute for scaling_governor is deleting other sysfs attributes
> and that got me to wondering if there were other areas where this occurred and I
> remembered it does!  In the USB code writing and reading to the  bConfiguration
> of a device may lead to the removal of "down stream" attributes.  The reading
> and writing of bConfiguration occurs in
> drivers/usb/core/sysfs.c:79
>
>
> /* configuration value is always present, and r/w */
> usb_actconfig_show(bConfigurationValue, "%u\n");
>
> static ssize_t bConfigurationValue_store(struct device *dev,
>                                           struct device_attribute *attr,
>                                           const char *buf, size_t count)
> {
>          struct usb_device       *udev = to_usb_device(dev);
>          int                     config, value;
>
>          if (sscanf(buf, "%d", &config) != 1 || config < -1 || config > 255)
>                  return -EINVAL;
>          usb_lock_device(udev);
>          value = usb_set_configuration(udev, config);
>          usb_unlock_device(udev);
>          return (value < 0) ? value : count;
> }
>
> ... and the next lines are IMO important here:
>
> static DEVICE_ATTR_IGNORE_LOCKDEP(bConfigurationValue, S_IRUGO | S_IWUSR,
>                  bConfigurationValue_show, bConfigurationValue_store);
>
> FWIW, it isn't *exactly* the same ... but commit
> 356c05d58af05d582e634b54b40050c73609617b explains a similarity between what is
> happening with our lockdep splat and the lockdep issues seen in USB.

This seems VERY different from our situation. I don't see an equivalent 
of a policy lock that's grabbed from both threads, but in opposite order.

If I'm not mistaken, the sysfs entry here uses some wait/complete pair 
to wait for something. But that's an equivalent of a semaphore with max 
count of 1. Lockdep just seems to be making it obvious by adding 
semaphore calls.

So, a semaphore equivalent deadlock with another semaphore. I believe 
this is a read deadlock.

-Saravana


-- 
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
hosted by The Linux Foundation

Re: [PATCH] cpufreq, store_scaling_governor requires policy->rwsem to be held for duration of changing governors [v2]

[Viresh Kumar]

On 1 August 2014 22:48, Stephen Boyd <sboyd@codeaurora.org> wrote:
> On 08/01/14 03:27, Prarit Bhargava wrote:
>>
>> Can you send me the test and the trace of the deadlock?  I'm not creating it with:
>>
>
> This was with conservative as the default, and switching to ondemand
>
> # cd /sys/devices/system/cpu/cpu2/cpufreq
> # ls
> affected_cpus                  scaling_available_governors
> conservative                   scaling_cur_freq
> cpuinfo_cur_freq               scaling_driver
> cpuinfo_max_freq               scaling_governor
> cpuinfo_min_freq               scaling_max_freq
> cpuinfo_transition_latency     scaling_min_freq
> related_cpus                   scaling_setspeed
> scaling_available_frequencies  stats
> # cat conservative/down_threshold
> 20
> # echo ondemand > scaling_governor
>
>  ======================================================
>  [ INFO: possible circular locking dependency detected ]
>  3.16.0-rc3-00039-ge1e38f124d87 #47 Not tainted
>  -------------------------------------------------------
>  sh/75 is trying to acquire lock:
>   (s_active#9){++++..}, at: [<c0358a94>] kernfs_remove_by_name_ns+0x3c/0x84

Can you please retry this on mainline? I wasn't able to reproduce it
now over 3.17.
I am trying this on Exynos b.L implementation..

Re: [PATCH] cpufreq, store_scaling_governor requires policy->rwsem to be held for duration of changing governors [v2]

[Prarit Bhargava]

On 10/13/2014 06:43 AM, Viresh Kumar wrote:
> On 1 August 2014 22:48, Stephen Boyd <sboyd@codeaurora.org> wrote:
>> On 08/01/14 03:27, Prarit Bhargava wrote:
>>>
>>> Can you send me the test and the trace of the deadlock?  I'm not creating it with:
>>>
>>
>> This was with conservative as the default, and switching to ondemand
>>
>> # cd /sys/devices/system/cpu/cpu2/cpufreq
>> # ls
>> affected_cpus                  scaling_available_governors
>> conservative                   scaling_cur_freq
>> cpuinfo_cur_freq               scaling_driver
>> cpuinfo_max_freq               scaling_governor
>> cpuinfo_min_freq               scaling_max_freq
>> cpuinfo_transition_latency     scaling_min_freq
>> related_cpus                   scaling_setspeed
>> scaling_available_frequencies  stats
>> # cat conservative/down_threshold
>> 20
>> # echo ondemand > scaling_governor
>>
>>  ======================================================
>>  [ INFO: possible circular locking dependency detected ]
>>  3.16.0-rc3-00039-ge1e38f124d87 #47 Not tainted
>>  -------------------------------------------------------
>>  sh/75 is trying to acquire lock:
>>   (s_active#9){++++..}, at: [<c0358a94>] kernfs_remove_by_name_ns+0x3c/0x84
> 
> Can you please retry this on mainline? I wasn't able to reproduce it
> now over 3.17.
> I am trying this on Exynos b.L implementation..

I have 100% reproducibility on latest mainline.

Viresh, please see my next post on the locking issues in cpufreq.

P.

>