All of lore.kernel.org
 help / color / mirror / Atom feed
* [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-14 20:42 ` Thomas Gleixner
  0 siblings, 0 replies; 282+ messages in thread
From: Thomas Gleixner @ 2020-09-14 20:42 UTC (permalink / raw)
  To: LKML
  Cc: linux-arch, Linus Torvalds, Sebastian Andrzej Siewior,
	Valentin Schneider, Richard Henderson, Ivan Kokshaysky,
	Matt Turner, linux-alpha, Jeff Dike, Richard Weinberger,
	Anton Ivanov, linux-um, Brian Cain, linux-hexagon,
	Geert Uytterhoeven, linux-m68k, Ingo Molnar, Peter Zijlstra,
	Juri Lelli, Vincent Guittot, Dietmar Eggemann, Steven Rostedt,
	Ben Segall, Mel Gorman, Daniel Bristot de Oliveira, Will Deacon,
	Andrew Morton, linux-mm, Ingo Molnar, Russell King,
	linux-arm-kernel, Chris Zankel, Max Filippov, linux-xtensa,
	Jani Nikula, Joonas Lahtinen, Rodrigo Vivi, David Airlie,
	Daniel Vetter, intel-gfx, dri-devel, Paul E. McKenney,
	Josh Triplett, Mathieu Desnoyers, Lai Jiangshan, Shuah Khan, rcu,
	linux-kselftest

Folks!

While working on various preempt count related things, I stumbled (again)
over the inconsistency of our preempt count handling.

The handling of preempt_count() is inconsistent accross kernel
configurations. On kernels which have PREEMPT_COUNT=n
preempt_disable/enable() and the lock/unlock functions are not affecting
the preempt count, only local_bh_disable/enable() and _bh variants of
locking, soft interrupt delivery, hard interrupt and NMI context affect it.

It's therefore impossible to have a consistent set of checks which provide
information about the context in which a function is called. In many cases
it makes sense to have seperate functions for seperate contexts, but there
are valid reasons to avoid that and handle different calling contexts
conditionally.

The lack of such indicators which work on all kernel configuratios is a
constant source of trouble because developers either do not understand the
implications or try to work around this inconsistency in weird
ways. Neither seem these issues be catched by reviewers and testing.

Recently merged code does:

	 gfp = preemptible() ? GFP_KERNEL : GFP_ATOMIC;

Looks obviously correct, except for the fact that preemptible() is
unconditionally false for CONFIF_PREEMPT_COUNT=n, i.e. all allocations in
that code use GFP_ATOMIC on such kernels.

Attempts to make preempt count unconditional and consistent have been
rejected in the past with handwaving performance arguments.

Freshly conducted benchmarks did not reveal any measurable impact from
enabling preempt count unconditionally. On kernels with CONFIG_PREEMPT_NONE
or CONFIG_PREEMPT_VOLUNTARY the preempt count is only incremented and
decremented but the result of the decrement is not tested. Contrary to that
enabling CONFIG_PREEMPT which tests the result has a small but measurable
impact due to the conditional branch/call.

It's about time to make essential functionality of the kernel consistent
accross the various preemption models.

The series is also available from git:

   git://git.kernel.org/pub/scm/linux/kernel/git/tglx/devel.git preempt

That's the first part of a larger effort related to preempt count:

 1) The analysis of the usage sites of in_interrupt(), in_atomic(),
    in_softirq() is still ongoing, but so far the number of buggy users is
    clearly the vast majority. There will be seperate patch series
    (currently 46 and counting) to address these issues once the analysis
    is complete in the next days.

 2) The long discussed state tracking of local irq disable in preempt count
    which accounts interrupt disabled sections as atomic and avoids issuing
    costly instructions (sti, cli, popf or their non X86 counterparts) when
    the state does not change, i.e. nested irq_save() or irq_restore(). I
    have this working on X86 already and contrary to my earlier attempts
    this was reasonably straight forward due to the recent entry/exit code
    consolidation.

    What I've not done yet is to optimize the preempt count handling
    of the [un]lock_irq* operations so they handle the interrupt disabled
    state and the preempt count modification in one go. That's an obvious
    add on, but correctness first ...

 3) Lazy interrupt disabling as a straight forward extension to #2. This
    avoids the actual disabling at the CPU level completely and catches an
    incoming interrupt in the low level entry code, modifies the interrupt
    disabled state on the return stack, notes the interrupt as pending in
    software and raises it again when interrupts are reenabled. This has
    still a few issues which I'm hunting down (cpuidle is unhappy ...)

Thanks,

	tglx
---
 arch/arm/include/asm/assembler.h                                 |   11 --
 arch/arm/kernel/iwmmxt.S                                         |    2 
 arch/arm/mach-ep93xx/crunch-bits.S                               |    2 
 arch/xtensa/kernel/entry.S                                       |    2 
 drivers/gpu/drm/i915/Kconfig.debug                               |    1 
 drivers/gpu/drm/i915/i915_utils.h                                |    3 
 include/linux/bit_spinlock.h                                     |    4 -
 include/linux/lockdep.h                                          |    6 -
 include/linux/pagemap.h                                          |    4 -
 include/linux/preempt.h                                          |   37 +---------
 include/linux/uaccess.h                                          |    6 -
 kernel/Kconfig.preempt                                           |    4 -
 kernel/sched/core.c                                              |    6 -
 lib/Kconfig.debug                                                |    3 
 lib/Kconfig.debug.rej                                            |   14 +--
 tools/testing/selftests/rcutorture/configs/rcu/SRCU-t            |    1 
 tools/testing/selftests/rcutorture/configs/rcu/SRCU-u            |    1 
 tools/testing/selftests/rcutorture/configs/rcu/TINY01            |    1 
 tools/testing/selftests/rcutorture/doc/TINY_RCU.txt              |    5 -
 tools/testing/selftests/rcutorture/doc/TREE_RCU-kconfig.txt      |    1 
 tools/testing/selftests/rcutorture/formal/srcu-cbmc/src/config.h |    1 
 21 files changed, 23 insertions(+), 92 deletions(-)



^ permalink raw reply	[flat|nested] 282+ messages in thread

* [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-14 20:42 ` Thomas Gleixner
  0 siblings, 0 replies; 282+ messages in thread
From: Thomas Gleixner @ 2020-09-14 20:42 UTC (permalink / raw)
  To: LKML
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Joonas Lahtinen, Lai Jiangshan, dri-devel, Ben Segall, linux-mm,
	linux-kselftest, linux-hexagon, Will Deacon, Ingo Molnar,
	Anton Ivanov, linux-arch, Vincent Guittot, Brian Cain,
	Richard Weinberger, Russell King, David Airlie, Ingo Molnar,
	Geert Uytterhoeven, Mel Gorman, intel-gfx, Matt Turner,
	Valentin Schneider, linux-xtensa, Shuah Khan, Paul E. McKenney,
	Jeff Dike, linux-um, Josh Triplett, Steven Rostedt, rcu,
	linux-m68k, Ivan Kokshaysky, Jani Nikula, Rodrigo Vivi,
	Dietmar Eggemann, linux-arm-kernel, Richard Henderson,
	Chris Zankel, Max Filippov, Linus Torvalds, Daniel Vetter,
	linux-alpha, Mathieu Desnoyers, Andrew Morton,
	Daniel Bristot de Oliveira

Folks!

While working on various preempt count related things, I stumbled (again)
over the inconsistency of our preempt count handling.

The handling of preempt_count() is inconsistent accross kernel
configurations. On kernels which have PREEMPT_COUNT=n
preempt_disable/enable() and the lock/unlock functions are not affecting
the preempt count, only local_bh_disable/enable() and _bh variants of
locking, soft interrupt delivery, hard interrupt and NMI context affect it.

It's therefore impossible to have a consistent set of checks which provide
information about the context in which a function is called. In many cases
it makes sense to have seperate functions for seperate contexts, but there
are valid reasons to avoid that and handle different calling contexts
conditionally.

The lack of such indicators which work on all kernel configuratios is a
constant source of trouble because developers either do not understand the
implications or try to work around this inconsistency in weird
ways. Neither seem these issues be catched by reviewers and testing.

Recently merged code does:

	 gfp = preemptible() ? GFP_KERNEL : GFP_ATOMIC;

Looks obviously correct, except for the fact that preemptible() is
unconditionally false for CONFIF_PREEMPT_COUNT=n, i.e. all allocations in
that code use GFP_ATOMIC on such kernels.

Attempts to make preempt count unconditional and consistent have been
rejected in the past with handwaving performance arguments.

Freshly conducted benchmarks did not reveal any measurable impact from
enabling preempt count unconditionally. On kernels with CONFIG_PREEMPT_NONE
or CONFIG_PREEMPT_VOLUNTARY the preempt count is only incremented and
decremented but the result of the decrement is not tested. Contrary to that
enabling CONFIG_PREEMPT which tests the result has a small but measurable
impact due to the conditional branch/call.

It's about time to make essential functionality of the kernel consistent
accross the various preemption models.

The series is also available from git:

   git://git.kernel.org/pub/scm/linux/kernel/git/tglx/devel.git preempt

That's the first part of a larger effort related to preempt count:

 1) The analysis of the usage sites of in_interrupt(), in_atomic(),
    in_softirq() is still ongoing, but so far the number of buggy users is
    clearly the vast majority. There will be seperate patch series
    (currently 46 and counting) to address these issues once the analysis
    is complete in the next days.

 2) The long discussed state tracking of local irq disable in preempt count
    which accounts interrupt disabled sections as atomic and avoids issuing
    costly instructions (sti, cli, popf or their non X86 counterparts) when
    the state does not change, i.e. nested irq_save() or irq_restore(). I
    have this working on X86 already and contrary to my earlier attempts
    this was reasonably straight forward due to the recent entry/exit code
    consolidation.

    What I've not done yet is to optimize the preempt count handling
    of the [un]lock_irq* operations so they handle the interrupt disabled
    state and the preempt count modification in one go. That's an obvious
    add on, but correctness first ...

 3) Lazy interrupt disabling as a straight forward extension to #2. This
    avoids the actual disabling at the CPU level completely and catches an
    incoming interrupt in the low level entry code, modifies the interrupt
    disabled state on the return stack, notes the interrupt as pending in
    software and raises it again when interrupts are reenabled. This has
    still a few issues which I'm hunting down (cpuidle is unhappy ...)

Thanks,

	tglx
---
 arch/arm/include/asm/assembler.h                                 |   11 --
 arch/arm/kernel/iwmmxt.S                                         |    2 
 arch/arm/mach-ep93xx/crunch-bits.S                               |    2 
 arch/xtensa/kernel/entry.S                                       |    2 
 drivers/gpu/drm/i915/Kconfig.debug                               |    1 
 drivers/gpu/drm/i915/i915_utils.h                                |    3 
 include/linux/bit_spinlock.h                                     |    4 -
 include/linux/lockdep.h                                          |    6 -
 include/linux/pagemap.h                                          |    4 -
 include/linux/preempt.h                                          |   37 +---------
 include/linux/uaccess.h                                          |    6 -
 kernel/Kconfig.preempt                                           |    4 -
 kernel/sched/core.c                                              |    6 -
 lib/Kconfig.debug                                                |    3 
 lib/Kconfig.debug.rej                                            |   14 +--
 tools/testing/selftests/rcutorture/configs/rcu/SRCU-t            |    1 
 tools/testing/selftests/rcutorture/configs/rcu/SRCU-u            |    1 
 tools/testing/selftests/rcutorture/configs/rcu/TINY01            |    1 
 tools/testing/selftests/rcutorture/doc/TINY_RCU.txt              |    5 -
 tools/testing/selftests/rcutorture/doc/TREE_RCU-kconfig.txt      |    1 
 tools/testing/selftests/rcutorture/formal/srcu-cbmc/src/config.h |    1 
 21 files changed, 23 insertions(+), 92 deletions(-)


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 282+ messages in thread

* [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-14 20:42 ` Thomas Gleixner
  0 siblings, 0 replies; 282+ messages in thread
From: Thomas Gleixner @ 2020-09-14 20:42 UTC (permalink / raw)
  To: LKML
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, linux-mm, linux-kselftest,
	linux-hexagon, Will Deacon, Ingo Molnar, Anton Ivanov,
	linux-arch, Vincent Guittot, Brian Cain, Richard Weinberger,
	Russell King, David Airlie, Ingo Molnar, Geert Uytterhoeven,
	Mel Gorman, intel-gfx, Matt Turner, Valentin Schneider,
	linux-xtensa, Shuah Khan, Paul E. McKenney, Jeff Dike, linux-um,
	Josh Triplett, Steven Rostedt, rcu, linux-m68k, Ivan Kokshaysky,
	Rodrigo Vivi, Dietmar Eggemann, linux-arm-kernel,
	Richard Henderson, Chris Zankel, Max Filippov, Linus Torvalds,
	linux-alpha, Mathieu Desnoyers, Andrew Morton,
	Daniel Bristot de Oliveira

Folks!

While working on various preempt count related things, I stumbled (again)
over the inconsistency of our preempt count handling.

The handling of preempt_count() is inconsistent accross kernel
configurations. On kernels which have PREEMPT_COUNT=n
preempt_disable/enable() and the lock/unlock functions are not affecting
the preempt count, only local_bh_disable/enable() and _bh variants of
locking, soft interrupt delivery, hard interrupt and NMI context affect it.

It's therefore impossible to have a consistent set of checks which provide
information about the context in which a function is called. In many cases
it makes sense to have seperate functions for seperate contexts, but there
are valid reasons to avoid that and handle different calling contexts
conditionally.

The lack of such indicators which work on all kernel configuratios is a
constant source of trouble because developers either do not understand the
implications or try to work around this inconsistency in weird
ways. Neither seem these issues be catched by reviewers and testing.

Recently merged code does:

	 gfp = preemptible() ? GFP_KERNEL : GFP_ATOMIC;

Looks obviously correct, except for the fact that preemptible() is
unconditionally false for CONFIF_PREEMPT_COUNT=n, i.e. all allocations in
that code use GFP_ATOMIC on such kernels.

Attempts to make preempt count unconditional and consistent have been
rejected in the past with handwaving performance arguments.

Freshly conducted benchmarks did not reveal any measurable impact from
enabling preempt count unconditionally. On kernels with CONFIG_PREEMPT_NONE
or CONFIG_PREEMPT_VOLUNTARY the preempt count is only incremented and
decremented but the result of the decrement is not tested. Contrary to that
enabling CONFIG_PREEMPT which tests the result has a small but measurable
impact due to the conditional branch/call.

It's about time to make essential functionality of the kernel consistent
accross the various preemption models.

The series is also available from git:

   git://git.kernel.org/pub/scm/linux/kernel/git/tglx/devel.git preempt

That's the first part of a larger effort related to preempt count:

 1) The analysis of the usage sites of in_interrupt(), in_atomic(),
    in_softirq() is still ongoing, but so far the number of buggy users is
    clearly the vast majority. There will be seperate patch series
    (currently 46 and counting) to address these issues once the analysis
    is complete in the next days.

 2) The long discussed state tracking of local irq disable in preempt count
    which accounts interrupt disabled sections as atomic and avoids issuing
    costly instructions (sti, cli, popf or their non X86 counterparts) when
    the state does not change, i.e. nested irq_save() or irq_restore(). I
    have this working on X86 already and contrary to my earlier attempts
    this was reasonably straight forward due to the recent entry/exit code
    consolidation.

    What I've not done yet is to optimize the preempt count handling
    of the [un]lock_irq* operations so they handle the interrupt disabled
    state and the preempt count modification in one go. That's an obvious
    add on, but correctness first ...

 3) Lazy interrupt disabling as a straight forward extension to #2. This
    avoids the actual disabling at the CPU level completely and catches an
    incoming interrupt in the low level entry code, modifies the interrupt
    disabled state on the return stack, notes the interrupt as pending in
    software and raises it again when interrupts are reenabled. This has
    still a few issues which I'm hunting down (cpuidle is unhappy ...)

Thanks,

	tglx
---
 arch/arm/include/asm/assembler.h                                 |   11 --
 arch/arm/kernel/iwmmxt.S                                         |    2 
 arch/arm/mach-ep93xx/crunch-bits.S                               |    2 
 arch/xtensa/kernel/entry.S                                       |    2 
 drivers/gpu/drm/i915/Kconfig.debug                               |    1 
 drivers/gpu/drm/i915/i915_utils.h                                |    3 
 include/linux/bit_spinlock.h                                     |    4 -
 include/linux/lockdep.h                                          |    6 -
 include/linux/pagemap.h                                          |    4 -
 include/linux/preempt.h                                          |   37 +---------
 include/linux/uaccess.h                                          |    6 -
 kernel/Kconfig.preempt                                           |    4 -
 kernel/sched/core.c                                              |    6 -
 lib/Kconfig.debug                                                |    3 
 lib/Kconfig.debug.rej                                            |   14 +--
 tools/testing/selftests/rcutorture/configs/rcu/SRCU-t            |    1 
 tools/testing/selftests/rcutorture/configs/rcu/SRCU-u            |    1 
 tools/testing/selftests/rcutorture/configs/rcu/TINY01            |    1 
 tools/testing/selftests/rcutorture/doc/TINY_RCU.txt              |    5 -
 tools/testing/selftests/rcutorture/doc/TREE_RCU-kconfig.txt      |    1 
 tools/testing/selftests/rcutorture/formal/srcu-cbmc/src/config.h |    1 
 21 files changed, 23 insertions(+), 92 deletions(-)


_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply	[flat|nested] 282+ messages in thread

* [Intel-gfx] [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-14 20:42 ` Thomas Gleixner
  0 siblings, 0 replies; 282+ messages in thread
From: Thomas Gleixner @ 2020-09-14 20:42 UTC (permalink / raw)
  To: LKML
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, linux-mm, linux-kselftest,
	linux-hexagon, Will Deacon, Ingo Molnar, Anton Ivanov,
	linux-arch, Brian Cain, Richard Weinberger, Russell King,
	David Airlie, Ingo Molnar, Geert Uytterhoeven, Mel Gorman,
	intel-gfx, Matt Turner, Valentin Schneider, linux-xtensa,
	Shuah Khan, Paul E. McKenney, Jeff Dike, linux-um, Josh Triplett,
	Steven Rostedt, rcu, linux-m68k, Ivan Kokshaysky,
	Dietmar Eggemann, linux-arm-kernel, Richard Henderson,
	Chris Zankel, Max Filippov, Linus Torvalds, linux-alpha,
	Mathieu Desnoyers, Andrew Morton, Daniel Bristot de Oliveira

Folks!

While working on various preempt count related things, I stumbled (again)
over the inconsistency of our preempt count handling.

The handling of preempt_count() is inconsistent accross kernel
configurations. On kernels which have PREEMPT_COUNT=n
preempt_disable/enable() and the lock/unlock functions are not affecting
the preempt count, only local_bh_disable/enable() and _bh variants of
locking, soft interrupt delivery, hard interrupt and NMI context affect it.

It's therefore impossible to have a consistent set of checks which provide
information about the context in which a function is called. In many cases
it makes sense to have seperate functions for seperate contexts, but there
are valid reasons to avoid that and handle different calling contexts
conditionally.

The lack of such indicators which work on all kernel configuratios is a
constant source of trouble because developers either do not understand the
implications or try to work around this inconsistency in weird
ways. Neither seem these issues be catched by reviewers and testing.

Recently merged code does:

	 gfp = preemptible() ? GFP_KERNEL : GFP_ATOMIC;

Looks obviously correct, except for the fact that preemptible() is
unconditionally false for CONFIF_PREEMPT_COUNT=n, i.e. all allocations in
that code use GFP_ATOMIC on such kernels.

Attempts to make preempt count unconditional and consistent have been
rejected in the past with handwaving performance arguments.

Freshly conducted benchmarks did not reveal any measurable impact from
enabling preempt count unconditionally. On kernels with CONFIG_PREEMPT_NONE
or CONFIG_PREEMPT_VOLUNTARY the preempt count is only incremented and
decremented but the result of the decrement is not tested. Contrary to that
enabling CONFIG_PREEMPT which tests the result has a small but measurable
impact due to the conditional branch/call.

It's about time to make essential functionality of the kernel consistent
accross the various preemption models.

The series is also available from git:

   git://git.kernel.org/pub/scm/linux/kernel/git/tglx/devel.git preempt

That's the first part of a larger effort related to preempt count:

 1) The analysis of the usage sites of in_interrupt(), in_atomic(),
    in_softirq() is still ongoing, but so far the number of buggy users is
    clearly the vast majority. There will be seperate patch series
    (currently 46 and counting) to address these issues once the analysis
    is complete in the next days.

 2) The long discussed state tracking of local irq disable in preempt count
    which accounts interrupt disabled sections as atomic and avoids issuing
    costly instructions (sti, cli, popf or their non X86 counterparts) when
    the state does not change, i.e. nested irq_save() or irq_restore(). I
    have this working on X86 already and contrary to my earlier attempts
    this was reasonably straight forward due to the recent entry/exit code
    consolidation.

    What I've not done yet is to optimize the preempt count handling
    of the [un]lock_irq* operations so they handle the interrupt disabled
    state and the preempt count modification in one go. That's an obvious
    add on, but correctness first ...

 3) Lazy interrupt disabling as a straight forward extension to #2. This
    avoids the actual disabling at the CPU level completely and catches an
    incoming interrupt in the low level entry code, modifies the interrupt
    disabled state on the return stack, notes the interrupt as pending in
    software and raises it again when interrupts are reenabled. This has
    still a few issues which I'm hunting down (cpuidle is unhappy ...)

Thanks,

	tglx
---
 arch/arm/include/asm/assembler.h                                 |   11 --
 arch/arm/kernel/iwmmxt.S                                         |    2 
 arch/arm/mach-ep93xx/crunch-bits.S                               |    2 
 arch/xtensa/kernel/entry.S                                       |    2 
 drivers/gpu/drm/i915/Kconfig.debug                               |    1 
 drivers/gpu/drm/i915/i915_utils.h                                |    3 
 include/linux/bit_spinlock.h                                     |    4 -
 include/linux/lockdep.h                                          |    6 -
 include/linux/pagemap.h                                          |    4 -
 include/linux/preempt.h                                          |   37 +---------
 include/linux/uaccess.h                                          |    6 -
 kernel/Kconfig.preempt                                           |    4 -
 kernel/sched/core.c                                              |    6 -
 lib/Kconfig.debug                                                |    3 
 lib/Kconfig.debug.rej                                            |   14 +--
 tools/testing/selftests/rcutorture/configs/rcu/SRCU-t            |    1 
 tools/testing/selftests/rcutorture/configs/rcu/SRCU-u            |    1 
 tools/testing/selftests/rcutorture/configs/rcu/TINY01            |    1 
 tools/testing/selftests/rcutorture/doc/TINY_RCU.txt              |    5 -
 tools/testing/selftests/rcutorture/doc/TREE_RCU-kconfig.txt      |    1 
 tools/testing/selftests/rcutorture/formal/srcu-cbmc/src/config.h |    1 
 21 files changed, 23 insertions(+), 92 deletions(-)


_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

^ permalink raw reply	[flat|nested] 282+ messages in thread

* [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-14 20:42 ` Thomas Gleixner
  0 siblings, 0 replies; 282+ messages in thread
From: Thomas Gleixner @ 2020-09-14 20:42 UTC (permalink / raw)
  To: LKML
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Joonas Lahtinen, Lai Jiangshan, dri-devel, Ben Segall, linux-mm,
	linux-kselftest, linux-hexagon, Will Deacon, Ingo Molnar,
	Anton Ivanov, linux-arch, Vincent Guittot, Brian Cain,
	Richard Weinberger, Russell King, David Airlie, Ingo Molnar,
	Geert Uytterhoeven, Mel Gorman, intel-gfx, Matt Turner, Valentin

Folks!

While working on various preempt count related things, I stumbled (again)
over the inconsistency of our preempt count handling.

The handling of preempt_count() is inconsistent accross kernel
configurations. On kernels which have PREEMPT_COUNT=n
preempt_disable/enable() and the lock/unlock functions are not affecting
the preempt count, only local_bh_disable/enable() and _bh variants of
locking, soft interrupt delivery, hard interrupt and NMI context affect it.

It's therefore impossible to have a consistent set of checks which provide
information about the context in which a function is called. In many cases
it makes sense to have seperate functions for seperate contexts, but there
are valid reasons to avoid that and handle different calling contexts
conditionally.

The lack of such indicators which work on all kernel configuratios is a
constant source of trouble because developers either do not understand the
implications or try to work around this inconsistency in weird
ways. Neither seem these issues be catched by reviewers and testing.

Recently merged code does:

	 gfp = preemptible() ? GFP_KERNEL : GFP_ATOMIC;

Looks obviously correct, except for the fact that preemptible() is
unconditionally false for CONFIF_PREEMPT_COUNT=n, i.e. all allocations in
that code use GFP_ATOMIC on such kernels.

Attempts to make preempt count unconditional and consistent have been
rejected in the past with handwaving performance arguments.

Freshly conducted benchmarks did not reveal any measurable impact from
enabling preempt count unconditionally. On kernels with CONFIG_PREEMPT_NONE
or CONFIG_PREEMPT_VOLUNTARY the preempt count is only incremented and
decremented but the result of the decrement is not tested. Contrary to that
enabling CONFIG_PREEMPT which tests the result has a small but measurable
impact due to the conditional branch/call.

It's about time to make essential functionality of the kernel consistent
accross the various preemption models.

The series is also available from git:

   git://git.kernel.org/pub/scm/linux/kernel/git/tglx/devel.git preempt

That's the first part of a larger effort related to preempt count:

 1) The analysis of the usage sites of in_interrupt(), in_atomic(),
    in_softirq() is still ongoing, but so far the number of buggy users is
    clearly the vast majority. There will be seperate patch series
    (currently 46 and counting) to address these issues once the analysis
    is complete in the next days.

 2) The long discussed state tracking of local irq disable in preempt count
    which accounts interrupt disabled sections as atomic and avoids issuing
    costly instructions (sti, cli, popf or their non X86 counterparts) when
    the state does not change, i.e. nested irq_save() or irq_restore(). I
    have this working on X86 already and contrary to my earlier attempts
    this was reasonably straight forward due to the recent entry/exit code
    consolidation.

    What I've not done yet is to optimize the preempt count handling
    of the [un]lock_irq* operations so they handle the interrupt disabled
    state and the preempt count modification in one go. That's an obvious
    add on, but correctness first ...

 3) Lazy interrupt disabling as a straight forward extension to #2. This
    avoids the actual disabling at the CPU level completely and catches an
    incoming interrupt in the low level entry code, modifies the interrupt
    disabled state on the return stack, notes the interrupt as pending in
    software and raises it again when interrupts are reenabled. This has
    still a few issues which I'm hunting down (cpuidle is unhappy ...)

Thanks,

	tglx
---
 arch/arm/include/asm/assembler.h                                 |   11 --
 arch/arm/kernel/iwmmxt.S                                         |    2 
 arch/arm/mach-ep93xx/crunch-bits.S                               |    2 
 arch/xtensa/kernel/entry.S                                       |    2 
 drivers/gpu/drm/i915/Kconfig.debug                               |    1 
 drivers/gpu/drm/i915/i915_utils.h                                |    3 
 include/linux/bit_spinlock.h                                     |    4 -
 include/linux/lockdep.h                                          |    6 -
 include/linux/pagemap.h                                          |    4 -
 include/linux/preempt.h                                          |   37 +---------
 include/linux/uaccess.h                                          |    6 -
 kernel/Kconfig.preempt                                           |    4 -
 kernel/sched/core.c                                              |    6 -
 lib/Kconfig.debug                                                |    3 
 lib/Kconfig.debug.rej                                            |   14 +--
 tools/testing/selftests/rcutorture/configs/rcu/SRCU-t            |    1 
 tools/testing/selftests/rcutorture/configs/rcu/SRCU-u            |    1 
 tools/testing/selftests/rcutorture/configs/rcu/TINY01            |    1 
 tools/testing/selftests/rcutorture/doc/TINY_RCU.txt              |    5 -
 tools/testing/selftests/rcutorture/doc/TREE_RCU-kconfig.txt      |    1 
 tools/testing/selftests/rcutorture/formal/srcu-cbmc/src/config.h |    1 
 21 files changed, 23 insertions(+), 92 deletions(-)

^ permalink raw reply	[flat|nested] 282+ messages in thread

* [patch 01/13] lib/debug: Remove pointless ARCH_NO_PREEMPT dependencies
  2020-09-14 20:42 ` Thomas Gleixner
                     ` (2 preceding siblings ...)
  (?)
@ 2020-09-14 20:42   ` Thomas Gleixner
  -1 siblings, 0 replies; 282+ messages in thread
From: Thomas Gleixner @ 2020-09-14 20:42 UTC (permalink / raw)
  To: LKML
  Cc: linux-arch, Linus Torvalds, Sebastian Andrzej Siewior,
	Valentin Schneider, Richard Henderson, Ivan Kokshaysky,
	Matt Turner, linux-alpha, Jeff Dike, Richard Weinberger,
	Anton Ivanov, linux-um, Brian Cain, linux-hexagon,
	Geert Uytterhoeven, linux-m68k, Ingo Molnar, Peter Zijlstra,
	Juri Lelli, Vincent Guittot, Dietmar Eggemann, Steven Rostedt,
	Ben Segall, Mel Gorman, Daniel Bristot de Oliveira, Will Deacon,
	Andrew Morton, linux-mm, Ingo Molnar, Russell King,
	linux-arm-kernel, Chris Zankel, Max Filippov, linux-xtensa,
	Jani Nikula, Joonas Lahtinen, Rodrigo Vivi, David Airlie,
	Daniel Vetter, intel-gfx, dri-devel, Paul E. McKenney,
	Josh Triplett, Mathieu Desnoyers, Lai Jiangshan, Shuah Khan, rcu,
	linux-kselftest

ARCH_NO_PREEMPT disables the selection of CONFIG_PREEMPT_VOLUNTARY and
CONFIG_PREEMPT, but architectures which set this config option still
support preempt count for hard and softirq accounting.

There is absolutely no reason to prevent lockdep from using the preempt
counter nor is there a reason to prevent the enablement of
CONFIG_DEBUG_ATOMIC_SLEEP on such architectures.

Remove the dependencies, which affects ALPHA, HEXAGON, M68K and UM.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Richard Henderson <rth@twiddle.net>
Cc: Ivan Kokshaysky <ink@jurassic.park.msu.ru>
Cc: Matt Turner <mattst88@gmail.com>
Cc: linux-alpha@vger.kernel.org
Cc: Jeff Dike <jdike@addtoit.com>
Cc: Richard Weinberger <richard@nod.at>
Cc: Anton Ivanov <anton.ivanov@cambridgegreys.com>
Cc: linux-um@lists.infradead.org
Cc: Brian Cain <bcain@codeaurora.org>
Cc: linux-hexagon@vger.kernel.org
Cc: Geert Uytterhoeven <geert@linux-m68k.org>
Cc: linux-m68k@lists.linux-m68k.org
---
 lib/Kconfig.debug |    3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

--- a/lib/Kconfig.debug
+++ b/lib/Kconfig.debug
@@ -1161,7 +1161,7 @@ config PROVE_LOCKING
 	select DEBUG_RWSEMS
 	select DEBUG_WW_MUTEX_SLOWPATH
 	select DEBUG_LOCK_ALLOC
-	select PREEMPT_COUNT if !ARCH_NO_PREEMPT
+	select PREEMPT_COUNT
 	select TRACE_IRQFLAGS
 	default n
 	help
@@ -1323,7 +1323,6 @@ config DEBUG_ATOMIC_SLEEP
 	bool "Sleep inside atomic section checking"
 	select PREEMPT_COUNT
 	depends on DEBUG_KERNEL
-	depends on !ARCH_NO_PREEMPT
 	help
 	  If you say Y here, various routines which may sleep will become very
 	  noisy if they are called inside atomic sections: when a spinlock is



^ permalink raw reply	[flat|nested] 282+ messages in thread

* [patch 01/13] lib/debug: Remove pointless ARCH_NO_PREEMPT dependencies
@ 2020-09-14 20:42   ` Thomas Gleixner
  0 siblings, 0 replies; 282+ messages in thread
From: Thomas Gleixner @ 2020-09-14 20:42 UTC (permalink / raw)
  To: LKML
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Joonas Lahtinen, Lai Jiangshan, dri-devel, Ben Segall, linux-mm,
	linux-kselftest, linux-hexagon, Will Deacon, Ingo Molnar,
	Anton Ivanov, linux-arch, Vincent Guittot, Brian Cain,
	Richard Weinberger, Russell King, David Airlie, Ingo Molnar,
	Geert Uytterhoeven, Mel Gorman, intel-gfx, Matt Turner,
	Valentin Schneider, linux-xtensa, Shuah Khan, Paul E. McKenney,
	Jeff Dike, linux-um, Josh Triplett, Steven Rostedt, rcu,
	linux-m68k, Ivan Kokshaysky, Jani Nikula, Rodrigo Vivi,
	Dietmar Eggemann, linux-arm-kernel, Richard Henderson,
	Chris Zankel, Max Filippov, Linus Torvalds, Daniel Vetter,
	linux-alpha, Mathieu Desnoyers, Andrew Morton,
	Daniel Bristot de Oliveira

ARCH_NO_PREEMPT disables the selection of CONFIG_PREEMPT_VOLUNTARY and
CONFIG_PREEMPT, but architectures which set this config option still
support preempt count for hard and softirq accounting.

There is absolutely no reason to prevent lockdep from using the preempt
counter nor is there a reason to prevent the enablement of
CONFIG_DEBUG_ATOMIC_SLEEP on such architectures.

Remove the dependencies, which affects ALPHA, HEXAGON, M68K and UM.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Richard Henderson <rth@twiddle.net>
Cc: Ivan Kokshaysky <ink@jurassic.park.msu.ru>
Cc: Matt Turner <mattst88@gmail.com>
Cc: linux-alpha@vger.kernel.org
Cc: Jeff Dike <jdike@addtoit.com>
Cc: Richard Weinberger <richard@nod.at>
Cc: Anton Ivanov <anton.ivanov@cambridgegreys.com>
Cc: linux-um@lists.infradead.org
Cc: Brian Cain <bcain@codeaurora.org>
Cc: linux-hexagon@vger.kernel.org
Cc: Geert Uytterhoeven <geert@linux-m68k.org>
Cc: linux-m68k@lists.linux-m68k.org
---
 lib/Kconfig.debug |    3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

--- a/lib/Kconfig.debug
+++ b/lib/Kconfig.debug
@@ -1161,7 +1161,7 @@ config PROVE_LOCKING
 	select DEBUG_RWSEMS
 	select DEBUG_WW_MUTEX_SLOWPATH
 	select DEBUG_LOCK_ALLOC
-	select PREEMPT_COUNT if !ARCH_NO_PREEMPT
+	select PREEMPT_COUNT
 	select TRACE_IRQFLAGS
 	default n
 	help
@@ -1323,7 +1323,6 @@ config DEBUG_ATOMIC_SLEEP
 	bool "Sleep inside atomic section checking"
 	select PREEMPT_COUNT
 	depends on DEBUG_KERNEL
-	depends on !ARCH_NO_PREEMPT
 	help
 	  If you say Y here, various routines which may sleep will become very
 	  noisy if they are called inside atomic sections: when a spinlock is


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 282+ messages in thread

* [patch 01/13] lib/debug: Remove pointless ARCH_NO_PREEMPT dependencies
@ 2020-09-14 20:42   ` Thomas Gleixner
  0 siblings, 0 replies; 282+ messages in thread
From: Thomas Gleixner @ 2020-09-14 20:42 UTC (permalink / raw)
  To: LKML
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, linux-mm, linux-kselftest,
	linux-hexagon, Will Deacon, Ingo Molnar, Anton Ivanov,
	linux-arch, Vincent Guittot, Brian Cain, Richard Weinberger,
	Russell King, David Airlie, Ingo Molnar, Geert Uytterhoeven,
	Mel Gorman, intel-gfx, Matt Turner, Valentin Schneider,
	linux-xtensa, Shuah Khan, Paul E. McKenney, Jeff Dike, linux-um,
	Josh Triplett, Steven Rostedt, rcu, linux-m68k, Ivan Kokshaysky,
	Rodrigo Vivi, Dietmar Eggemann, linux-arm-kernel,
	Richard Henderson, Chris Zankel, Max Filippov, Linus Torvalds,
	linux-alpha, Mathieu Desnoyers, Andrew Morton,
	Daniel Bristot de Oliveira

ARCH_NO_PREEMPT disables the selection of CONFIG_PREEMPT_VOLUNTARY and
CONFIG_PREEMPT, but architectures which set this config option still
support preempt count for hard and softirq accounting.

There is absolutely no reason to prevent lockdep from using the preempt
counter nor is there a reason to prevent the enablement of
CONFIG_DEBUG_ATOMIC_SLEEP on such architectures.

Remove the dependencies, which affects ALPHA, HEXAGON, M68K and UM.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Richard Henderson <rth@twiddle.net>
Cc: Ivan Kokshaysky <ink@jurassic.park.msu.ru>
Cc: Matt Turner <mattst88@gmail.com>
Cc: linux-alpha@vger.kernel.org
Cc: Jeff Dike <jdike@addtoit.com>
Cc: Richard Weinberger <richard@nod.at>
Cc: Anton Ivanov <anton.ivanov@cambridgegreys.com>
Cc: linux-um@lists.infradead.org
Cc: Brian Cain <bcain@codeaurora.org>
Cc: linux-hexagon@vger.kernel.org
Cc: Geert Uytterhoeven <geert@linux-m68k.org>
Cc: linux-m68k@lists.linux-m68k.org
---
 lib/Kconfig.debug |    3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

--- a/lib/Kconfig.debug
+++ b/lib/Kconfig.debug
@@ -1161,7 +1161,7 @@ config PROVE_LOCKING
 	select DEBUG_RWSEMS
 	select DEBUG_WW_MUTEX_SLOWPATH
 	select DEBUG_LOCK_ALLOC
-	select PREEMPT_COUNT if !ARCH_NO_PREEMPT
+	select PREEMPT_COUNT
 	select TRACE_IRQFLAGS
 	default n
 	help
@@ -1323,7 +1323,6 @@ config DEBUG_ATOMIC_SLEEP
 	bool "Sleep inside atomic section checking"
 	select PREEMPT_COUNT
 	depends on DEBUG_KERNEL
-	depends on !ARCH_NO_PREEMPT
 	help
 	  If you say Y here, various routines which may sleep will become very
 	  noisy if they are called inside atomic sections: when a spinlock is

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply	[flat|nested] 282+ messages in thread

* [Intel-gfx] [patch 01/13] lib/debug: Remove pointless ARCH_NO_PREEMPT dependencies
@ 2020-09-14 20:42   ` Thomas Gleixner
  0 siblings, 0 replies; 282+ messages in thread
From: Thomas Gleixner @ 2020-09-14 20:42 UTC (permalink / raw)
  To: LKML
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, linux-mm, linux-kselftest,
	linux-hexagon, Will Deacon, Ingo Molnar, Anton Ivanov,
	linux-arch, Brian Cain, Richard Weinberger, Russell King,
	David Airlie, Ingo Molnar, Geert Uytterhoeven, Mel Gorman,
	intel-gfx, Matt Turner, Valentin Schneider, linux-xtensa,
	Shuah Khan, Paul E. McKenney, Jeff Dike, linux-um, Josh Triplett,
	Steven Rostedt, rcu, linux-m68k, Ivan Kokshaysky,
	Dietmar Eggemann, linux-arm-kernel, Richard Henderson,
	Chris Zankel, Max Filippov, Linus Torvalds, linux-alpha,
	Mathieu Desnoyers, Andrew Morton, Daniel Bristot de Oliveira

ARCH_NO_PREEMPT disables the selection of CONFIG_PREEMPT_VOLUNTARY and
CONFIG_PREEMPT, but architectures which set this config option still
support preempt count for hard and softirq accounting.

There is absolutely no reason to prevent lockdep from using the preempt
counter nor is there a reason to prevent the enablement of
CONFIG_DEBUG_ATOMIC_SLEEP on such architectures.

Remove the dependencies, which affects ALPHA, HEXAGON, M68K and UM.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Richard Henderson <rth@twiddle.net>
Cc: Ivan Kokshaysky <ink@jurassic.park.msu.ru>
Cc: Matt Turner <mattst88@gmail.com>
Cc: linux-alpha@vger.kernel.org
Cc: Jeff Dike <jdike@addtoit.com>
Cc: Richard Weinberger <richard@nod.at>
Cc: Anton Ivanov <anton.ivanov@cambridgegreys.com>
Cc: linux-um@lists.infradead.org
Cc: Brian Cain <bcain@codeaurora.org>
Cc: linux-hexagon@vger.kernel.org
Cc: Geert Uytterhoeven <geert@linux-m68k.org>
Cc: linux-m68k@lists.linux-m68k.org
---
 lib/Kconfig.debug |    3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

--- a/lib/Kconfig.debug
+++ b/lib/Kconfig.debug
@@ -1161,7 +1161,7 @@ config PROVE_LOCKING
 	select DEBUG_RWSEMS
 	select DEBUG_WW_MUTEX_SLOWPATH
 	select DEBUG_LOCK_ALLOC
-	select PREEMPT_COUNT if !ARCH_NO_PREEMPT
+	select PREEMPT_COUNT
 	select TRACE_IRQFLAGS
 	default n
 	help
@@ -1323,7 +1323,6 @@ config DEBUG_ATOMIC_SLEEP
 	bool "Sleep inside atomic section checking"
 	select PREEMPT_COUNT
 	depends on DEBUG_KERNEL
-	depends on !ARCH_NO_PREEMPT
 	help
 	  If you say Y here, various routines which may sleep will become very
 	  noisy if they are called inside atomic sections: when a spinlock is

_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

^ permalink raw reply	[flat|nested] 282+ messages in thread

* [patch 01/13] lib/debug: Remove pointless ARCH_NO_PREEMPT dependencies
@ 2020-09-14 20:42   ` Thomas Gleixner
  0 siblings, 0 replies; 282+ messages in thread
From: Thomas Gleixner @ 2020-09-14 20:42 UTC (permalink / raw)
  To: LKML
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Joonas Lahtinen, Lai Jiangshan, dri-devel, Ben Segall, linux-mm,
	linux-kselftest, linux-hexagon, Will Deacon, Ingo Molnar,
	Anton Ivanov, linux-arch, Vincent Guittot, Brian Cain,
	Richard Weinberger, Russell King, David Airlie, Ingo Molnar,
	Geert Uytterhoeven, Mel Gorman, intel-gfx, Matt Turner, Valentin

ARCH_NO_PREEMPT disables the selection of CONFIG_PREEMPT_VOLUNTARY and
CONFIG_PREEMPT, but architectures which set this config option still
support preempt count for hard and softirq accounting.

There is absolutely no reason to prevent lockdep from using the preempt
counter nor is there a reason to prevent the enablement of
CONFIG_DEBUG_ATOMIC_SLEEP on such architectures.

Remove the dependencies, which affects ALPHA, HEXAGON, M68K and UM.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Richard Henderson <rth@twiddle.net>
Cc: Ivan Kokshaysky <ink@jurassic.park.msu.ru>
Cc: Matt Turner <mattst88@gmail.com>
Cc: linux-alpha@vger.kernel.org
Cc: Jeff Dike <jdike@addtoit.com>
Cc: Richard Weinberger <richard@nod.at>
Cc: Anton Ivanov <anton.ivanov@cambridgegreys.com>
Cc: linux-um@lists.infradead.org
Cc: Brian Cain <bcain@codeaurora.org>
Cc: linux-hexagon@vger.kernel.org
Cc: Geert Uytterhoeven <geert@linux-m68k.org>
Cc: linux-m68k@lists.linux-m68k.org
---
 lib/Kconfig.debug |    3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

--- a/lib/Kconfig.debug
+++ b/lib/Kconfig.debug
@@ -1161,7 +1161,7 @@ config PROVE_LOCKING
 	select DEBUG_RWSEMS
 	select DEBUG_WW_MUTEX_SLOWPATH
 	select DEBUG_LOCK_ALLOC
-	select PREEMPT_COUNT if !ARCH_NO_PREEMPT
+	select PREEMPT_COUNT
 	select TRACE_IRQFLAGS
 	default n
 	help
@@ -1323,7 +1323,6 @@ config DEBUG_ATOMIC_SLEEP
 	bool "Sleep inside atomic section checking"
 	select PREEMPT_COUNT
 	depends on DEBUG_KERNEL
-	depends on !ARCH_NO_PREEMPT
 	help
 	  If you say Y here, various routines which may sleep will become very
 	  noisy if they are called inside atomic sections: when a spinlock is

^ permalink raw reply	[flat|nested] 282+ messages in thread

* [patch 02/13] preempt: Make preempt count unconditional
  2020-09-14 20:42 ` Thomas Gleixner
                     ` (2 preceding siblings ...)
  (?)
@ 2020-09-14 20:42   ` Thomas Gleixner
  -1 siblings, 0 replies; 282+ messages in thread
From: Thomas Gleixner @ 2020-09-14 20:42 UTC (permalink / raw)
  To: LKML
  Cc: linux-arch, Linus Torvalds, Sebastian Andrzej Siewior,
	Valentin Schneider, Richard Henderson, Ivan Kokshaysky,
	Matt Turner, linux-alpha, Jeff Dike, Richard Weinberger,
	Anton Ivanov, linux-um, Brian Cain, linux-hexagon,
	Geert Uytterhoeven, linux-m68k, Ingo Molnar, Peter Zijlstra,
	Juri Lelli, Vincent Guittot, Dietmar Eggemann, Steven Rostedt,
	Ben Segall, Mel Gorman, Daniel Bristot de Oliveira, Will Deacon,
	Andrew Morton, linux-mm, Ingo Molnar, Russell King,
	linux-arm-kernel, Chris Zankel, Max Filippov, linux-xtensa,
	Jani Nikula, Joonas Lahtinen, Rodrigo Vivi, David Airlie,
	Daniel Vetter, intel-gfx, dri-devel, Paul E. McKenney,
	Josh Triplett, Mathieu Desnoyers, Lai Jiangshan, Shuah Khan, rcu,
	linux-kselftest

The handling of preempt_count() is inconsistent accross kernel
configurations. On kernels which have PREEMPT_COUNT=n
preempt_disable/enable() and the lock/unlock functions are not affecting
the preempt count, only local_bh_disable/enable() and _bh variants of
locking, soft interrupt delivery, hard interrupt and NMI context affect it.

It's therefore impossible to have a consistent set of checks which provide
information about the context in which a function is called. In many cases
it makes sense to have seperate functions for seperate contexts, but there
are valid reasons to avoid that and handle different calling contexts
conditionally.

The lack of such indicators which work on all kernel configuratios is a
constant source of trouble because developers either do not understand the
implications or try to work around this inconsistency in weird
ways. Neither seem these issues be catched by reviewers and testing.

Recently merged code does:

	 gfp = preemptible() ? GFP_KERNEL : GFP_ATOMIC;

Looks obviously correct, except for the fact that preemptible() is
unconditionally false for CONFIF_PREEMPT_COUNT=n, i.e. all allocations in
that code use GFP_ATOMIC on such kernels.

Attempts to make preempt count unconditional and consistent have been
rejected in the past with handwaving performance arguments.

Freshly conducted benchmarks did not reveal any measurable impact from
enabling preempt count unconditionally. On kernels with CONFIG_PREEMPT_NONE
or CONFIG_PREEMPT_VOLUNTARY the preempt count is only incremented and
decremented but the result of the decrement is not tested. Contrary to that
enabling CONFIG_PREEMPT which tests the result has a small but measurable
impact due to the conditional branch/call.

It's about time to make essential functionality of the kernel consistent
accross the various preemption models.

Enable CONFIG_PREEMPT_COUNT unconditionally. Follow up changes will remove
the #ifdeffery and remove the config option at the end.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 kernel/Kconfig.preempt |    3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

--- a/kernel/Kconfig.preempt
+++ b/kernel/Kconfig.preempt
@@ -75,8 +75,7 @@ config PREEMPT_RT
 endchoice
 
 config PREEMPT_COUNT
-       bool
+       def_bool y
 
 config PREEMPTION
        bool
-       select PREEMPT_COUNT



^ permalink raw reply	[flat|nested] 282+ messages in thread

* [patch 02/13] preempt: Make preempt count unconditional
@ 2020-09-14 20:42   ` Thomas Gleixner
  0 siblings, 0 replies; 282+ messages in thread
From: Thomas Gleixner @ 2020-09-14 20:42 UTC (permalink / raw)
  To: LKML
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Joonas Lahtinen, Lai Jiangshan, dri-devel, Ben Segall, linux-mm,
	linux-kselftest, linux-hexagon, Will Deacon, Ingo Molnar,
	Anton Ivanov, linux-arch, Vincent Guittot, Brian Cain,
	Richard Weinberger, Russell King, David Airlie, Ingo Molnar,
	Geert Uytterhoeven, Mel Gorman, intel-gfx, Matt Turner,
	Valentin Schneider, linux-xtensa, Shuah Khan, Paul E. McKenney,
	Jeff Dike, linux-um, Josh Triplett, Steven Rostedt, rcu,
	linux-m68k, Ivan Kokshaysky, Jani Nikula, Rodrigo Vivi,
	Dietmar Eggemann, linux-arm-kernel, Richard Henderson,
	Chris Zankel, Max Filippov, Linus Torvalds, Daniel Vetter,
	linux-alpha, Mathieu Desnoyers, Andrew Morton,
	Daniel Bristot de Oliveira

The handling of preempt_count() is inconsistent accross kernel
configurations. On kernels which have PREEMPT_COUNT=n
preempt_disable/enable() and the lock/unlock functions are not affecting
the preempt count, only local_bh_disable/enable() and _bh variants of
locking, soft interrupt delivery, hard interrupt and NMI context affect it.

It's therefore impossible to have a consistent set of checks which provide
information about the context in which a function is called. In many cases
it makes sense to have seperate functions for seperate contexts, but there
are valid reasons to avoid that and handle different calling contexts
conditionally.

The lack of such indicators which work on all kernel configuratios is a
constant source of trouble because developers either do not understand the
implications or try to work around this inconsistency in weird
ways. Neither seem these issues be catched by reviewers and testing.

Recently merged code does:

	 gfp = preemptible() ? GFP_KERNEL : GFP_ATOMIC;

Looks obviously correct, except for the fact that preemptible() is
unconditionally false for CONFIF_PREEMPT_COUNT=n, i.e. all allocations in
that code use GFP_ATOMIC on such kernels.

Attempts to make preempt count unconditional and consistent have been
rejected in the past with handwaving performance arguments.

Freshly conducted benchmarks did not reveal any measurable impact from
enabling preempt count unconditionally. On kernels with CONFIG_PREEMPT_NONE
or CONFIG_PREEMPT_VOLUNTARY the preempt count is only incremented and
decremented but the result of the decrement is not tested. Contrary to that
enabling CONFIG_PREEMPT which tests the result has a small but measurable
impact due to the conditional branch/call.

It's about time to make essential functionality of the kernel consistent
accross the various preemption models.

Enable CONFIG_PREEMPT_COUNT unconditionally. Follow up changes will remove
the #ifdeffery and remove the config option at the end.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 kernel/Kconfig.preempt |    3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

--- a/kernel/Kconfig.preempt
+++ b/kernel/Kconfig.preempt
@@ -75,8 +75,7 @@ config PREEMPT_RT
 endchoice
 
 config PREEMPT_COUNT
-       bool
+       def_bool y
 
 config PREEMPTION
        bool
-       select PREEMPT_COUNT


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 282+ messages in thread

* [patch 02/13] preempt: Make preempt count unconditional
@ 2020-09-14 20:42   ` Thomas Gleixner
  0 siblings, 0 replies; 282+ messages in thread
From: Thomas Gleixner @ 2020-09-14 20:42 UTC (permalink / raw)
  To: LKML
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, linux-mm, linux-kselftest,
	linux-hexagon, Will Deacon, Ingo Molnar, Anton Ivanov,
	linux-arch, Vincent Guittot, Brian Cain, Richard Weinberger,
	Russell King, David Airlie, Ingo Molnar, Geert Uytterhoeven,
	Mel Gorman, intel-gfx, Matt Turner, Valentin Schneider,
	linux-xtensa, Shuah Khan, Paul E. McKenney, Jeff Dike, linux-um,
	Josh Triplett, Steven Rostedt, rcu, linux-m68k, Ivan Kokshaysky,
	Rodrigo Vivi, Dietmar Eggemann, linux-arm-kernel,
	Richard Henderson, Chris Zankel, Max Filippov, Linus Torvalds,
	linux-alpha, Mathieu Desnoyers, Andrew Morton,
	Daniel Bristot de Oliveira

The handling of preempt_count() is inconsistent accross kernel
configurations. On kernels which have PREEMPT_COUNT=n
preempt_disable/enable() and the lock/unlock functions are not affecting
the preempt count, only local_bh_disable/enable() and _bh variants of
locking, soft interrupt delivery, hard interrupt and NMI context affect it.

It's therefore impossible to have a consistent set of checks which provide
information about the context in which a function is called. In many cases
it makes sense to have seperate functions for seperate contexts, but there
are valid reasons to avoid that and handle different calling contexts
conditionally.

The lack of such indicators which work on all kernel configuratios is a
constant source of trouble because developers either do not understand the
implications or try to work around this inconsistency in weird
ways. Neither seem these issues be catched by reviewers and testing.

Recently merged code does:

	 gfp = preemptible() ? GFP_KERNEL : GFP_ATOMIC;

Looks obviously correct, except for the fact that preemptible() is
unconditionally false for CONFIF_PREEMPT_COUNT=n, i.e. all allocations in
that code use GFP_ATOMIC on such kernels.

Attempts to make preempt count unconditional and consistent have been
rejected in the past with handwaving performance arguments.

Freshly conducted benchmarks did not reveal any measurable impact from
enabling preempt count unconditionally. On kernels with CONFIG_PREEMPT_NONE
or CONFIG_PREEMPT_VOLUNTARY the preempt count is only incremented and
decremented but the result of the decrement is not tested. Contrary to that
enabling CONFIG_PREEMPT which tests the result has a small but measurable
impact due to the conditional branch/call.

It's about time to make essential functionality of the kernel consistent
accross the various preemption models.

Enable CONFIG_PREEMPT_COUNT unconditionally. Follow up changes will remove
the #ifdeffery and remove the config option at the end.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 kernel/Kconfig.preempt |    3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

--- a/kernel/Kconfig.preempt
+++ b/kernel/Kconfig.preempt
@@ -75,8 +75,7 @@ config PREEMPT_RT
 endchoice
 
 config PREEMPT_COUNT
-       bool
+       def_bool y
 
 config PREEMPTION
        bool
-       select PREEMPT_COUNT

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply	[flat|nested] 282+ messages in thread

* [Intel-gfx] [patch 02/13] preempt: Make preempt count unconditional
@ 2020-09-14 20:42   ` Thomas Gleixner
  0 siblings, 0 replies; 282+ messages in thread
From: Thomas Gleixner @ 2020-09-14 20:42 UTC (permalink / raw)
  To: LKML
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, linux-mm, linux-kselftest,
	linux-hexagon, Will Deacon, Ingo Molnar, Anton Ivanov,
	linux-arch, Brian Cain, Richard Weinberger, Russell King,
	David Airlie, Ingo Molnar, Geert Uytterhoeven, Mel Gorman,
	intel-gfx, Matt Turner, Valentin Schneider, linux-xtensa,
	Shuah Khan, Paul E. McKenney, Jeff Dike, linux-um, Josh Triplett,
	Steven Rostedt, rcu, linux-m68k, Ivan Kokshaysky,
	Dietmar Eggemann, linux-arm-kernel, Richard Henderson,
	Chris Zankel, Max Filippov, Linus Torvalds, linux-alpha,
	Mathieu Desnoyers, Andrew Morton, Daniel Bristot de Oliveira

The handling of preempt_count() is inconsistent accross kernel
configurations. On kernels which have PREEMPT_COUNT=n
preempt_disable/enable() and the lock/unlock functions are not affecting
the preempt count, only local_bh_disable/enable() and _bh variants of
locking, soft interrupt delivery, hard interrupt and NMI context affect it.

It's therefore impossible to have a consistent set of checks which provide
information about the context in which a function is called. In many cases
it makes sense to have seperate functions for seperate contexts, but there
are valid reasons to avoid that and handle different calling contexts
conditionally.

The lack of such indicators which work on all kernel configuratios is a
constant source of trouble because developers either do not understand the
implications or try to work around this inconsistency in weird
ways. Neither seem these issues be catched by reviewers and testing.

Recently merged code does:

	 gfp = preemptible() ? GFP_KERNEL : GFP_ATOMIC;

Looks obviously correct, except for the fact that preemptible() is
unconditionally false for CONFIF_PREEMPT_COUNT=n, i.e. all allocations in
that code use GFP_ATOMIC on such kernels.

Attempts to make preempt count unconditional and consistent have been
rejected in the past with handwaving performance arguments.

Freshly conducted benchmarks did not reveal any measurable impact from
enabling preempt count unconditionally. On kernels with CONFIG_PREEMPT_NONE
or CONFIG_PREEMPT_VOLUNTARY the preempt count is only incremented and
decremented but the result of the decrement is not tested. Contrary to that
enabling CONFIG_PREEMPT which tests the result has a small but measurable
impact due to the conditional branch/call.

It's about time to make essential functionality of the kernel consistent
accross the various preemption models.

Enable CONFIG_PREEMPT_COUNT unconditionally. Follow up changes will remove
the #ifdeffery and remove the config option at the end.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 kernel/Kconfig.preempt |    3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

--- a/kernel/Kconfig.preempt
+++ b/kernel/Kconfig.preempt
@@ -75,8 +75,7 @@ config PREEMPT_RT
 endchoice
 
 config PREEMPT_COUNT
-       bool
+       def_bool y
 
 config PREEMPTION
        bool
-       select PREEMPT_COUNT

_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

^ permalink raw reply	[flat|nested] 282+ messages in thread

* [patch 02/13] preempt: Make preempt count unconditional
@ 2020-09-14 20:42   ` Thomas Gleixner
  0 siblings, 0 replies; 282+ messages in thread
From: Thomas Gleixner @ 2020-09-14 20:42 UTC (permalink / raw)
  To: LKML
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Joonas Lahtinen, Lai Jiangshan, dri-devel, Ben Segall, linux-mm,
	linux-kselftest, linux-hexagon, Will Deacon, Ingo Molnar,
	Anton Ivanov, linux-arch, Vincent Guittot, Brian Cain,
	Richard Weinberger, Russell King, David Airlie, Ingo Molnar,
	Geert Uytterhoeven, Mel Gorman, intel-gfx, Matt Turner, Valentin

The handling of preempt_count() is inconsistent accross kernel
configurations. On kernels which have PREEMPT_COUNT=n
preempt_disable/enable() and the lock/unlock functions are not affecting
the preempt count, only local_bh_disable/enable() and _bh variants of
locking, soft interrupt delivery, hard interrupt and NMI context affect it.

It's therefore impossible to have a consistent set of checks which provide
information about the context in which a function is called. In many cases
it makes sense to have seperate functions for seperate contexts, but there
are valid reasons to avoid that and handle different calling contexts
conditionally.

The lack of such indicators which work on all kernel configuratios is a
constant source of trouble because developers either do not understand the
implications or try to work around this inconsistency in weird
ways. Neither seem these issues be catched by reviewers and testing.

Recently merged code does:

	 gfp = preemptible() ? GFP_KERNEL : GFP_ATOMIC;

Looks obviously correct, except for the fact that preemptible() is
unconditionally false for CONFIF_PREEMPT_COUNT=n, i.e. all allocations in
that code use GFP_ATOMIC on such kernels.

Attempts to make preempt count unconditional and consistent have been
rejected in the past with handwaving performance arguments.

Freshly conducted benchmarks did not reveal any measurable impact from
enabling preempt count unconditionally. On kernels with CONFIG_PREEMPT_NONE
or CONFIG_PREEMPT_VOLUNTARY the preempt count is only incremented and
decremented but the result of the decrement is not tested. Contrary to that
enabling CONFIG_PREEMPT which tests the result has a small but measurable
impact due to the conditional branch/call.

It's about time to make essential functionality of the kernel consistent
accross the various preemption models.

Enable CONFIG_PREEMPT_COUNT unconditionally. Follow up changes will remove
the #ifdeffery and remove the config option at the end.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 kernel/Kconfig.preempt |    3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

--- a/kernel/Kconfig.preempt
+++ b/kernel/Kconfig.preempt
@@ -75,8 +75,7 @@ config PREEMPT_RT
 endchoice
 
 config PREEMPT_COUNT
-       bool
+       def_bool y
 
 config PREEMPTION
        bool
-       select PREEMPT_COUNT

^ permalink raw reply	[flat|nested] 282+ messages in thread

* [patch 03/13] preempt: Clenaup PREEMPT_COUNT leftovers
  2020-09-14 20:42 ` Thomas Gleixner
                     ` (2 preceding siblings ...)
  (?)
@ 2020-09-14 20:42   ` Thomas Gleixner
  -1 siblings, 0 replies; 282+ messages in thread
From: Thomas Gleixner @ 2020-09-14 20:42 UTC (permalink / raw)
  To: LKML
  Cc: linux-arch, Linus Torvalds, Sebastian Andrzej Siewior,
	Valentin Schneider, Ingo Molnar, Peter Zijlstra, Juri Lelli,
	Vincent Guittot, Dietmar Eggemann, Steven Rostedt, Ben Segall,
	Mel Gorman, Daniel Bristot de Oliveira, Richard Henderson,
	Ivan Kokshaysky, Matt Turner, linux-alpha, Jeff Dike,
	Richard Weinberger, Anton Ivanov, linux-um, Brian Cain,
	linux-hexagon, Geert Uytterhoeven, linux-m68k, Will Deacon,
	Andrew Morton, linux-mm, Ingo Molnar, Russell King,
	linux-arm-kernel, Chris Zankel, Max Filippov, linux-xtensa,
	Jani Nikula, Joonas Lahtinen, Rodrigo Vivi, David Airlie,
	Daniel Vetter, intel-gfx, dri-devel, Paul E. McKenney,
	Josh Triplett, Mathieu Desnoyers, Lai Jiangshan, Shuah Khan, rcu,
	linux-kselftest

CONFIG_PREEMPT_COUNT is now unconditionally enabled and will be
removed. Cleanup the leftovers before doing so.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Juri Lelli <juri.lelli@redhat.com>
Cc: Vincent Guittot <vincent.guittot@linaro.org>
Cc: Dietmar Eggemann <dietmar.eggemann@arm.com>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Ben Segall <bsegall@google.com>
Cc: Mel Gorman <mgorman@suse.de>
Cc: Daniel Bristot de Oliveira <bristot@redhat.com>
---
 include/linux/preempt.h |   37 ++++---------------------------------
 1 file changed, 4 insertions(+), 33 deletions(-)

--- a/include/linux/preempt.h
+++ b/include/linux/preempt.h
@@ -56,8 +56,7 @@
 #define PREEMPT_DISABLED	(PREEMPT_DISABLE_OFFSET + PREEMPT_ENABLED)
 
 /*
- * Disable preemption until the scheduler is running -- use an unconditional
- * value so that it also works on !PREEMPT_COUNT kernels.
+ * Disable preemption until the scheduler is running.
  *
  * Reset by start_kernel()->sched_init()->init_idle()->init_idle_preempt_count().
  */
@@ -69,7 +68,6 @@
  *
  *    preempt_count() == 2*PREEMPT_DISABLE_OFFSET
  *
- * Note: PREEMPT_DISABLE_OFFSET is 0 for !PREEMPT_COUNT kernels.
  * Note: See finish_task_switch().
  */
 #define FORK_PREEMPT_COUNT	(2*PREEMPT_DISABLE_OFFSET + PREEMPT_ENABLED)
@@ -106,11 +104,7 @@
 /*
  * The preempt_count offset after preempt_disable();
  */
-#if defined(CONFIG_PREEMPT_COUNT)
-# define PREEMPT_DISABLE_OFFSET	PREEMPT_OFFSET
-#else
-# define PREEMPT_DISABLE_OFFSET	0
-#endif
+#define PREEMPT_DISABLE_OFFSET	PREEMPT_OFFSET
 
 /*
  * The preempt_count offset after spin_lock()
@@ -122,8 +116,8 @@
  *
  *  spin_lock_bh()
  *
- * Which need to disable both preemption (CONFIG_PREEMPT_COUNT) and
- * softirqs, such that unlock sequences of:
+ * Which need to disable both preemption and softirqs, such that unlock
+ * sequences of:
  *
  *  spin_unlock();
  *  local_bh_enable();
@@ -164,8 +158,6 @@ extern void preempt_count_sub(int val);
 #define preempt_count_inc() preempt_count_add(1)
 #define preempt_count_dec() preempt_count_sub(1)
 
-#ifdef CONFIG_PREEMPT_COUNT
-
 #define preempt_disable() \
 do { \
 	preempt_count_inc(); \
@@ -231,27 +223,6 @@ do { \
 	__preempt_count_dec(); \
 } while (0)
 
-#else /* !CONFIG_PREEMPT_COUNT */
-
-/*
- * Even if we don't have any preemption, we need preempt disable/enable
- * to be barriers, so that we don't have things like get_user/put_user
- * that can cause faults and scheduling migrate into our preempt-protected
- * region.
- */
-#define preempt_disable()			barrier()
-#define sched_preempt_enable_no_resched()	barrier()
-#define preempt_enable_no_resched()		barrier()
-#define preempt_enable()			barrier()
-#define preempt_check_resched()			do { } while (0)
-
-#define preempt_disable_notrace()		barrier()
-#define preempt_enable_no_resched_notrace()	barrier()
-#define preempt_enable_notrace()		barrier()
-#define preemptible()				0
-
-#endif /* CONFIG_PREEMPT_COUNT */
-
 #ifdef MODULE
 /*
  * Modules have no business playing preemption tricks.



^ permalink raw reply	[flat|nested] 282+ messages in thread

* [patch 03/13] preempt: Clenaup PREEMPT_COUNT leftovers
@ 2020-09-14 20:42   ` Thomas Gleixner
  0 siblings, 0 replies; 282+ messages in thread
From: Thomas Gleixner @ 2020-09-14 20:42 UTC (permalink / raw)
  To: LKML
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Joonas Lahtinen, Lai Jiangshan, dri-devel, Ben Segall, linux-mm,
	linux-kselftest, linux-hexagon, Will Deacon, Ingo Molnar,
	Anton Ivanov, linux-arch, Vincent Guittot, Brian Cain,
	Richard Weinberger, Russell King, David Airlie, Ingo Molnar,
	Geert Uytterhoeven, Mel Gorman, intel-gfx, Matt Turner,
	Valentin Schneider, linux-xtensa, Shuah Khan, Paul E. McKenney,
	Jeff Dike, linux-um, Josh Triplett, Steven Rostedt, rcu,
	Linus Torvalds, Ivan Kokshaysky, Jani Nikula, Rodrigo Vivi,
	Dietmar Eggemann, linux-arm-kernel, Richard Henderson,
	Chris Zankel, Max Filippov, linux-m68k, Daniel Vetter,
	linux-alpha, Mathieu Desnoyers, Andrew Morton,
	Daniel Bristot de Oliveira

CONFIG_PREEMPT_COUNT is now unconditionally enabled and will be
removed. Cleanup the leftovers before doing so.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Juri Lelli <juri.lelli@redhat.com>
Cc: Vincent Guittot <vincent.guittot@linaro.org>
Cc: Dietmar Eggemann <dietmar.eggemann@arm.com>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Ben Segall <bsegall@google.com>
Cc: Mel Gorman <mgorman@suse.de>
Cc: Daniel Bristot de Oliveira <bristot@redhat.com>
---
 include/linux/preempt.h |   37 ++++---------------------------------
 1 file changed, 4 insertions(+), 33 deletions(-)

--- a/include/linux/preempt.h
+++ b/include/linux/preempt.h
@@ -56,8 +56,7 @@
 #define PREEMPT_DISABLED	(PREEMPT_DISABLE_OFFSET + PREEMPT_ENABLED)
 
 /*
- * Disable preemption until the scheduler is running -- use an unconditional
- * value so that it also works on !PREEMPT_COUNT kernels.
+ * Disable preemption until the scheduler is running.
  *
  * Reset by start_kernel()->sched_init()->init_idle()->init_idle_preempt_count().
  */
@@ -69,7 +68,6 @@
  *
  *    preempt_count() == 2*PREEMPT_DISABLE_OFFSET
  *
- * Note: PREEMPT_DISABLE_OFFSET is 0 for !PREEMPT_COUNT kernels.
  * Note: See finish_task_switch().
  */
 #define FORK_PREEMPT_COUNT	(2*PREEMPT_DISABLE_OFFSET + PREEMPT_ENABLED)
@@ -106,11 +104,7 @@
 /*
  * The preempt_count offset after preempt_disable();
  */
-#if defined(CONFIG_PREEMPT_COUNT)
-# define PREEMPT_DISABLE_OFFSET	PREEMPT_OFFSET
-#else
-# define PREEMPT_DISABLE_OFFSET	0
-#endif
+#define PREEMPT_DISABLE_OFFSET	PREEMPT_OFFSET
 
 /*
  * The preempt_count offset after spin_lock()
@@ -122,8 +116,8 @@
  *
  *  spin_lock_bh()
  *
- * Which need to disable both preemption (CONFIG_PREEMPT_COUNT) and
- * softirqs, such that unlock sequences of:
+ * Which need to disable both preemption and softirqs, such that unlock
+ * sequences of:
  *
  *  spin_unlock();
  *  local_bh_enable();
@@ -164,8 +158,6 @@ extern void preempt_count_sub(int val);
 #define preempt_count_inc() preempt_count_add(1)
 #define preempt_count_dec() preempt_count_sub(1)
 
-#ifdef CONFIG_PREEMPT_COUNT
-
 #define preempt_disable() \
 do { \
 	preempt_count_inc(); \
@@ -231,27 +223,6 @@ do { \
 	__preempt_count_dec(); \
 } while (0)
 
-#else /* !CONFIG_PREEMPT_COUNT */
-
-/*
- * Even if we don't have any preemption, we need preempt disable/enable
- * to be barriers, so that we don't have things like get_user/put_user
- * that can cause faults and scheduling migrate into our preempt-protected
- * region.
- */
-#define preempt_disable()			barrier()
-#define sched_preempt_enable_no_resched()	barrier()
-#define preempt_enable_no_resched()		barrier()
-#define preempt_enable()			barrier()
-#define preempt_check_resched()			do { } while (0)
-
-#define preempt_disable_notrace()		barrier()
-#define preempt_enable_no_resched_notrace()	barrier()
-#define preempt_enable_notrace()		barrier()
-#define preemptible()				0
-
-#endif /* CONFIG_PREEMPT_COUNT */
-
 #ifdef MODULE
 /*
  * Modules have no business playing preemption tricks.


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 282+ messages in thread

* [patch 03/13] preempt: Clenaup PREEMPT_COUNT leftovers
@ 2020-09-14 20:42   ` Thomas Gleixner
  0 siblings, 0 replies; 282+ messages in thread
From: Thomas Gleixner @ 2020-09-14 20:42 UTC (permalink / raw)
  To: LKML
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, linux-mm, linux-kselftest,
	linux-hexagon, Will Deacon, Ingo Molnar, Anton Ivanov,
	linux-arch, Vincent Guittot, Brian Cain, Richard Weinberger,
	Russell King, David Airlie, Ingo Molnar, Geert Uytterhoeven,
	Mel Gorman, intel-gfx, Matt Turner, Valentin Schneider,
	linux-xtensa, Shuah Khan, Paul E. McKenney, Jeff Dike, linux-um,
	Josh Triplett, Steven Rostedt, rcu, Linus Torvalds,
	Ivan Kokshaysky, Rodrigo Vivi, Dietmar Eggemann,
	linux-arm-kernel, Richard Henderson, Chris Zankel, Max Filippov,
	linux-m68k, linux-alpha, Mathieu Desnoyers, Andrew Morton,
	Daniel Bristot de Oliveira

CONFIG_PREEMPT_COUNT is now unconditionally enabled and will be
removed. Cleanup the leftovers before doing so.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Juri Lelli <juri.lelli@redhat.com>
Cc: Vincent Guittot <vincent.guittot@linaro.org>
Cc: Dietmar Eggemann <dietmar.eggemann@arm.com>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Ben Segall <bsegall@google.com>
Cc: Mel Gorman <mgorman@suse.de>
Cc: Daniel Bristot de Oliveira <bristot@redhat.com>
---
 include/linux/preempt.h |   37 ++++---------------------------------
 1 file changed, 4 insertions(+), 33 deletions(-)

--- a/include/linux/preempt.h
+++ b/include/linux/preempt.h
@@ -56,8 +56,7 @@
 #define PREEMPT_DISABLED	(PREEMPT_DISABLE_OFFSET + PREEMPT_ENABLED)
 
 /*
- * Disable preemption until the scheduler is running -- use an unconditional
- * value so that it also works on !PREEMPT_COUNT kernels.
+ * Disable preemption until the scheduler is running.
  *
  * Reset by start_kernel()->sched_init()->init_idle()->init_idle_preempt_count().
  */
@@ -69,7 +68,6 @@
  *
  *    preempt_count() == 2*PREEMPT_DISABLE_OFFSET
  *
- * Note: PREEMPT_DISABLE_OFFSET is 0 for !PREEMPT_COUNT kernels.
  * Note: See finish_task_switch().
  */
 #define FORK_PREEMPT_COUNT	(2*PREEMPT_DISABLE_OFFSET + PREEMPT_ENABLED)
@@ -106,11 +104,7 @@
 /*
  * The preempt_count offset after preempt_disable();
  */
-#if defined(CONFIG_PREEMPT_COUNT)
-# define PREEMPT_DISABLE_OFFSET	PREEMPT_OFFSET
-#else
-# define PREEMPT_DISABLE_OFFSET	0
-#endif
+#define PREEMPT_DISABLE_OFFSET	PREEMPT_OFFSET
 
 /*
  * The preempt_count offset after spin_lock()
@@ -122,8 +116,8 @@
  *
  *  spin_lock_bh()
  *
- * Which need to disable both preemption (CONFIG_PREEMPT_COUNT) and
- * softirqs, such that unlock sequences of:
+ * Which need to disable both preemption and softirqs, such that unlock
+ * sequences of:
  *
  *  spin_unlock();
  *  local_bh_enable();
@@ -164,8 +158,6 @@ extern void preempt_count_sub(int val);
 #define preempt_count_inc() preempt_count_add(1)
 #define preempt_count_dec() preempt_count_sub(1)
 
-#ifdef CONFIG_PREEMPT_COUNT
-
 #define preempt_disable() \
 do { \
 	preempt_count_inc(); \
@@ -231,27 +223,6 @@ do { \
 	__preempt_count_dec(); \
 } while (0)
 
-#else /* !CONFIG_PREEMPT_COUNT */
-
-/*
- * Even if we don't have any preemption, we need preempt disable/enable
- * to be barriers, so that we don't have things like get_user/put_user
- * that can cause faults and scheduling migrate into our preempt-protected
- * region.
- */
-#define preempt_disable()			barrier()
-#define sched_preempt_enable_no_resched()	barrier()
-#define preempt_enable_no_resched()		barrier()
-#define preempt_enable()			barrier()
-#define preempt_check_resched()			do { } while (0)
-
-#define preempt_disable_notrace()		barrier()
-#define preempt_enable_no_resched_notrace()	barrier()
-#define preempt_enable_notrace()		barrier()
-#define preemptible()				0
-
-#endif /* CONFIG_PREEMPT_COUNT */
-
 #ifdef MODULE
 /*
  * Modules have no business playing preemption tricks.

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply	[flat|nested] 282+ messages in thread

* [Intel-gfx] [patch 03/13] preempt: Clenaup PREEMPT_COUNT leftovers
@ 2020-09-14 20:42   ` Thomas Gleixner
  0 siblings, 0 replies; 282+ messages in thread
From: Thomas Gleixner @ 2020-09-14 20:42 UTC (permalink / raw)
  To: LKML
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, linux-mm, linux-kselftest,
	linux-hexagon, Will Deacon, Ingo Molnar, Anton Ivanov,
	linux-arch, Brian Cain, Richard Weinberger, Russell King,
	David Airlie, Ingo Molnar, Geert Uytterhoeven, Mel Gorman,
	intel-gfx, Matt Turner, Valentin Schneider, linux-xtensa,
	Shuah Khan, Paul E. McKenney, Jeff Dike, linux-um, Josh Triplett,
	Steven Rostedt, rcu, Linus Torvalds, Ivan Kokshaysky,
	Dietmar Eggemann, linux-arm-kernel, Richard Henderson,
	Chris Zankel, Max Filippov, linux-m68k, linux-alpha,
	Mathieu Desnoyers, Andrew Morton, Daniel Bristot de Oliveira

CONFIG_PREEMPT_COUNT is now unconditionally enabled and will be
removed. Cleanup the leftovers before doing so.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Juri Lelli <juri.lelli@redhat.com>
Cc: Vincent Guittot <vincent.guittot@linaro.org>
Cc: Dietmar Eggemann <dietmar.eggemann@arm.com>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Ben Segall <bsegall@google.com>
Cc: Mel Gorman <mgorman@suse.de>
Cc: Daniel Bristot de Oliveira <bristot@redhat.com>
---
 include/linux/preempt.h |   37 ++++---------------------------------
 1 file changed, 4 insertions(+), 33 deletions(-)

--- a/include/linux/preempt.h
+++ b/include/linux/preempt.h
@@ -56,8 +56,7 @@
 #define PREEMPT_DISABLED	(PREEMPT_DISABLE_OFFSET + PREEMPT_ENABLED)
 
 /*
- * Disable preemption until the scheduler is running -- use an unconditional
- * value so that it also works on !PREEMPT_COUNT kernels.
+ * Disable preemption until the scheduler is running.
  *
  * Reset by start_kernel()->sched_init()->init_idle()->init_idle_preempt_count().
  */
@@ -69,7 +68,6 @@
  *
  *    preempt_count() == 2*PREEMPT_DISABLE_OFFSET
  *
- * Note: PREEMPT_DISABLE_OFFSET is 0 for !PREEMPT_COUNT kernels.
  * Note: See finish_task_switch().
  */
 #define FORK_PREEMPT_COUNT	(2*PREEMPT_DISABLE_OFFSET + PREEMPT_ENABLED)
@@ -106,11 +104,7 @@
 /*
  * The preempt_count offset after preempt_disable();
  */
-#if defined(CONFIG_PREEMPT_COUNT)
-# define PREEMPT_DISABLE_OFFSET	PREEMPT_OFFSET
-#else
-# define PREEMPT_DISABLE_OFFSET	0
-#endif
+#define PREEMPT_DISABLE_OFFSET	PREEMPT_OFFSET
 
 /*
  * The preempt_count offset after spin_lock()
@@ -122,8 +116,8 @@
  *
  *  spin_lock_bh()
  *
- * Which need to disable both preemption (CONFIG_PREEMPT_COUNT) and
- * softirqs, such that unlock sequences of:
+ * Which need to disable both preemption and softirqs, such that unlock
+ * sequences of:
  *
  *  spin_unlock();
  *  local_bh_enable();
@@ -164,8 +158,6 @@ extern void preempt_count_sub(int val);
 #define preempt_count_inc() preempt_count_add(1)
 #define preempt_count_dec() preempt_count_sub(1)
 
-#ifdef CONFIG_PREEMPT_COUNT
-
 #define preempt_disable() \
 do { \
 	preempt_count_inc(); \
@@ -231,27 +223,6 @@ do { \
 	__preempt_count_dec(); \
 } while (0)
 
-#else /* !CONFIG_PREEMPT_COUNT */
-
-/*
- * Even if we don't have any preemption, we need preempt disable/enable
- * to be barriers, so that we don't have things like get_user/put_user
- * that can cause faults and scheduling migrate into our preempt-protected
- * region.
- */
-#define preempt_disable()			barrier()
-#define sched_preempt_enable_no_resched()	barrier()
-#define preempt_enable_no_resched()		barrier()
-#define preempt_enable()			barrier()
-#define preempt_check_resched()			do { } while (0)
-
-#define preempt_disable_notrace()		barrier()
-#define preempt_enable_no_resched_notrace()	barrier()
-#define preempt_enable_notrace()		barrier()
-#define preemptible()				0
-
-#endif /* CONFIG_PREEMPT_COUNT */
-
 #ifdef MODULE
 /*
  * Modules have no business playing preemption tricks.

_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

^ permalink raw reply	[flat|nested] 282+ messages in thread

* [patch 03/13] preempt: Clenaup PREEMPT_COUNT leftovers
@ 2020-09-14 20:42   ` Thomas Gleixner
  0 siblings, 0 replies; 282+ messages in thread
From: Thomas Gleixner @ 2020-09-14 20:42 UTC (permalink / raw)
  To: LKML
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Joonas Lahtinen, Lai Jiangshan, dri-devel, Ben Segall, linux-mm,
	linux-kselftest, linux-hexagon, Will Deacon, Ingo Molnar,
	Anton Ivanov, linux-arch, Vincent Guittot, Brian Cain,
	Richard Weinberger, Russell King, David Airlie, Ingo Molnar,
	Geert Uytterhoeven, Mel Gorman, intel-gfx, Matt Turner, Valentin

CONFIG_PREEMPT_COUNT is now unconditionally enabled and will be
removed. Cleanup the leftovers before doing so.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Juri Lelli <juri.lelli@redhat.com>
Cc: Vincent Guittot <vincent.guittot@linaro.org>
Cc: Dietmar Eggemann <dietmar.eggemann@arm.com>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Ben Segall <bsegall@google.com>
Cc: Mel Gorman <mgorman@suse.de>
Cc: Daniel Bristot de Oliveira <bristot@redhat.com>
---
 include/linux/preempt.h |   37 ++++---------------------------------
 1 file changed, 4 insertions(+), 33 deletions(-)

--- a/include/linux/preempt.h
+++ b/include/linux/preempt.h
@@ -56,8 +56,7 @@
 #define PREEMPT_DISABLED	(PREEMPT_DISABLE_OFFSET + PREEMPT_ENABLED)
 
 /*
- * Disable preemption until the scheduler is running -- use an unconditional
- * value so that it also works on !PREEMPT_COUNT kernels.
+ * Disable preemption until the scheduler is running.
  *
  * Reset by start_kernel()->sched_init()->init_idle()->init_idle_preempt_count().
  */
@@ -69,7 +68,6 @@
  *
  *    preempt_count() == 2*PREEMPT_DISABLE_OFFSET
  *
- * Note: PREEMPT_DISABLE_OFFSET is 0 for !PREEMPT_COUNT kernels.
  * Note: See finish_task_switch().
  */
 #define FORK_PREEMPT_COUNT	(2*PREEMPT_DISABLE_OFFSET + PREEMPT_ENABLED)
@@ -106,11 +104,7 @@
 /*
  * The preempt_count offset after preempt_disable();
  */
-#if defined(CONFIG_PREEMPT_COUNT)
-# define PREEMPT_DISABLE_OFFSET	PREEMPT_OFFSET
-#else
-# define PREEMPT_DISABLE_OFFSET	0
-#endif
+#define PREEMPT_DISABLE_OFFSET	PREEMPT_OFFSET
 
 /*
  * The preempt_count offset after spin_lock()
@@ -122,8 +116,8 @@
  *
  *  spin_lock_bh()
  *
- * Which need to disable both preemption (CONFIG_PREEMPT_COUNT) and
- * softirqs, such that unlock sequences of:
+ * Which need to disable both preemption and softirqs, such that unlock
+ * sequences of:
  *
  *  spin_unlock();
  *  local_bh_enable();
@@ -164,8 +158,6 @@ extern void preempt_count_sub(int val);
 #define preempt_count_inc() preempt_count_add(1)
 #define preempt_count_dec() preempt_count_sub(1)
 
-#ifdef CONFIG_PREEMPT_COUNT
-
 #define preempt_disable() \
 do { \
 	preempt_count_inc(); \
@@ -231,27 +223,6 @@ do { \
 	__preempt_count_dec(); \
 } while (0)
 
-#else /* !CONFIG_PREEMPT_COUNT */
-
-/*
- * Even if we don't have any preemption, we need preempt disable/enable
- * to be barriers, so that we don't have things like get_user/put_user
- * that can cause faults and scheduling migrate into our preempt-protected
- * region.
- */
-#define preempt_disable()			barrier()
-#define sched_preempt_enable_no_resched()	barrier()
-#define preempt_enable_no_resched()		barrier()
-#define preempt_enable()			barrier()
-#define preempt_check_resched()			do { } while (0)
-
-#define preempt_disable_notrace()		barrier()
-#define preempt_enable_no_resched_notrace()	barrier()
-#define preempt_enable_notrace()		barrier()
-#define preemptible()				0
-
-#endif /* CONFIG_PREEMPT_COUNT */
-
 #ifdef MODULE
 /*
  * Modules have no business playing preemption tricks.

^ permalink raw reply	[flat|nested] 282+ messages in thread

* [patch 04/13] lockdep: Clenaup PREEMPT_COUNT leftovers
  2020-09-14 20:42 ` Thomas Gleixner
                     ` (2 preceding siblings ...)
  (?)
@ 2020-09-14 20:42   ` Thomas Gleixner
  -1 siblings, 0 replies; 282+ messages in thread
From: Thomas Gleixner @ 2020-09-14 20:42 UTC (permalink / raw)
  To: LKML
  Cc: linux-arch, Linus Torvalds, Sebastian Andrzej Siewior,
	Valentin Schneider, Peter Zijlstra, Ingo Molnar, Will Deacon,
	Richard Henderson, Ivan Kokshaysky, Matt Turner, linux-alpha,
	Jeff Dike, Richard Weinberger, Anton Ivanov, linux-um,
	Brian Cain, linux-hexagon, Geert Uytterhoeven, linux-m68k,
	Juri Lelli, Vincent Guittot, Dietmar Eggemann, Steven Rostedt,
	Ben Segall, Mel Gorman, Daniel Bristot de Oliveira,
	Andrew Morton, linux-mm, Ingo Molnar, Russell King,
	linux-arm-kernel, Chris Zankel, Max Filippov, linux-xtensa,
	Jani Nikula, Joonas Lahtinen, Rodrigo Vivi, David Airlie,
	Daniel Vetter, intel-gfx, dri-devel, Paul E. McKenney,
	Josh Triplett, Mathieu Desnoyers, Lai Jiangshan, Shuah Khan, rcu,
	linux-kselftest

CONFIG_PREEMPT_COUNT is now unconditionally enabled and will be
removed. Cleanup the leftovers before doing so.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Will Deacon <will@kernel.org>
---
 include/linux/lockdep.h |    6 ++----
 lib/Kconfig.debug       |    1 -
 2 files changed, 2 insertions(+), 5 deletions(-)

--- a/include/linux/lockdep.h
+++ b/include/linux/lockdep.h
@@ -585,16 +585,14 @@ do {									\
 
 #define lockdep_assert_preemption_enabled()				\
 do {									\
-	WARN_ON_ONCE(IS_ENABLED(CONFIG_PREEMPT_COUNT)	&&		\
-		     debug_locks			&&		\
+	WARN_ON_ONCE(debug_locks			&&		\
 		     (preempt_count() != 0		||		\
 		      !raw_cpu_read(hardirqs_enabled)));		\
 } while (0)
 
 #define lockdep_assert_preemption_disabled()				\
 do {									\
-	WARN_ON_ONCE(IS_ENABLED(CONFIG_PREEMPT_COUNT)	&&		\
-		     debug_locks			&&		\
+	WARN_ON_ONCE(debug_locks			&&		\
 		     (preempt_count() == 0		&&		\
 		      raw_cpu_read(hardirqs_enabled)));			\
 } while (0)
--- a/lib/Kconfig.debug
+++ b/lib/Kconfig.debug
@@ -1161,7 +1161,6 @@ config PROVE_LOCKING
 	select DEBUG_RWSEMS
 	select DEBUG_WW_MUTEX_SLOWPATH
 	select DEBUG_LOCK_ALLOC
-	select PREEMPT_COUNT
 	select TRACE_IRQFLAGS
 	default n
 	help



^ permalink raw reply	[flat|nested] 282+ messages in thread

* [patch 04/13] lockdep: Clenaup PREEMPT_COUNT leftovers
@ 2020-09-14 20:42   ` Thomas Gleixner
  0 siblings, 0 replies; 282+ messages in thread
From: Thomas Gleixner @ 2020-09-14 20:42 UTC (permalink / raw)
  To: LKML
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Joonas Lahtinen, Lai Jiangshan, dri-devel, Ben Segall, linux-mm,
	linux-kselftest, linux-hexagon, Will Deacon, Ingo Molnar,
	Anton Ivanov, linux-arch, Vincent Guittot, Brian Cain,
	Richard Weinberger, Russell King, David Airlie, Ingo Molnar,
	Geert Uytterhoeven, Mel Gorman, intel-gfx, Matt Turner,
	Valentin Schneider, linux-xtensa, Shuah Khan, Paul E. McKenney,
	Jeff Dike, linux-um, Josh Triplett, Steven Rostedt, rcu,
	linux-m68k, Ivan Kokshaysky, Jani Nikula, Rodrigo Vivi,
	Dietmar Eggemann, linux-arm-kernel, Richard Henderson,
	Chris Zankel, Max Filippov, Linus Torvalds, Daniel Vetter,
	linux-alpha, Mathieu Desnoyers, Andrew Morton,
	Daniel Bristot de Oliveira

CONFIG_PREEMPT_COUNT is now unconditionally enabled and will be
removed. Cleanup the leftovers before doing so.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Will Deacon <will@kernel.org>
---
 include/linux/lockdep.h |    6 ++----
 lib/Kconfig.debug       |    1 -
 2 files changed, 2 insertions(+), 5 deletions(-)

--- a/include/linux/lockdep.h
+++ b/include/linux/lockdep.h
@@ -585,16 +585,14 @@ do {									\
 
 #define lockdep_assert_preemption_enabled()				\
 do {									\
-	WARN_ON_ONCE(IS_ENABLED(CONFIG_PREEMPT_COUNT)	&&		\
-		     debug_locks			&&		\
+	WARN_ON_ONCE(debug_locks			&&		\
 		     (preempt_count() != 0		||		\
 		      !raw_cpu_read(hardirqs_enabled)));		\
 } while (0)
 
 #define lockdep_assert_preemption_disabled()				\
 do {									\
-	WARN_ON_ONCE(IS_ENABLED(CONFIG_PREEMPT_COUNT)	&&		\
-		     debug_locks			&&		\
+	WARN_ON_ONCE(debug_locks			&&		\
 		     (preempt_count() == 0		&&		\
 		      raw_cpu_read(hardirqs_enabled)));			\
 } while (0)
--- a/lib/Kconfig.debug
+++ b/lib/Kconfig.debug
@@ -1161,7 +1161,6 @@ config PROVE_LOCKING
 	select DEBUG_RWSEMS
 	select DEBUG_WW_MUTEX_SLOWPATH
 	select DEBUG_LOCK_ALLOC
-	select PREEMPT_COUNT
 	select TRACE_IRQFLAGS
 	default n
 	help


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 282+ messages in thread

* [patch 04/13] lockdep: Clenaup PREEMPT_COUNT leftovers
@ 2020-09-14 20:42   ` Thomas Gleixner
  0 siblings, 0 replies; 282+ messages in thread
From: Thomas Gleixner @ 2020-09-14 20:42 UTC (permalink / raw)
  To: LKML
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, linux-mm, linux-kselftest,
	linux-hexagon, Will Deacon, Ingo Molnar, Anton Ivanov,
	linux-arch, Vincent Guittot, Brian Cain, Richard Weinberger,
	Russell King, David Airlie, Ingo Molnar, Geert Uytterhoeven,
	Mel Gorman, intel-gfx, Matt Turner, Valentin Schneider,
	linux-xtensa, Shuah Khan, Paul E. McKenney, Jeff Dike, linux-um,
	Josh Triplett, Steven Rostedt, rcu, linux-m68k, Ivan Kokshaysky,
	Rodrigo Vivi, Dietmar Eggemann, linux-arm-kernel,
	Richard Henderson, Chris Zankel, Max Filippov, Linus Torvalds,
	linux-alpha, Mathieu Desnoyers, Andrew Morton,
	Daniel Bristot de Oliveira

CONFIG_PREEMPT_COUNT is now unconditionally enabled and will be
removed. Cleanup the leftovers before doing so.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Will Deacon <will@kernel.org>
---
 include/linux/lockdep.h |    6 ++----
 lib/Kconfig.debug       |    1 -
 2 files changed, 2 insertions(+), 5 deletions(-)

--- a/include/linux/lockdep.h
+++ b/include/linux/lockdep.h
@@ -585,16 +585,14 @@ do {									\
 
 #define lockdep_assert_preemption_enabled()				\
 do {									\
-	WARN_ON_ONCE(IS_ENABLED(CONFIG_PREEMPT_COUNT)	&&		\
-		     debug_locks			&&		\
+	WARN_ON_ONCE(debug_locks			&&		\
 		     (preempt_count() != 0		||		\
 		      !raw_cpu_read(hardirqs_enabled)));		\
 } while (0)
 
 #define lockdep_assert_preemption_disabled()				\
 do {									\
-	WARN_ON_ONCE(IS_ENABLED(CONFIG_PREEMPT_COUNT)	&&		\
-		     debug_locks			&&		\
+	WARN_ON_ONCE(debug_locks			&&		\
 		     (preempt_count() == 0		&&		\
 		      raw_cpu_read(hardirqs_enabled)));			\
 } while (0)
--- a/lib/Kconfig.debug
+++ b/lib/Kconfig.debug
@@ -1161,7 +1161,6 @@ config PROVE_LOCKING
 	select DEBUG_RWSEMS
 	select DEBUG_WW_MUTEX_SLOWPATH
 	select DEBUG_LOCK_ALLOC
-	select PREEMPT_COUNT
 	select TRACE_IRQFLAGS
 	default n
 	help

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply	[flat|nested] 282+ messages in thread

* [Intel-gfx] [patch 04/13] lockdep: Clenaup PREEMPT_COUNT leftovers
@ 2020-09-14 20:42   ` Thomas Gleixner
  0 siblings, 0 replies; 282+ messages in thread
From: Thomas Gleixner @ 2020-09-14 20:42 UTC (permalink / raw)
  To: LKML
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, linux-mm, linux-kselftest,
	linux-hexagon, Will Deacon, Ingo Molnar, Anton Ivanov,
	linux-arch, Brian Cain, Richard Weinberger, Russell King,
	David Airlie, Ingo Molnar, Geert Uytterhoeven, Mel Gorman,
	intel-gfx, Matt Turner, Valentin Schneider, linux-xtensa,
	Shuah Khan, Paul E. McKenney, Jeff Dike, linux-um, Josh Triplett,
	Steven Rostedt, rcu, linux-m68k, Ivan Kokshaysky,
	Dietmar Eggemann, linux-arm-kernel, Richard Henderson,
	Chris Zankel, Max Filippov, Linus Torvalds, linux-alpha,
	Mathieu Desnoyers, Andrew Morton, Daniel Bristot de Oliveira

CONFIG_PREEMPT_COUNT is now unconditionally enabled and will be
removed. Cleanup the leftovers before doing so.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Will Deacon <will@kernel.org>
---
 include/linux/lockdep.h |    6 ++----
 lib/Kconfig.debug       |    1 -
 2 files changed, 2 insertions(+), 5 deletions(-)

--- a/include/linux/lockdep.h
+++ b/include/linux/lockdep.h
@@ -585,16 +585,14 @@ do {									\
 
 #define lockdep_assert_preemption_enabled()				\
 do {									\
-	WARN_ON_ONCE(IS_ENABLED(CONFIG_PREEMPT_COUNT)	&&		\
-		     debug_locks			&&		\
+	WARN_ON_ONCE(debug_locks			&&		\
 		     (preempt_count() != 0		||		\
 		      !raw_cpu_read(hardirqs_enabled)));		\
 } while (0)
 
 #define lockdep_assert_preemption_disabled()				\
 do {									\
-	WARN_ON_ONCE(IS_ENABLED(CONFIG_PREEMPT_COUNT)	&&		\
-		     debug_locks			&&		\
+	WARN_ON_ONCE(debug_locks			&&		\
 		     (preempt_count() == 0		&&		\
 		      raw_cpu_read(hardirqs_enabled)));			\
 } while (0)
--- a/lib/Kconfig.debug
+++ b/lib/Kconfig.debug
@@ -1161,7 +1161,6 @@ config PROVE_LOCKING
 	select DEBUG_RWSEMS
 	select DEBUG_WW_MUTEX_SLOWPATH
 	select DEBUG_LOCK_ALLOC
-	select PREEMPT_COUNT
 	select TRACE_IRQFLAGS
 	default n
 	help

_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

^ permalink raw reply	[flat|nested] 282+ messages in thread

* [patch 04/13] lockdep: Clenaup PREEMPT_COUNT leftovers
@ 2020-09-14 20:42   ` Thomas Gleixner
  0 siblings, 0 replies; 282+ messages in thread
From: Thomas Gleixner @ 2020-09-14 20:42 UTC (permalink / raw)
  To: LKML
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Joonas Lahtinen, Lai Jiangshan, dri-devel, Ben Segall, linux-mm,
	linux-kselftest, linux-hexagon, Will Deacon, Ingo Molnar,
	Anton Ivanov, linux-arch, Vincent Guittot, Brian Cain,
	Richard Weinberger, Russell King, David Airlie, Ingo Molnar,
	Geert Uytterhoeven, Mel Gorman, intel-gfx, Matt Turner, Valentin

CONFIG_PREEMPT_COUNT is now unconditionally enabled and will be
removed. Cleanup the leftovers before doing so.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Will Deacon <will@kernel.org>
---
 include/linux/lockdep.h |    6 ++----
 lib/Kconfig.debug       |    1 -
 2 files changed, 2 insertions(+), 5 deletions(-)

--- a/include/linux/lockdep.h
+++ b/include/linux/lockdep.h
@@ -585,16 +585,14 @@ do {									\
 
 #define lockdep_assert_preemption_enabled()				\
 do {									\
-	WARN_ON_ONCE(IS_ENABLED(CONFIG_PREEMPT_COUNT)	&&		\
-		     debug_locks			&&		\
+	WARN_ON_ONCE(debug_locks			&&		\
 		     (preempt_count() != 0		||		\
 		      !raw_cpu_read(hardirqs_enabled)));		\
 } while (0)
 
 #define lockdep_assert_preemption_disabled()				\
 do {									\
-	WARN_ON_ONCE(IS_ENABLED(CONFIG_PREEMPT_COUNT)	&&		\
-		     debug_locks			&&		\
+	WARN_ON_ONCE(debug_locks			&&		\
 		     (preempt_count() == 0		&&		\
 		      raw_cpu_read(hardirqs_enabled)));			\
 } while (0)
--- a/lib/Kconfig.debug
+++ b/lib/Kconfig.debug
@@ -1161,7 +1161,6 @@ config PROVE_LOCKING
 	select DEBUG_RWSEMS
 	select DEBUG_WW_MUTEX_SLOWPATH
 	select DEBUG_LOCK_ALLOC
-	select PREEMPT_COUNT
 	select TRACE_IRQFLAGS
 	default n
 	help

^ permalink raw reply	[flat|nested] 282+ messages in thread

* [patch 05/13] mm/pagemap: Clenaup PREEMPT_COUNT leftovers
  2020-09-14 20:42 ` Thomas Gleixner
                     ` (2 preceding siblings ...)
  (?)
@ 2020-09-14 20:42   ` Thomas Gleixner
  -1 siblings, 0 replies; 282+ messages in thread
From: Thomas Gleixner @ 2020-09-14 20:42 UTC (permalink / raw)
  To: LKML
  Cc: linux-arch, Linus Torvalds, Sebastian Andrzej Siewior,
	Valentin Schneider, Andrew Morton, linux-mm, Richard Henderson,
	Ivan Kokshaysky, Matt Turner, linux-alpha, Jeff Dike,
	Richard Weinberger, Anton Ivanov, linux-um, Brian Cain,
	linux-hexagon, Geert Uytterhoeven, linux-m68k, Ingo Molnar,
	Peter Zijlstra, Juri Lelli, Vincent Guittot, Dietmar Eggemann,
	Steven Rostedt, Ben Segall, Mel Gorman,
	Daniel Bristot de Oliveira, Will Deacon, Ingo Molnar,
	Russell King, linux-arm-kernel, Chris Zankel, Max Filippov,
	linux-xtensa, Jani Nikula, Joonas Lahtinen, Rodrigo Vivi,
	David Airlie, Daniel Vetter, intel-gfx, dri-devel,
	Paul E. McKenney, Josh Triplett, Mathieu Desnoyers,
	Lai Jiangshan, Shuah Khan, rcu, linux-kselftest

CONFIG_PREEMPT_COUNT is now unconditionally enabled and will be
removed. Cleanup the leftovers before doing so.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: linux-mm@kvack.org
---
 include/linux/pagemap.h |    4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)

--- a/include/linux/pagemap.h
+++ b/include/linux/pagemap.h
@@ -168,9 +168,7 @@ void release_pages(struct page **pages,
 static inline int __page_cache_add_speculative(struct page *page, int count)
 {
 #ifdef CONFIG_TINY_RCU
-# ifdef CONFIG_PREEMPT_COUNT
-	VM_BUG_ON(!in_atomic() && !irqs_disabled());
-# endif
+	VM_BUG_ON(preemptible())
 	/*
 	 * Preempt must be disabled here - we rely on rcu_read_lock doing
 	 * this for us.



^ permalink raw reply	[flat|nested] 282+ messages in thread

* [patch 05/13] mm/pagemap: Clenaup PREEMPT_COUNT leftovers
@ 2020-09-14 20:42   ` Thomas Gleixner
  0 siblings, 0 replies; 282+ messages in thread
From: Thomas Gleixner @ 2020-09-14 20:42 UTC (permalink / raw)
  To: LKML
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Joonas Lahtinen, Lai Jiangshan, dri-devel, Ben Segall, linux-mm,
	linux-kselftest, linux-hexagon, Will Deacon, Ingo Molnar,
	Anton Ivanov, linux-arch, Vincent Guittot, Brian Cain,
	Richard Weinberger, Russell King, David Airlie, Ingo Molnar,
	Geert Uytterhoeven, Mel Gorman, intel-gfx, Matt Turner,
	Valentin Schneider, linux-xtensa, Shuah Khan, Paul E. McKenney,
	Jeff Dike, linux-um, Josh Triplett, Steven Rostedt, rcu,
	linux-m68k, Ivan Kokshaysky, Jani Nikula, Rodrigo Vivi,
	Dietmar Eggemann, linux-arm-kernel, Richard Henderson,
	Chris Zankel, Max Filippov, Linus Torvalds, Daniel Vetter,
	linux-alpha, Mathieu Desnoyers, Andrew Morton,
	Daniel Bristot de Oliveira

CONFIG_PREEMPT_COUNT is now unconditionally enabled and will be
removed. Cleanup the leftovers before doing so.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: linux-mm@kvack.org
---
 include/linux/pagemap.h |    4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)

--- a/include/linux/pagemap.h
+++ b/include/linux/pagemap.h
@@ -168,9 +168,7 @@ void release_pages(struct page **pages,
 static inline int __page_cache_add_speculative(struct page *page, int count)
 {
 #ifdef CONFIG_TINY_RCU
-# ifdef CONFIG_PREEMPT_COUNT
-	VM_BUG_ON(!in_atomic() && !irqs_disabled());
-# endif
+	VM_BUG_ON(preemptible())
 	/*
 	 * Preempt must be disabled here - we rely on rcu_read_lock doing
 	 * this for us.


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 282+ messages in thread

* [patch 05/13] mm/pagemap: Clenaup PREEMPT_COUNT leftovers
@ 2020-09-14 20:42   ` Thomas Gleixner
  0 siblings, 0 replies; 282+ messages in thread
From: Thomas Gleixner @ 2020-09-14 20:42 UTC (permalink / raw)
  To: LKML
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, linux-mm, linux-kselftest,
	linux-hexagon, Will Deacon, Ingo Molnar, Anton Ivanov,
	linux-arch, Vincent Guittot, Brian Cain, Richard Weinberger,
	Russell King, David Airlie, Ingo Molnar, Geert Uytterhoeven,
	Mel Gorman, intel-gfx, Matt Turner, Valentin Schneider,
	linux-xtensa, Shuah Khan, Paul E. McKenney, Jeff Dike, linux-um,
	Josh Triplett, Steven Rostedt, rcu, linux-m68k, Ivan Kokshaysky,
	Rodrigo Vivi, Dietmar Eggemann, linux-arm-kernel,
	Richard Henderson, Chris Zankel, Max Filippov, Linus Torvalds,
	linux-alpha, Mathieu Desnoyers, Andrew Morton,
	Daniel Bristot de Oliveira

CONFIG_PREEMPT_COUNT is now unconditionally enabled and will be
removed. Cleanup the leftovers before doing so.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: linux-mm@kvack.org
---
 include/linux/pagemap.h |    4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)

--- a/include/linux/pagemap.h
+++ b/include/linux/pagemap.h
@@ -168,9 +168,7 @@ void release_pages(struct page **pages,
 static inline int __page_cache_add_speculative(struct page *page, int count)
 {
 #ifdef CONFIG_TINY_RCU
-# ifdef CONFIG_PREEMPT_COUNT
-	VM_BUG_ON(!in_atomic() && !irqs_disabled());
-# endif
+	VM_BUG_ON(preemptible())
 	/*
 	 * Preempt must be disabled here - we rely on rcu_read_lock doing
 	 * this for us.

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply	[flat|nested] 282+ messages in thread

* [Intel-gfx] [patch 05/13] mm/pagemap: Clenaup PREEMPT_COUNT leftovers
@ 2020-09-14 20:42   ` Thomas Gleixner
  0 siblings, 0 replies; 282+ messages in thread
From: Thomas Gleixner @ 2020-09-14 20:42 UTC (permalink / raw)
  To: LKML
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, linux-mm, linux-kselftest,
	linux-hexagon, Will Deacon, Ingo Molnar, Anton Ivanov,
	linux-arch, Brian Cain, Richard Weinberger, Russell King,
	David Airlie, Ingo Molnar, Geert Uytterhoeven, Mel Gorman,
	intel-gfx, Matt Turner, Valentin Schneider, linux-xtensa,
	Shuah Khan, Paul E. McKenney, Jeff Dike, linux-um, Josh Triplett,
	Steven Rostedt, rcu, linux-m68k, Ivan Kokshaysky,
	Dietmar Eggemann, linux-arm-kernel, Richard Henderson,
	Chris Zankel, Max Filippov, Linus Torvalds, linux-alpha,
	Mathieu Desnoyers, Andrew Morton, Daniel Bristot de Oliveira

CONFIG_PREEMPT_COUNT is now unconditionally enabled and will be
removed. Cleanup the leftovers before doing so.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: linux-mm@kvack.org
---
 include/linux/pagemap.h |    4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)

--- a/include/linux/pagemap.h
+++ b/include/linux/pagemap.h
@@ -168,9 +168,7 @@ void release_pages(struct page **pages,
 static inline int __page_cache_add_speculative(struct page *page, int count)
 {
 #ifdef CONFIG_TINY_RCU
-# ifdef CONFIG_PREEMPT_COUNT
-	VM_BUG_ON(!in_atomic() && !irqs_disabled());
-# endif
+	VM_BUG_ON(preemptible())
 	/*
 	 * Preempt must be disabled here - we rely on rcu_read_lock doing
 	 * this for us.

_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

^ permalink raw reply	[flat|nested] 282+ messages in thread

* [patch 05/13] mm/pagemap: Clenaup PREEMPT_COUNT leftovers
@ 2020-09-14 20:42   ` Thomas Gleixner
  0 siblings, 0 replies; 282+ messages in thread
From: Thomas Gleixner @ 2020-09-14 20:42 UTC (permalink / raw)
  To: LKML
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Joonas Lahtinen, Lai Jiangshan, dri-devel, Ben Segall, linux-mm,
	linux-kselftest, linux-hexagon, Will Deacon, Ingo Molnar,
	Anton Ivanov, linux-arch, Vincent Guittot, Brian Cain,
	Richard Weinberger, Russell King, David Airlie, Ingo Molnar,
	Geert Uytterhoeven, Mel Gorman, intel-gfx, Matt Turner, Valentin

CONFIG_PREEMPT_COUNT is now unconditionally enabled and will be
removed. Cleanup the leftovers before doing so.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: linux-mm@kvack.org
---
 include/linux/pagemap.h |    4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)

--- a/include/linux/pagemap.h
+++ b/include/linux/pagemap.h
@@ -168,9 +168,7 @@ void release_pages(struct page **pages,
 static inline int __page_cache_add_speculative(struct page *page, int count)
 {
 #ifdef CONFIG_TINY_RCU
-# ifdef CONFIG_PREEMPT_COUNT
-	VM_BUG_ON(!in_atomic() && !irqs_disabled());
-# endif
+	VM_BUG_ON(preemptible())
 	/*
 	 * Preempt must be disabled here - we rely on rcu_read_lock doing
 	 * this for us.

^ permalink raw reply	[flat|nested] 282+ messages in thread

* [patch 06/13] locking/bitspinlock: Clenaup PREEMPT_COUNT leftovers
  2020-09-14 20:42 ` Thomas Gleixner
                     ` (2 preceding siblings ...)
  (?)
@ 2020-09-14 20:42   ` Thomas Gleixner
  -1 siblings, 0 replies; 282+ messages in thread
From: Thomas Gleixner @ 2020-09-14 20:42 UTC (permalink / raw)
  To: LKML
  Cc: linux-arch, Linus Torvalds, Sebastian Andrzej Siewior,
	Valentin Schneider, Richard Henderson, Ivan Kokshaysky,
	Matt Turner, linux-alpha, Jeff Dike, Richard Weinberger,
	Anton Ivanov, linux-um, Brian Cain, linux-hexagon,
	Geert Uytterhoeven, linux-m68k, Ingo Molnar, Peter Zijlstra,
	Juri Lelli, Vincent Guittot, Dietmar Eggemann, Steven Rostedt,
	Ben Segall, Mel Gorman, Daniel Bristot de Oliveira, Will Deacon,
	Andrew Morton, linux-mm, Ingo Molnar, Russell King,
	linux-arm-kernel, Chris Zankel, Max Filippov, linux-xtensa,
	Jani Nikula, Joonas Lahtinen, Rodrigo Vivi, David Airlie,
	Daniel Vetter, intel-gfx, dri-devel, Paul E. McKenney,
	Josh Triplett, Mathieu Desnoyers, Lai Jiangshan, Shuah Khan, rcu,
	linux-kselftest

CONFIG_PREEMPT_COUNT is now unconditionally enabled and will be
removed. Cleanup the leftovers before doing so.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 include/linux/bit_spinlock.h |    4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)

--- a/include/linux/bit_spinlock.h
+++ b/include/linux/bit_spinlock.h
@@ -90,10 +90,8 @@ static inline int bit_spin_is_locked(int
 {
 #if defined(CONFIG_SMP) || defined(CONFIG_DEBUG_SPINLOCK)
 	return test_bit(bitnum, addr);
-#elif defined CONFIG_PREEMPT_COUNT
-	return preempt_count();
 #else
-	return 1;
+	return preempt_count();
 #endif
 }
 



^ permalink raw reply	[flat|nested] 282+ messages in thread

* [patch 06/13] locking/bitspinlock: Clenaup PREEMPT_COUNT leftovers
@ 2020-09-14 20:42   ` Thomas Gleixner
  0 siblings, 0 replies; 282+ messages in thread
From: Thomas Gleixner @ 2020-09-14 20:42 UTC (permalink / raw)
  To: LKML
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Joonas Lahtinen, Lai Jiangshan, dri-devel, Ben Segall, linux-mm,
	linux-kselftest, linux-hexagon, Will Deacon, Ingo Molnar,
	Anton Ivanov, linux-arch, Vincent Guittot, Brian Cain,
	Richard Weinberger, Russell King, David Airlie, Ingo Molnar,
	Geert Uytterhoeven, Mel Gorman, intel-gfx, Matt Turner,
	Valentin Schneider, linux-xtensa, Shuah Khan, Paul E. McKenney,
	Jeff Dike, linux-um, Josh Triplett, Steven Rostedt, rcu,
	linux-m68k, Ivan Kokshaysky, Jani Nikula, Rodrigo Vivi,
	Dietmar Eggemann, linux-arm-kernel, Richard Henderson,
	Chris Zankel, Max Filippov, Linus Torvalds, Daniel Vetter,
	linux-alpha, Mathieu Desnoyers, Andrew Morton,
	Daniel Bristot de Oliveira

CONFIG_PREEMPT_COUNT is now unconditionally enabled and will be
removed. Cleanup the leftovers before doing so.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 include/linux/bit_spinlock.h |    4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)

--- a/include/linux/bit_spinlock.h
+++ b/include/linux/bit_spinlock.h
@@ -90,10 +90,8 @@ static inline int bit_spin_is_locked(int
 {
 #if defined(CONFIG_SMP) || defined(CONFIG_DEBUG_SPINLOCK)
 	return test_bit(bitnum, addr);
-#elif defined CONFIG_PREEMPT_COUNT
-	return preempt_count();
 #else
-	return 1;
+	return preempt_count();
 #endif
 }
 


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 282+ messages in thread

* [patch 06/13] locking/bitspinlock: Clenaup PREEMPT_COUNT leftovers
@ 2020-09-14 20:42   ` Thomas Gleixner
  0 siblings, 0 replies; 282+ messages in thread
From: Thomas Gleixner @ 2020-09-14 20:42 UTC (permalink / raw)
  To: LKML
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, linux-mm, linux-kselftest,
	linux-hexagon, Will Deacon, Ingo Molnar, Anton Ivanov,
	linux-arch, Vincent Guittot, Brian Cain, Richard Weinberger,
	Russell King, David Airlie, Ingo Molnar, Geert Uytterhoeven,
	Mel Gorman, intel-gfx, Matt Turner, Valentin Schneider,
	linux-xtensa, Shuah Khan, Paul E. McKenney, Jeff Dike, linux-um,
	Josh Triplett, Steven Rostedt, rcu, linux-m68k, Ivan Kokshaysky,
	Rodrigo Vivi, Dietmar Eggemann, linux-arm-kernel,
	Richard Henderson, Chris Zankel, Max Filippov, Linus Torvalds,
	linux-alpha, Mathieu Desnoyers, Andrew Morton,
	Daniel Bristot de Oliveira

CONFIG_PREEMPT_COUNT is now unconditionally enabled and will be
removed. Cleanup the leftovers before doing so.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 include/linux/bit_spinlock.h |    4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)

--- a/include/linux/bit_spinlock.h
+++ b/include/linux/bit_spinlock.h
@@ -90,10 +90,8 @@ static inline int bit_spin_is_locked(int
 {
 #if defined(CONFIG_SMP) || defined(CONFIG_DEBUG_SPINLOCK)
 	return test_bit(bitnum, addr);
-#elif defined CONFIG_PREEMPT_COUNT
-	return preempt_count();
 #else
-	return 1;
+	return preempt_count();
 #endif
 }
 

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply	[flat|nested] 282+ messages in thread

* [Intel-gfx] [patch 06/13] locking/bitspinlock: Clenaup PREEMPT_COUNT leftovers
@ 2020-09-14 20:42   ` Thomas Gleixner
  0 siblings, 0 replies; 282+ messages in thread
From: Thomas Gleixner @ 2020-09-14 20:42 UTC (permalink / raw)
  To: LKML
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, linux-mm, linux-kselftest,
	linux-hexagon, Will Deacon, Ingo Molnar, Anton Ivanov,
	linux-arch, Brian Cain, Richard Weinberger, Russell King,
	David Airlie, Ingo Molnar, Geert Uytterhoeven, Mel Gorman,
	intel-gfx, Matt Turner, Valentin Schneider, linux-xtensa,
	Shuah Khan, Paul E. McKenney, Jeff Dike, linux-um, Josh Triplett,
	Steven Rostedt, rcu, linux-m68k, Ivan Kokshaysky,
	Dietmar Eggemann, linux-arm-kernel, Richard Henderson,
	Chris Zankel, Max Filippov, Linus Torvalds, linux-alpha,
	Mathieu Desnoyers, Andrew Morton, Daniel Bristot de Oliveira

CONFIG_PREEMPT_COUNT is now unconditionally enabled and will be
removed. Cleanup the leftovers before doing so.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 include/linux/bit_spinlock.h |    4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)

--- a/include/linux/bit_spinlock.h
+++ b/include/linux/bit_spinlock.h
@@ -90,10 +90,8 @@ static inline int bit_spin_is_locked(int
 {
 #if defined(CONFIG_SMP) || defined(CONFIG_DEBUG_SPINLOCK)
 	return test_bit(bitnum, addr);
-#elif defined CONFIG_PREEMPT_COUNT
-	return preempt_count();
 #else
-	return 1;
+	return preempt_count();
 #endif
 }
 

_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

^ permalink raw reply	[flat|nested] 282+ messages in thread

* [patch 06/13] locking/bitspinlock: Clenaup PREEMPT_COUNT leftovers
@ 2020-09-14 20:42   ` Thomas Gleixner
  0 siblings, 0 replies; 282+ messages in thread
From: Thomas Gleixner @ 2020-09-14 20:42 UTC (permalink / raw)
  To: LKML
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Joonas Lahtinen, Lai Jiangshan, dri-devel, Ben Segall, linux-mm,
	linux-kselftest, linux-hexagon, Will Deacon, Ingo Molnar,
	Anton Ivanov, linux-arch, Vincent Guittot, Brian Cain,
	Richard Weinberger, Russell King, David Airlie, Ingo Molnar,
	Geert Uytterhoeven, Mel Gorman, intel-gfx, Matt Turner, Valentin

CONFIG_PREEMPT_COUNT is now unconditionally enabled and will be
removed. Cleanup the leftovers before doing so.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 include/linux/bit_spinlock.h |    4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)

--- a/include/linux/bit_spinlock.h
+++ b/include/linux/bit_spinlock.h
@@ -90,10 +90,8 @@ static inline int bit_spin_is_locked(int
 {
 #if defined(CONFIG_SMP) || defined(CONFIG_DEBUG_SPINLOCK)
 	return test_bit(bitnum, addr);
-#elif defined CONFIG_PREEMPT_COUNT
-	return preempt_count();
 #else
-	return 1;
+	return preempt_count();
 #endif
 }

^ permalink raw reply	[flat|nested] 282+ messages in thread

* [patch 07/13] uaccess: Clenaup PREEMPT_COUNT leftovers
  2020-09-14 20:42 ` Thomas Gleixner
                     ` (2 preceding siblings ...)
  (?)
@ 2020-09-14 20:42   ` Thomas Gleixner
  -1 siblings, 0 replies; 282+ messages in thread
From: Thomas Gleixner @ 2020-09-14 20:42 UTC (permalink / raw)
  To: LKML
  Cc: linux-arch, Linus Torvalds, Sebastian Andrzej Siewior,
	Valentin Schneider, Richard Henderson, Ivan Kokshaysky,
	Matt Turner, linux-alpha, Jeff Dike, Richard Weinberger,
	Anton Ivanov, linux-um, Brian Cain, linux-hexagon,
	Geert Uytterhoeven, linux-m68k, Ingo Molnar, Peter Zijlstra,
	Juri Lelli, Vincent Guittot, Dietmar Eggemann, Steven Rostedt,
	Ben Segall, Mel Gorman, Daniel Bristot de Oliveira, Will Deacon,
	Andrew Morton, linux-mm, Ingo Molnar, Russell King,
	linux-arm-kernel, Chris Zankel, Max Filippov, linux-xtensa,
	Jani Nikula, Joonas Lahtinen, Rodrigo Vivi, David Airlie,
	Daniel Vetter, intel-gfx, dri-devel, Paul E. McKenney,
	Josh Triplett, Mathieu Desnoyers, Lai Jiangshan, Shuah Khan, rcu,
	linux-kselftest

CONFIG_PREEMPT_COUNT is now unconditionally enabled and will be
removed. Cleanup the leftovers before doing so.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 include/linux/uaccess.h |    6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

--- a/include/linux/uaccess.h
+++ b/include/linux/uaccess.h
@@ -230,9 +230,9 @@ static inline bool pagefault_disabled(vo
  *
  * This function should only be used by the fault handlers. Other users should
  * stick to pagefault_disabled().
- * Please NEVER use preempt_disable() to disable the fault handler. With
- * !CONFIG_PREEMPT_COUNT, this is like a NOP. So the handler won't be disabled.
- * in_atomic() will report different values based on !CONFIG_PREEMPT_COUNT.
+ *
+ * Please NEVER use preempt_disable() or local_irq_disable() to disable the
+ * fault handler.
  */
 #define faulthandler_disabled() (pagefault_disabled() || in_atomic())
 



^ permalink raw reply	[flat|nested] 282+ messages in thread

* [patch 07/13] uaccess: Clenaup PREEMPT_COUNT leftovers
@ 2020-09-14 20:42   ` Thomas Gleixner
  0 siblings, 0 replies; 282+ messages in thread
From: Thomas Gleixner @ 2020-09-14 20:42 UTC (permalink / raw)
  To: LKML
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Joonas Lahtinen, Lai Jiangshan, dri-devel, Ben Segall, linux-mm,
	linux-kselftest, linux-hexagon, Will Deacon, Ingo Molnar,
	Anton Ivanov, linux-arch, Vincent Guittot, Brian Cain,
	Richard Weinberger, Russell King, David Airlie, Ingo Molnar,
	Geert Uytterhoeven, Mel Gorman, intel-gfx, Matt Turner,
	Valentin Schneider, linux-xtensa, Shuah Khan, Paul E. McKenney,
	Jeff Dike, linux-um, Josh Triplett, Steven Rostedt, rcu,
	linux-m68k, Ivan Kokshaysky, Jani Nikula, Rodrigo Vivi,
	Dietmar Eggemann, linux-arm-kernel, Richard Henderson,
	Chris Zankel, Max Filippov, Linus Torvalds, Daniel Vetter,
	linux-alpha, Mathieu Desnoyers, Andrew Morton,
	Daniel Bristot de Oliveira

CONFIG_PREEMPT_COUNT is now unconditionally enabled and will be
removed. Cleanup the leftovers before doing so.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 include/linux/uaccess.h |    6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

--- a/include/linux/uaccess.h
+++ b/include/linux/uaccess.h
@@ -230,9 +230,9 @@ static inline bool pagefault_disabled(vo
  *
  * This function should only be used by the fault handlers. Other users should
  * stick to pagefault_disabled().
- * Please NEVER use preempt_disable() to disable the fault handler. With
- * !CONFIG_PREEMPT_COUNT, this is like a NOP. So the handler won't be disabled.
- * in_atomic() will report different values based on !CONFIG_PREEMPT_COUNT.
+ *
+ * Please NEVER use preempt_disable() or local_irq_disable() to disable the
+ * fault handler.
  */
 #define faulthandler_disabled() (pagefault_disabled() || in_atomic())
 


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 282+ messages in thread

* [patch 07/13] uaccess: Clenaup PREEMPT_COUNT leftovers
@ 2020-09-14 20:42   ` Thomas Gleixner
  0 siblings, 0 replies; 282+ messages in thread
From: Thomas Gleixner @ 2020-09-14 20:42 UTC (permalink / raw)
  To: LKML
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, linux-mm, linux-kselftest,
	linux-hexagon, Will Deacon, Ingo Molnar, Anton Ivanov,
	linux-arch, Vincent Guittot, Brian Cain, Richard Weinberger,
	Russell King, David Airlie, Ingo Molnar, Geert Uytterhoeven,
	Mel Gorman, intel-gfx, Matt Turner, Valentin Schneider,
	linux-xtensa, Shuah Khan, Paul E. McKenney, Jeff Dike, linux-um,
	Josh Triplett, Steven Rostedt, rcu, linux-m68k, Ivan Kokshaysky,
	Rodrigo Vivi, Dietmar Eggemann, linux-arm-kernel,
	Richard Henderson, Chris Zankel, Max Filippov, Linus Torvalds,
	linux-alpha, Mathieu Desnoyers, Andrew Morton,
	Daniel Bristot de Oliveira

CONFIG_PREEMPT_COUNT is now unconditionally enabled and will be
removed. Cleanup the leftovers before doing so.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 include/linux/uaccess.h |    6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

--- a/include/linux/uaccess.h
+++ b/include/linux/uaccess.h
@@ -230,9 +230,9 @@ static inline bool pagefault_disabled(vo
  *
  * This function should only be used by the fault handlers. Other users should
  * stick to pagefault_disabled().
- * Please NEVER use preempt_disable() to disable the fault handler. With
- * !CONFIG_PREEMPT_COUNT, this is like a NOP. So the handler won't be disabled.
- * in_atomic() will report different values based on !CONFIG_PREEMPT_COUNT.
+ *
+ * Please NEVER use preempt_disable() or local_irq_disable() to disable the
+ * fault handler.
  */
 #define faulthandler_disabled() (pagefault_disabled() || in_atomic())
 

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply	[flat|nested] 282+ messages in thread

* [Intel-gfx] [patch 07/13] uaccess: Clenaup PREEMPT_COUNT leftovers
@ 2020-09-14 20:42   ` Thomas Gleixner
  0 siblings, 0 replies; 282+ messages in thread
From: Thomas Gleixner @ 2020-09-14 20:42 UTC (permalink / raw)
  To: LKML
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, linux-mm, linux-kselftest,
	linux-hexagon, Will Deacon, Ingo Molnar, Anton Ivanov,
	linux-arch, Brian Cain, Richard Weinberger, Russell King,
	David Airlie, Ingo Molnar, Geert Uytterhoeven, Mel Gorman,
	intel-gfx, Matt Turner, Valentin Schneider, linux-xtensa,
	Shuah Khan, Paul E. McKenney, Jeff Dike, linux-um, Josh Triplett,
	Steven Rostedt, rcu, linux-m68k, Ivan Kokshaysky,
	Dietmar Eggemann, linux-arm-kernel, Richard Henderson,
	Chris Zankel, Max Filippov, Linus Torvalds, linux-alpha,
	Mathieu Desnoyers, Andrew Morton, Daniel Bristot de Oliveira

CONFIG_PREEMPT_COUNT is now unconditionally enabled and will be
removed. Cleanup the leftovers before doing so.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 include/linux/uaccess.h |    6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

--- a/include/linux/uaccess.h
+++ b/include/linux/uaccess.h
@@ -230,9 +230,9 @@ static inline bool pagefault_disabled(vo
  *
  * This function should only be used by the fault handlers. Other users should
  * stick to pagefault_disabled().
- * Please NEVER use preempt_disable() to disable the fault handler. With
- * !CONFIG_PREEMPT_COUNT, this is like a NOP. So the handler won't be disabled.
- * in_atomic() will report different values based on !CONFIG_PREEMPT_COUNT.
+ *
+ * Please NEVER use preempt_disable() or local_irq_disable() to disable the
+ * fault handler.
  */
 #define faulthandler_disabled() (pagefault_disabled() || in_atomic())
 

_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

^ permalink raw reply	[flat|nested] 282+ messages in thread

* [patch 07/13] uaccess: Clenaup PREEMPT_COUNT leftovers
@ 2020-09-14 20:42   ` Thomas Gleixner
  0 siblings, 0 replies; 282+ messages in thread
From: Thomas Gleixner @ 2020-09-14 20:42 UTC (permalink / raw)
  To: LKML
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Joonas Lahtinen, Lai Jiangshan, dri-devel, Ben Segall, linux-mm,
	linux-kselftest, linux-hexagon, Will Deacon, Ingo Molnar,
	Anton Ivanov, linux-arch, Vincent Guittot, Brian Cain,
	Richard Weinberger, Russell King, David Airlie, Ingo Molnar,
	Geert Uytterhoeven, Mel Gorman, intel-gfx, Matt Turner, Valentin

CONFIG_PREEMPT_COUNT is now unconditionally enabled and will be
removed. Cleanup the leftovers before doing so.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 include/linux/uaccess.h |    6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

--- a/include/linux/uaccess.h
+++ b/include/linux/uaccess.h
@@ -230,9 +230,9 @@ static inline bool pagefault_disabled(vo
  *
  * This function should only be used by the fault handlers. Other users should
  * stick to pagefault_disabled().
- * Please NEVER use preempt_disable() to disable the fault handler. With
- * !CONFIG_PREEMPT_COUNT, this is like a NOP. So the handler won't be disabled.
- * in_atomic() will report different values based on !CONFIG_PREEMPT_COUNT.
+ *
+ * Please NEVER use preempt_disable() or local_irq_disable() to disable the
+ * fault handler.
  */
 #define faulthandler_disabled() (pagefault_disabled() || in_atomic())

^ permalink raw reply	[flat|nested] 282+ messages in thread

* [patch 08/13] sched: Clenaup PREEMPT_COUNT leftovers
  2020-09-14 20:42 ` Thomas Gleixner
                     ` (2 preceding siblings ...)
  (?)
@ 2020-09-14 20:42   ` Thomas Gleixner
  -1 siblings, 0 replies; 282+ messages in thread
From: Thomas Gleixner @ 2020-09-14 20:42 UTC (permalink / raw)
  To: LKML
  Cc: linux-arch, Linus Torvalds, Sebastian Andrzej Siewior,
	Valentin Schneider, Ingo Molnar, Peter Zijlstra, Juri Lelli,
	Vincent Guittot, Dietmar Eggemann, Steven Rostedt, Ben Segall,
	Mel Gorman, Daniel Bristot de Oliveira, Richard Henderson,
	Ivan Kokshaysky, Matt Turner, linux-alpha, Jeff Dike,
	Richard Weinberger, Anton Ivanov, linux-um, Brian Cain,
	linux-hexagon, Geert Uytterhoeven, linux-m68k, Ingo Molnar,
	Will Deacon, Andrew Morton, linux-mm, Russell King,
	linux-arm-kernel, Chris Zankel, Max Filippov, linux-xtensa,
	Jani Nikula, Joonas Lahtinen, Rodrigo Vivi, David Airlie,
	Daniel Vetter, intel-gfx, dri-devel, Paul E. McKenney,
	Josh Triplett, Mathieu Desnoyers, Lai Jiangshan, Shuah Khan, rcu,
	linux-kselftest

CONFIG_PREEMPT_COUNT is now unconditionally enabled and will be
removed. Cleanup the leftovers before doing so.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Juri Lelli <juri.lelli@redhat.com>
Cc: Vincent Guittot <vincent.guittot@linaro.org>
Cc: Dietmar Eggemann <dietmar.eggemann@arm.com>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Ben Segall <bsegall@google.com>
Cc: Mel Gorman <mgorman@suse.de>
Cc: Daniel Bristot de Oliveira <bristot@redhat.com>
---
 kernel/sched/core.c |    6 +-----
 lib/Kconfig.debug   |    1 -
 2 files changed, 1 insertion(+), 6 deletions(-)

--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -3706,8 +3706,7 @@ asmlinkage __visible void schedule_tail(
 	 * finish_task_switch() for details.
 	 *
 	 * finish_task_switch() will drop rq->lock() and lower preempt_count
-	 * and the preempt_enable() will end up enabling preemption (on
-	 * PREEMPT_COUNT kernels).
+	 * and the preempt_enable() will end up enabling preemption.
 	 */
 
 	rq = finish_task_switch(prev);
@@ -7311,9 +7310,6 @@ void __cant_sleep(const char *file, int
 	if (irqs_disabled())
 		return;
 
-	if (!IS_ENABLED(CONFIG_PREEMPT_COUNT))
-		return;
-
 	if (preempt_count() > preempt_offset)
 		return;
 
--- a/lib/Kconfig.debug
+++ b/lib/Kconfig.debug
@@ -1320,7 +1320,6 @@ config DEBUG_LOCKDEP
 
 config DEBUG_ATOMIC_SLEEP
 	bool "Sleep inside atomic section checking"
-	select PREEMPT_COUNT
 	depends on DEBUG_KERNEL
 	help
 	  If you say Y here, various routines which may sleep will become very



^ permalink raw reply	[flat|nested] 282+ messages in thread

* [patch 08/13] sched: Clenaup PREEMPT_COUNT leftovers
@ 2020-09-14 20:42   ` Thomas Gleixner
  0 siblings, 0 replies; 282+ messages in thread
From: Thomas Gleixner @ 2020-09-14 20:42 UTC (permalink / raw)
  To: LKML
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Joonas Lahtinen, Lai Jiangshan, dri-devel, Ben Segall, linux-mm,
	linux-kselftest, linux-hexagon, Will Deacon, Ingo Molnar,
	Anton Ivanov, linux-arch, Vincent Guittot, Brian Cain,
	Richard Weinberger, Russell King, David Airlie, Ingo Molnar,
	Geert Uytterhoeven, Mel Gorman, intel-gfx, Matt Turner,
	Valentin Schneider, linux-xtensa, Shuah Khan, Paul E. McKenney,
	Jeff Dike, linux-um, Josh Triplett, Steven Rostedt, rcu,
	Linus Torvalds, Ivan Kokshaysky, Jani Nikula, Rodrigo Vivi,
	Dietmar Eggemann, linux-arm-kernel, Richard Henderson,
	Chris Zankel, Max Filippov, linux-m68k, Daniel Vetter,
	linux-alpha, Mathieu Desnoyers, Andrew Morton,
	Daniel Bristot de Oliveira

CONFIG_PREEMPT_COUNT is now unconditionally enabled and will be
removed. Cleanup the leftovers before doing so.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Juri Lelli <juri.lelli@redhat.com>
Cc: Vincent Guittot <vincent.guittot@linaro.org>
Cc: Dietmar Eggemann <dietmar.eggemann@arm.com>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Ben Segall <bsegall@google.com>
Cc: Mel Gorman <mgorman@suse.de>
Cc: Daniel Bristot de Oliveira <bristot@redhat.com>
---
 kernel/sched/core.c |    6 +-----
 lib/Kconfig.debug   |    1 -
 2 files changed, 1 insertion(+), 6 deletions(-)

--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -3706,8 +3706,7 @@ asmlinkage __visible void schedule_tail(
 	 * finish_task_switch() for details.
 	 *
 	 * finish_task_switch() will drop rq->lock() and lower preempt_count
-	 * and the preempt_enable() will end up enabling preemption (on
-	 * PREEMPT_COUNT kernels).
+	 * and the preempt_enable() will end up enabling preemption.
 	 */
 
 	rq = finish_task_switch(prev);
@@ -7311,9 +7310,6 @@ void __cant_sleep(const char *file, int
 	if (irqs_disabled())
 		return;
 
-	if (!IS_ENABLED(CONFIG_PREEMPT_COUNT))
-		return;
-
 	if (preempt_count() > preempt_offset)
 		return;
 
--- a/lib/Kconfig.debug
+++ b/lib/Kconfig.debug
@@ -1320,7 +1320,6 @@ config DEBUG_LOCKDEP
 
 config DEBUG_ATOMIC_SLEEP
 	bool "Sleep inside atomic section checking"
-	select PREEMPT_COUNT
 	depends on DEBUG_KERNEL
 	help
 	  If you say Y here, various routines which may sleep will become very


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 282+ messages in thread

* [patch 08/13] sched: Clenaup PREEMPT_COUNT leftovers
@ 2020-09-14 20:42   ` Thomas Gleixner
  0 siblings, 0 replies; 282+ messages in thread
From: Thomas Gleixner @ 2020-09-14 20:42 UTC (permalink / raw)
  To: LKML
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, linux-mm, linux-kselftest,
	linux-hexagon, Will Deacon, Ingo Molnar, Anton Ivanov,
	linux-arch, Vincent Guittot, Brian Cain, Richard Weinberger,
	Russell King, David Airlie, Ingo Molnar, Geert Uytterhoeven,
	Mel Gorman, intel-gfx, Matt Turner, Valentin Schneider,
	linux-xtensa, Shuah Khan, Paul E. McKenney, Jeff Dike, linux-um,
	Josh Triplett, Steven Rostedt, rcu, Linus Torvalds,
	Ivan Kokshaysky, Rodrigo Vivi, Dietmar Eggemann,
	linux-arm-kernel, Richard Henderson, Chris Zankel, Max Filippov,
	linux-m68k, linux-alpha, Mathieu Desnoyers, Andrew Morton,
	Daniel Bristot de Oliveira

CONFIG_PREEMPT_COUNT is now unconditionally enabled and will be
removed. Cleanup the leftovers before doing so.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Juri Lelli <juri.lelli@redhat.com>
Cc: Vincent Guittot <vincent.guittot@linaro.org>
Cc: Dietmar Eggemann <dietmar.eggemann@arm.com>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Ben Segall <bsegall@google.com>
Cc: Mel Gorman <mgorman@suse.de>
Cc: Daniel Bristot de Oliveira <bristot@redhat.com>
---
 kernel/sched/core.c |    6 +-----
 lib/Kconfig.debug   |    1 -
 2 files changed, 1 insertion(+), 6 deletions(-)

--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -3706,8 +3706,7 @@ asmlinkage __visible void schedule_tail(
 	 * finish_task_switch() for details.
 	 *
 	 * finish_task_switch() will drop rq->lock() and lower preempt_count
-	 * and the preempt_enable() will end up enabling preemption (on
-	 * PREEMPT_COUNT kernels).
+	 * and the preempt_enable() will end up enabling preemption.
 	 */
 
 	rq = finish_task_switch(prev);
@@ -7311,9 +7310,6 @@ void __cant_sleep(const char *file, int
 	if (irqs_disabled())
 		return;
 
-	if (!IS_ENABLED(CONFIG_PREEMPT_COUNT))
-		return;
-
 	if (preempt_count() > preempt_offset)
 		return;
 
--- a/lib/Kconfig.debug
+++ b/lib/Kconfig.debug
@@ -1320,7 +1320,6 @@ config DEBUG_LOCKDEP
 
 config DEBUG_ATOMIC_SLEEP
 	bool "Sleep inside atomic section checking"
-	select PREEMPT_COUNT
 	depends on DEBUG_KERNEL
 	help
 	  If you say Y here, various routines which may sleep will become very

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply	[flat|nested] 282+ messages in thread

* [Intel-gfx] [patch 08/13] sched: Clenaup PREEMPT_COUNT leftovers
@ 2020-09-14 20:42   ` Thomas Gleixner
  0 siblings, 0 replies; 282+ messages in thread
From: Thomas Gleixner @ 2020-09-14 20:42 UTC (permalink / raw)
  To: LKML
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, linux-mm, linux-kselftest,
	linux-hexagon, Will Deacon, Ingo Molnar, Anton Ivanov,
	linux-arch, Brian Cain, Richard Weinberger, Russell King,
	David Airlie, Ingo Molnar, Geert Uytterhoeven, Mel Gorman,
	intel-gfx, Matt Turner, Valentin Schneider, linux-xtensa,
	Shuah Khan, Paul E. McKenney, Jeff Dike, linux-um, Josh Triplett,
	Steven Rostedt, rcu, Linus Torvalds, Ivan Kokshaysky,
	Dietmar Eggemann, linux-arm-kernel, Richard Henderson,
	Chris Zankel, Max Filippov, linux-m68k, linux-alpha,
	Mathieu Desnoyers, Andrew Morton, Daniel Bristot de Oliveira

CONFIG_PREEMPT_COUNT is now unconditionally enabled and will be
removed. Cleanup the leftovers before doing so.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Juri Lelli <juri.lelli@redhat.com>
Cc: Vincent Guittot <vincent.guittot@linaro.org>
Cc: Dietmar Eggemann <dietmar.eggemann@arm.com>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Ben Segall <bsegall@google.com>
Cc: Mel Gorman <mgorman@suse.de>
Cc: Daniel Bristot de Oliveira <bristot@redhat.com>
---
 kernel/sched/core.c |    6 +-----
 lib/Kconfig.debug   |    1 -
 2 files changed, 1 insertion(+), 6 deletions(-)

--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -3706,8 +3706,7 @@ asmlinkage __visible void schedule_tail(
 	 * finish_task_switch() for details.
 	 *
 	 * finish_task_switch() will drop rq->lock() and lower preempt_count
-	 * and the preempt_enable() will end up enabling preemption (on
-	 * PREEMPT_COUNT kernels).
+	 * and the preempt_enable() will end up enabling preemption.
 	 */
 
 	rq = finish_task_switch(prev);
@@ -7311,9 +7310,6 @@ void __cant_sleep(const char *file, int
 	if (irqs_disabled())
 		return;
 
-	if (!IS_ENABLED(CONFIG_PREEMPT_COUNT))
-		return;
-
 	if (preempt_count() > preempt_offset)
 		return;
 
--- a/lib/Kconfig.debug
+++ b/lib/Kconfig.debug
@@ -1320,7 +1320,6 @@ config DEBUG_LOCKDEP
 
 config DEBUG_ATOMIC_SLEEP
 	bool "Sleep inside atomic section checking"
-	select PREEMPT_COUNT
 	depends on DEBUG_KERNEL
 	help
 	  If you say Y here, various routines which may sleep will become very

_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

^ permalink raw reply	[flat|nested] 282+ messages in thread

* [patch 08/13] sched: Clenaup PREEMPT_COUNT leftovers
@ 2020-09-14 20:42   ` Thomas Gleixner
  0 siblings, 0 replies; 282+ messages in thread
From: Thomas Gleixner @ 2020-09-14 20:42 UTC (permalink / raw)
  To: LKML
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Joonas Lahtinen, Lai Jiangshan, dri-devel, Ben Segall, linux-mm,
	linux-kselftest, linux-hexagon, Will Deacon, Ingo Molnar,
	Anton Ivanov, linux-arch, Vincent Guittot, Brian Cain,
	Richard Weinberger, Russell King, David Airlie, Ingo Molnar,
	Geert Uytterhoeven, Mel Gorman, intel-gfx, Matt Turner, Valentin

CONFIG_PREEMPT_COUNT is now unconditionally enabled and will be
removed. Cleanup the leftovers before doing so.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Juri Lelli <juri.lelli@redhat.com>
Cc: Vincent Guittot <vincent.guittot@linaro.org>
Cc: Dietmar Eggemann <dietmar.eggemann@arm.com>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Ben Segall <bsegall@google.com>
Cc: Mel Gorman <mgorman@suse.de>
Cc: Daniel Bristot de Oliveira <bristot@redhat.com>
---
 kernel/sched/core.c |    6 +-----
 lib/Kconfig.debug   |    1 -
 2 files changed, 1 insertion(+), 6 deletions(-)

--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -3706,8 +3706,7 @@ asmlinkage __visible void schedule_tail(
 	 * finish_task_switch() for details.
 	 *
 	 * finish_task_switch() will drop rq->lock() and lower preempt_count
-	 * and the preempt_enable() will end up enabling preemption (on
-	 * PREEMPT_COUNT kernels).
+	 * and the preempt_enable() will end up enabling preemption.
 	 */
 
 	rq = finish_task_switch(prev);
@@ -7311,9 +7310,6 @@ void __cant_sleep(const char *file, int
 	if (irqs_disabled())
 		return;
 
-	if (!IS_ENABLED(CONFIG_PREEMPT_COUNT))
-		return;
-
 	if (preempt_count() > preempt_offset)
 		return;
 
--- a/lib/Kconfig.debug
+++ b/lib/Kconfig.debug
@@ -1320,7 +1320,6 @@ config DEBUG_LOCKDEP
 
 config DEBUG_ATOMIC_SLEEP
 	bool "Sleep inside atomic section checking"
-	select PREEMPT_COUNT
 	depends on DEBUG_KERNEL
 	help
 	  If you say Y here, various routines which may sleep will become very

^ permalink raw reply	[flat|nested] 282+ messages in thread

* [patch 09/13] ARM: Clenaup PREEMPT_COUNT leftovers
  2020-09-14 20:42 ` Thomas Gleixner
                     ` (2 preceding siblings ...)
  (?)
@ 2020-09-14 20:42   ` Thomas Gleixner
  -1 siblings, 0 replies; 282+ messages in thread
From: Thomas Gleixner @ 2020-09-14 20:42 UTC (permalink / raw)
  To: LKML
  Cc: linux-arch, Linus Torvalds, Sebastian Andrzej Siewior,
	Valentin Schneider, Russell King, linux-arm-kernel,
	Richard Henderson, Ivan Kokshaysky, Matt Turner, linux-alpha,
	Jeff Dike, Richard Weinberger, Anton Ivanov, linux-um,
	Brian Cain, linux-hexagon, Geert Uytterhoeven, linux-m68k,
	Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
	Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman,
	Daniel Bristot de Oliveira, Will Deacon, Andrew Morton, linux-mm,
	Ingo Molnar, Chris Zankel, Max Filippov, linux-xtensa,
	Jani Nikula, Joonas Lahtinen, Rodrigo Vivi, David Airlie,
	Daniel Vetter, intel-gfx, dri-devel, Paul E. McKenney,
	Josh Triplett, Mathieu Desnoyers, Lai Jiangshan, Shuah Khan, rcu,
	linux-kselftest

CONFIG_PREEMPT_COUNT is now unconditionally enabled and will be
removed. Cleanup the leftovers before doing so.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Russell King <linux@armlinux.org.uk>
Cc: linux-arm-kernel@lists.infradead.org
---
 arch/arm/include/asm/assembler.h   |   11 -----------
 arch/arm/kernel/iwmmxt.S           |    2 --
 arch/arm/mach-ep93xx/crunch-bits.S |    2 --
 3 files changed, 15 deletions(-)

--- a/arch/arm/include/asm/assembler.h
+++ b/arch/arm/include/asm/assembler.h
@@ -212,7 +212,6 @@
 /*
  * Increment/decrement the preempt count.
  */
-#ifdef CONFIG_PREEMPT_COUNT
 	.macro	inc_preempt_count, ti, tmp
 	ldr	\tmp, [\ti, #TI_PREEMPT]	@ get preempt count
 	add	\tmp, \tmp, #1			@ increment it
@@ -229,16 +228,6 @@
 	get_thread_info \ti
 	dec_preempt_count \ti, \tmp
 	.endm
-#else
-	.macro	inc_preempt_count, ti, tmp
-	.endm
-
-	.macro	dec_preempt_count, ti, tmp
-	.endm
-
-	.macro	dec_preempt_count_ti, ti, tmp
-	.endm
-#endif
 
 #define USERL(l, x...)				\
 9999:	x;					\
--- a/arch/arm/kernel/iwmmxt.S
+++ b/arch/arm/kernel/iwmmxt.S
@@ -94,9 +94,7 @@ ENTRY(iwmmxt_task_enable)
 	mov	r2, r2				@ cpwait
 	bl	concan_save
 
-#ifdef CONFIG_PREEMPT_COUNT
 	get_thread_info r10
-#endif
 4:	dec_preempt_count r10, r3
 	ret	r9				@ normal exit from exception
 
--- a/arch/arm/mach-ep93xx/crunch-bits.S
+++ b/arch/arm/mach-ep93xx/crunch-bits.S
@@ -191,9 +191,7 @@ ENTRY(crunch_task_enable)
 	cfldr64		mvdx15, [r0, #CRUNCH_MVDX15]
 
 1:
-#ifdef CONFIG_PREEMPT_COUNT
 	get_thread_info r10
-#endif
 2:	dec_preempt_count r10, r3
 	ret	lr
 



^ permalink raw reply	[flat|nested] 282+ messages in thread

* [patch 09/13] ARM: Clenaup PREEMPT_COUNT leftovers
@ 2020-09-14 20:42   ` Thomas Gleixner
  0 siblings, 0 replies; 282+ messages in thread
From: Thomas Gleixner @ 2020-09-14 20:42 UTC (permalink / raw)
  To: LKML
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Joonas Lahtinen, Lai Jiangshan, dri-devel, Ben Segall, linux-mm,
	linux-kselftest, linux-hexagon, Will Deacon, Ingo Molnar,
	Anton Ivanov, linux-arch, Vincent Guittot, Brian Cain,
	Richard Weinberger, Russell King, David Airlie, Ingo Molnar,
	Geert Uytterhoeven, Mel Gorman, intel-gfx, Matt Turner,
	Valentin Schneider, linux-xtensa, Shuah Khan, Paul E. McKenney,
	Jeff Dike, linux-um, Josh Triplett, Steven Rostedt, rcu,
	linux-m68k, Ivan Kokshaysky, Jani Nikula, Rodrigo Vivi,
	Dietmar Eggemann, linux-arm-kernel, Richard Henderson,
	Chris Zankel, Max Filippov, Linus Torvalds, Daniel Vetter,
	linux-alpha, Mathieu Desnoyers, Andrew Morton,
	Daniel Bristot de Oliveira

CONFIG_PREEMPT_COUNT is now unconditionally enabled and will be
removed. Cleanup the leftovers before doing so.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Russell King <linux@armlinux.org.uk>
Cc: linux-arm-kernel@lists.infradead.org
---
 arch/arm/include/asm/assembler.h   |   11 -----------
 arch/arm/kernel/iwmmxt.S           |    2 --
 arch/arm/mach-ep93xx/crunch-bits.S |    2 --
 3 files changed, 15 deletions(-)

--- a/arch/arm/include/asm/assembler.h
+++ b/arch/arm/include/asm/assembler.h
@@ -212,7 +212,6 @@
 /*
  * Increment/decrement the preempt count.
  */
-#ifdef CONFIG_PREEMPT_COUNT
 	.macro	inc_preempt_count, ti, tmp
 	ldr	\tmp, [\ti, #TI_PREEMPT]	@ get preempt count
 	add	\tmp, \tmp, #1			@ increment it
@@ -229,16 +228,6 @@
 	get_thread_info \ti
 	dec_preempt_count \ti, \tmp
 	.endm
-#else
-	.macro	inc_preempt_count, ti, tmp
-	.endm
-
-	.macro	dec_preempt_count, ti, tmp
-	.endm
-
-	.macro	dec_preempt_count_ti, ti, tmp
-	.endm
-#endif
 
 #define USERL(l, x...)				\
 9999:	x;					\
--- a/arch/arm/kernel/iwmmxt.S
+++ b/arch/arm/kernel/iwmmxt.S
@@ -94,9 +94,7 @@ ENTRY(iwmmxt_task_enable)
 	mov	r2, r2				@ cpwait
 	bl	concan_save
 
-#ifdef CONFIG_PREEMPT_COUNT
 	get_thread_info r10
-#endif
 4:	dec_preempt_count r10, r3
 	ret	r9				@ normal exit from exception
 
--- a/arch/arm/mach-ep93xx/crunch-bits.S
+++ b/arch/arm/mach-ep93xx/crunch-bits.S
@@ -191,9 +191,7 @@ ENTRY(crunch_task_enable)
 	cfldr64		mvdx15, [r0, #CRUNCH_MVDX15]
 
 1:
-#ifdef CONFIG_PREEMPT_COUNT
 	get_thread_info r10
-#endif
 2:	dec_preempt_count r10, r3
 	ret	lr
 


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 282+ messages in thread

* [patch 09/13] ARM: Clenaup PREEMPT_COUNT leftovers
@ 2020-09-14 20:42   ` Thomas Gleixner
  0 siblings, 0 replies; 282+ messages in thread
From: Thomas Gleixner @ 2020-09-14 20:42 UTC (permalink / raw)
  To: LKML
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, linux-mm, linux-kselftest,
	linux-hexagon, Will Deacon, Ingo Molnar, Anton Ivanov,
	linux-arch, Vincent Guittot, Brian Cain, Richard Weinberger,
	Russell King, David Airlie, Ingo Molnar, Geert Uytterhoeven,
	Mel Gorman, intel-gfx, Matt Turner, Valentin Schneider,
	linux-xtensa, Shuah Khan, Paul E. McKenney, Jeff Dike, linux-um,
	Josh Triplett, Steven Rostedt, rcu, linux-m68k, Ivan Kokshaysky,
	Rodrigo Vivi, Dietmar Eggemann, linux-arm-kernel,
	Richard Henderson, Chris Zankel, Max Filippov, Linus Torvalds,
	linux-alpha, Mathieu Desnoyers, Andrew Morton,
	Daniel Bristot de Oliveira

CONFIG_PREEMPT_COUNT is now unconditionally enabled and will be
removed. Cleanup the leftovers before doing so.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Russell King <linux@armlinux.org.uk>
Cc: linux-arm-kernel@lists.infradead.org
---
 arch/arm/include/asm/assembler.h   |   11 -----------
 arch/arm/kernel/iwmmxt.S           |    2 --
 arch/arm/mach-ep93xx/crunch-bits.S |    2 --
 3 files changed, 15 deletions(-)

--- a/arch/arm/include/asm/assembler.h
+++ b/arch/arm/include/asm/assembler.h
@@ -212,7 +212,6 @@
 /*
  * Increment/decrement the preempt count.
  */
-#ifdef CONFIG_PREEMPT_COUNT
 	.macro	inc_preempt_count, ti, tmp
 	ldr	\tmp, [\ti, #TI_PREEMPT]	@ get preempt count
 	add	\tmp, \tmp, #1			@ increment it
@@ -229,16 +228,6 @@
 	get_thread_info \ti
 	dec_preempt_count \ti, \tmp
 	.endm
-#else
-	.macro	inc_preempt_count, ti, tmp
-	.endm
-
-	.macro	dec_preempt_count, ti, tmp
-	.endm
-
-	.macro	dec_preempt_count_ti, ti, tmp
-	.endm
-#endif
 
 #define USERL(l, x...)				\
 9999:	x;					\
--- a/arch/arm/kernel/iwmmxt.S
+++ b/arch/arm/kernel/iwmmxt.S
@@ -94,9 +94,7 @@ ENTRY(iwmmxt_task_enable)
 	mov	r2, r2				@ cpwait
 	bl	concan_save
 
-#ifdef CONFIG_PREEMPT_COUNT
 	get_thread_info r10
-#endif
 4:	dec_preempt_count r10, r3
 	ret	r9				@ normal exit from exception
 
--- a/arch/arm/mach-ep93xx/crunch-bits.S
+++ b/arch/arm/mach-ep93xx/crunch-bits.S
@@ -191,9 +191,7 @@ ENTRY(crunch_task_enable)
 	cfldr64		mvdx15, [r0, #CRUNCH_MVDX15]
 
 1:
-#ifdef CONFIG_PREEMPT_COUNT
 	get_thread_info r10
-#endif
 2:	dec_preempt_count r10, r3
 	ret	lr
 

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply	[flat|nested] 282+ messages in thread

* [Intel-gfx] [patch 09/13] ARM: Clenaup PREEMPT_COUNT leftovers
@ 2020-09-14 20:42   ` Thomas Gleixner
  0 siblings, 0 replies; 282+ messages in thread
From: Thomas Gleixner @ 2020-09-14 20:42 UTC (permalink / raw)
  To: LKML
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, linux-mm, linux-kselftest,
	linux-hexagon, Will Deacon, Ingo Molnar, Anton Ivanov,
	linux-arch, Brian Cain, Richard Weinberger, Russell King,
	David Airlie, Ingo Molnar, Geert Uytterhoeven, Mel Gorman,
	intel-gfx, Matt Turner, Valentin Schneider, linux-xtensa,
	Shuah Khan, Paul E. McKenney, Jeff Dike, linux-um, Josh Triplett,
	Steven Rostedt, rcu, linux-m68k, Ivan Kokshaysky,
	Dietmar Eggemann, linux-arm-kernel, Richard Henderson,
	Chris Zankel, Max Filippov, Linus Torvalds, linux-alpha,
	Mathieu Desnoyers, Andrew Morton, Daniel Bristot de Oliveira

CONFIG_PREEMPT_COUNT is now unconditionally enabled and will be
removed. Cleanup the leftovers before doing so.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Russell King <linux@armlinux.org.uk>
Cc: linux-arm-kernel@lists.infradead.org
---
 arch/arm/include/asm/assembler.h   |   11 -----------
 arch/arm/kernel/iwmmxt.S           |    2 --
 arch/arm/mach-ep93xx/crunch-bits.S |    2 --
 3 files changed, 15 deletions(-)

--- a/arch/arm/include/asm/assembler.h
+++ b/arch/arm/include/asm/assembler.h
@@ -212,7 +212,6 @@
 /*
  * Increment/decrement the preempt count.
  */
-#ifdef CONFIG_PREEMPT_COUNT
 	.macro	inc_preempt_count, ti, tmp
 	ldr	\tmp, [\ti, #TI_PREEMPT]	@ get preempt count
 	add	\tmp, \tmp, #1			@ increment it
@@ -229,16 +228,6 @@
 	get_thread_info \ti
 	dec_preempt_count \ti, \tmp
 	.endm
-#else
-	.macro	inc_preempt_count, ti, tmp
-	.endm
-
-	.macro	dec_preempt_count, ti, tmp
-	.endm
-
-	.macro	dec_preempt_count_ti, ti, tmp
-	.endm
-#endif
 
 #define USERL(l, x...)				\
 9999:	x;					\
--- a/arch/arm/kernel/iwmmxt.S
+++ b/arch/arm/kernel/iwmmxt.S
@@ -94,9 +94,7 @@ ENTRY(iwmmxt_task_enable)
 	mov	r2, r2				@ cpwait
 	bl	concan_save
 
-#ifdef CONFIG_PREEMPT_COUNT
 	get_thread_info r10
-#endif
 4:	dec_preempt_count r10, r3
 	ret	r9				@ normal exit from exception
 
--- a/arch/arm/mach-ep93xx/crunch-bits.S
+++ b/arch/arm/mach-ep93xx/crunch-bits.S
@@ -191,9 +191,7 @@ ENTRY(crunch_task_enable)
 	cfldr64		mvdx15, [r0, #CRUNCH_MVDX15]
 
 1:
-#ifdef CONFIG_PREEMPT_COUNT
 	get_thread_info r10
-#endif
 2:	dec_preempt_count r10, r3
 	ret	lr
 

_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

^ permalink raw reply	[flat|nested] 282+ messages in thread

* [patch 09/13] ARM: Clenaup PREEMPT_COUNT leftovers
@ 2020-09-14 20:42   ` Thomas Gleixner
  0 siblings, 0 replies; 282+ messages in thread
From: Thomas Gleixner @ 2020-09-14 20:42 UTC (permalink / raw)
  To: LKML
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Joonas Lahtinen, Lai Jiangshan, dri-devel, Ben Segall, linux-mm,
	linux-kselftest, linux-hexagon, Will Deacon, Ingo Molnar,
	Anton Ivanov, linux-arch, Vincent Guittot, Brian Cain,
	Richard Weinberger, Russell King, David Airlie, Ingo Molnar,
	Geert Uytterhoeven, Mel Gorman, intel-gfx, Matt Turner, Valentin

CONFIG_PREEMPT_COUNT is now unconditionally enabled and will be
removed. Cleanup the leftovers before doing so.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Russell King <linux@armlinux.org.uk>
Cc: linux-arm-kernel@lists.infradead.org
---
 arch/arm/include/asm/assembler.h   |   11 -----------
 arch/arm/kernel/iwmmxt.S           |    2 --
 arch/arm/mach-ep93xx/crunch-bits.S |    2 --
 3 files changed, 15 deletions(-)

--- a/arch/arm/include/asm/assembler.h
+++ b/arch/arm/include/asm/assembler.h
@@ -212,7 +212,6 @@
 /*
  * Increment/decrement the preempt count.
  */
-#ifdef CONFIG_PREEMPT_COUNT
 	.macro	inc_preempt_count, ti, tmp
 	ldr	\tmp, [\ti, #TI_PREEMPT]	@ get preempt count
 	add	\tmp, \tmp, #1			@ increment it
@@ -229,16 +228,6 @@
 	get_thread_info \ti
 	dec_preempt_count \ti, \tmp
 	.endm
-#else
-	.macro	inc_preempt_count, ti, tmp
-	.endm
-
-	.macro	dec_preempt_count, ti, tmp
-	.endm
-
-	.macro	dec_preempt_count_ti, ti, tmp
-	.endm
-#endif
 
 #define USERL(l, x...)				\
 9999:	x;					\
--- a/arch/arm/kernel/iwmmxt.S
+++ b/arch/arm/kernel/iwmmxt.S
@@ -94,9 +94,7 @@ ENTRY(iwmmxt_task_enable)
 	mov	r2, r2				@ cpwait
 	bl	concan_save
 
-#ifdef CONFIG_PREEMPT_COUNT
 	get_thread_info r10
-#endif
 4:	dec_preempt_count r10, r3
 	ret	r9				@ normal exit from exception
 
--- a/arch/arm/mach-ep93xx/crunch-bits.S
+++ b/arch/arm/mach-ep93xx/crunch-bits.S
@@ -191,9 +191,7 @@ ENTRY(crunch_task_enable)
 	cfldr64		mvdx15, [r0, #CRUNCH_MVDX15]
 
 1:
-#ifdef CONFIG_PREEMPT_COUNT
 	get_thread_info r10
-#endif
 2:	dec_preempt_count r10, r3
 	ret	lr

^ permalink raw reply	[flat|nested] 282+ messages in thread

* [patch 10/13] xtensa: Clenaup PREEMPT_COUNT leftovers
  2020-09-14 20:42 ` Thomas Gleixner
                     ` (2 preceding siblings ...)
  (?)
@ 2020-09-14 20:42   ` Thomas Gleixner
  -1 siblings, 0 replies; 282+ messages in thread
From: Thomas Gleixner @ 2020-09-14 20:42 UTC (permalink / raw)
  To: LKML
  Cc: linux-arch, Linus Torvalds, Sebastian Andrzej Siewior,
	Valentin Schneider, Chris Zankel, Max Filippov, linux-xtensa,
	Richard Henderson, Ivan Kokshaysky, Matt Turner, linux-alpha,
	Jeff Dike, Richard Weinberger, Anton Ivanov, linux-um,
	Brian Cain, linux-hexagon, Geert Uytterhoeven, linux-m68k,
	Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
	Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman,
	Daniel Bristot de Oliveira, Will Deacon, Andrew Morton, linux-mm,
	Ingo Molnar, Russell King, linux-arm-kernel, Jani Nikula,
	Joonas Lahtinen, Rodrigo Vivi, David Airlie, Daniel Vetter,
	intel-gfx, dri-devel, Paul E. McKenney, Josh Triplett,
	Mathieu Desnoyers, Lai Jiangshan, Shuah Khan, rcu,
	linux-kselftest

CONFIG_PREEMPT_COUNT is now unconditionally enabled and will be
removed. Cleanup the leftovers before doing so.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Chris Zankel <chris@zankel.net>
Cc: Max Filippov <jcmvbkbc@gmail.com>
Cc: linux-xtensa@linux-xtensa.org
---
 arch/xtensa/kernel/entry.S |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

--- a/arch/xtensa/kernel/entry.S
+++ b/arch/xtensa/kernel/entry.S
@@ -819,7 +819,7 @@ ENTRY(debug_exception)
 	 * preemption if we have HW breakpoints to preserve DEBUGCAUSE.DBNUM
 	 * meaning.
 	 */
-#if defined(CONFIG_PREEMPT_COUNT) && defined(CONFIG_HAVE_HW_BREAKPOINT)
+#ifdef CONFIG_HAVE_HW_BREAKPOINT
 	GET_THREAD_INFO(a2, a1)
 	l32i	a3, a2, TI_PRE_COUNT
 	addi	a3, a3, 1



^ permalink raw reply	[flat|nested] 282+ messages in thread

* [patch 10/13] xtensa: Clenaup PREEMPT_COUNT leftovers
@ 2020-09-14 20:42   ` Thomas Gleixner
  0 siblings, 0 replies; 282+ messages in thread
From: Thomas Gleixner @ 2020-09-14 20:42 UTC (permalink / raw)
  To: LKML
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Joonas Lahtinen, Lai Jiangshan, dri-devel, Ben Segall,
	Max Filippov, linux-kselftest, linux-hexagon, Will Deacon,
	Ingo Molnar, Anton Ivanov, linux-arch, Vincent Guittot,
	Brian Cain, Richard Weinberger, Russell King, David Airlie,
	Ingo Molnar, Geert Uytterhoeven, Mel Gorman, intel-gfx,
	Matt Turner, Valentin Schneider, linux-xtensa, Shuah Khan,
	Paul E. McKenney, Jeff Dike, linux-um, Josh Triplett,
	Steven Rostedt, rcu, linux-m68k, Ivan Kokshaysky, Jani Nikula,
	Rodrigo Vivi, Dietmar Eggemann, linux-arm-kernel,
	Richard Henderson, Chris Zankel, linux-mm, Linus Torvalds,
	Daniel Vetter, linux-alpha, Mathieu Desnoyers, Andrew Morton,
	Daniel Bristot de Oliveira

CONFIG_PREEMPT_COUNT is now unconditionally enabled and will be
removed. Cleanup the leftovers before doing so.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Chris Zankel <chris@zankel.net>
Cc: Max Filippov <jcmvbkbc@gmail.com>
Cc: linux-xtensa@linux-xtensa.org
---
 arch/xtensa/kernel/entry.S |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

--- a/arch/xtensa/kernel/entry.S
+++ b/arch/xtensa/kernel/entry.S
@@ -819,7 +819,7 @@ ENTRY(debug_exception)
 	 * preemption if we have HW breakpoints to preserve DEBUGCAUSE.DBNUM
 	 * meaning.
 	 */
-#if defined(CONFIG_PREEMPT_COUNT) && defined(CONFIG_HAVE_HW_BREAKPOINT)
+#ifdef CONFIG_HAVE_HW_BREAKPOINT
 	GET_THREAD_INFO(a2, a1)
 	l32i	a3, a2, TI_PRE_COUNT
 	addi	a3, a3, 1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 282+ messages in thread

* [patch 10/13] xtensa: Clenaup PREEMPT_COUNT leftovers
@ 2020-09-14 20:42   ` Thomas Gleixner
  0 siblings, 0 replies; 282+ messages in thread
From: Thomas Gleixner @ 2020-09-14 20:42 UTC (permalink / raw)
  To: LKML
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, Max Filippov,
	linux-kselftest, linux-hexagon, Will Deacon, Ingo Molnar,
	Anton Ivanov, linux-arch, Vincent Guittot, Brian Cain,
	Richard Weinberger, Russell King, David Airlie, Ingo Molnar,
	Geert Uytterhoeven, Mel Gorman, intel-gfx, Matt Turner,
	Valentin Schneider, linux-xtensa, Shuah Khan, Paul E. McKenney,
	Jeff Dike, linux-um, Josh Triplett, Steven Rostedt, rcu,
	linux-m68k, Ivan Kokshaysky, Rodrigo Vivi, Dietmar Eggemann,
	linux-arm-kernel, Richard Henderson, Chris Zankel, linux-mm,
	Linus Torvalds, linux-alpha, Mathieu Desnoyers, Andrew Morton,
	Daniel Bristot de Oliveira

CONFIG_PREEMPT_COUNT is now unconditionally enabled and will be
removed. Cleanup the leftovers before doing so.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Chris Zankel <chris@zankel.net>
Cc: Max Filippov <jcmvbkbc@gmail.com>
Cc: linux-xtensa@linux-xtensa.org
---
 arch/xtensa/kernel/entry.S |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

--- a/arch/xtensa/kernel/entry.S
+++ b/arch/xtensa/kernel/entry.S
@@ -819,7 +819,7 @@ ENTRY(debug_exception)
 	 * preemption if we have HW breakpoints to preserve DEBUGCAUSE.DBNUM
 	 * meaning.
 	 */
-#if defined(CONFIG_PREEMPT_COUNT) && defined(CONFIG_HAVE_HW_BREAKPOINT)
+#ifdef CONFIG_HAVE_HW_BREAKPOINT
 	GET_THREAD_INFO(a2, a1)
 	l32i	a3, a2, TI_PRE_COUNT
 	addi	a3, a3, 1

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply	[flat|nested] 282+ messages in thread

* [Intel-gfx] [patch 10/13] xtensa: Clenaup PREEMPT_COUNT leftovers
@ 2020-09-14 20:42   ` Thomas Gleixner
  0 siblings, 0 replies; 282+ messages in thread
From: Thomas Gleixner @ 2020-09-14 20:42 UTC (permalink / raw)
  To: LKML
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, Max Filippov,
	linux-kselftest, linux-hexagon, Will Deacon, Ingo Molnar,
	Anton Ivanov, linux-arch, Brian Cain, Richard Weinberger,
	Russell King, David Airlie, Ingo Molnar, Geert Uytterhoeven,
	Mel Gorman, intel-gfx, Matt Turner, Valentin Schneider,
	linux-xtensa, Shuah Khan, Paul E. McKenney, Jeff Dike, linux-um,
	Josh Triplett, Steven Rostedt, rcu, linux-m68k, Ivan Kokshaysky,
	Dietmar Eggemann, linux-arm-kernel, Richard Henderson,
	Chris Zankel, linux-mm, Linus Torvalds, linux-alpha,
	Mathieu Desnoyers, Andrew Morton, Daniel Bristot de Oliveira

CONFIG_PREEMPT_COUNT is now unconditionally enabled and will be
removed. Cleanup the leftovers before doing so.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Chris Zankel <chris@zankel.net>
Cc: Max Filippov <jcmvbkbc@gmail.com>
Cc: linux-xtensa@linux-xtensa.org
---
 arch/xtensa/kernel/entry.S |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

--- a/arch/xtensa/kernel/entry.S
+++ b/arch/xtensa/kernel/entry.S
@@ -819,7 +819,7 @@ ENTRY(debug_exception)
 	 * preemption if we have HW breakpoints to preserve DEBUGCAUSE.DBNUM
 	 * meaning.
 	 */
-#if defined(CONFIG_PREEMPT_COUNT) && defined(CONFIG_HAVE_HW_BREAKPOINT)
+#ifdef CONFIG_HAVE_HW_BREAKPOINT
 	GET_THREAD_INFO(a2, a1)
 	l32i	a3, a2, TI_PRE_COUNT
 	addi	a3, a3, 1

_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

^ permalink raw reply	[flat|nested] 282+ messages in thread

* [patch 10/13] xtensa: Clenaup PREEMPT_COUNT leftovers
@ 2020-09-14 20:42   ` Thomas Gleixner
  0 siblings, 0 replies; 282+ messages in thread
From: Thomas Gleixner @ 2020-09-14 20:42 UTC (permalink / raw)
  To: LKML
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Joonas Lahtinen, Lai Jiangshan, dri-devel, Ben Segall,
	Max Filippov, linux-kselftest, linux-hexagon, Will Deacon,
	Ingo Molnar, Anton Ivanov, linux-arch, Vincent Guittot,
	Brian Cain, Richard Weinberger, Russell King, David Airlie,
	Ingo Molnar, Geert Uytterhoeven, Mel Gorman, intel-gfx,
	Matt Turner

CONFIG_PREEMPT_COUNT is now unconditionally enabled and will be
removed. Cleanup the leftovers before doing so.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Chris Zankel <chris@zankel.net>
Cc: Max Filippov <jcmvbkbc@gmail.com>
Cc: linux-xtensa@linux-xtensa.org
---
 arch/xtensa/kernel/entry.S |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

--- a/arch/xtensa/kernel/entry.S
+++ b/arch/xtensa/kernel/entry.S
@@ -819,7 +819,7 @@ ENTRY(debug_exception)
 	 * preemption if we have HW breakpoints to preserve DEBUGCAUSE.DBNUM
 	 * meaning.
 	 */
-#if defined(CONFIG_PREEMPT_COUNT) && defined(CONFIG_HAVE_HW_BREAKPOINT)
+#ifdef CONFIG_HAVE_HW_BREAKPOINT
 	GET_THREAD_INFO(a2, a1)
 	l32i	a3, a2, TI_PRE_COUNT
 	addi	a3, a3, 1

^ permalink raw reply	[flat|nested] 282+ messages in thread

* [patch 11/13] drm/i915: Clenaup PREEMPT_COUNT leftovers
  2020-09-14 20:42 ` Thomas Gleixner
                     ` (2 preceding siblings ...)
  (?)
@ 2020-09-14 20:42   ` Thomas Gleixner
  -1 siblings, 0 replies; 282+ messages in thread
From: Thomas Gleixner @ 2020-09-14 20:42 UTC (permalink / raw)
  To: LKML
  Cc: linux-arch, Linus Torvalds, Sebastian Andrzej Siewior,
	Valentin Schneider, Jani Nikula, Joonas Lahtinen, Rodrigo Vivi,
	David Airlie, Daniel Vetter, intel-gfx, dri-devel,
	Richard Henderson, Ivan Kokshaysky, Matt Turner, linux-alpha,
	Jeff Dike, Richard Weinberger, Anton Ivanov, linux-um,
	Brian Cain, linux-hexagon, Geert Uytterhoeven, linux-m68k,
	Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
	Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman,
	Daniel Bristot de Oliveira, Will Deacon, Andrew Morton, linux-mm,
	Ingo Molnar, Russell King, linux-arm-kernel, Chris Zankel,
	Max Filippov, linux-xtensa, Paul E. McKenney, Josh Triplett,
	Mathieu Desnoyers, Lai Jiangshan, Shuah Khan, rcu,
	linux-kselftest

CONFIG_PREEMPT_COUNT is now unconditionally enabled and will be
removed. Cleanup the leftovers before doing so.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Jani Nikula <jani.nikula@linux.intel.com>
Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
Cc: David Airlie <airlied@linux.ie>
Cc: Daniel Vetter <daniel@ffwll.ch>
Cc: intel-gfx@lists.freedesktop.org
Cc: dri-devel@lists.freedesktop.org
---
 drivers/gpu/drm/i915/Kconfig.debug |    1 -
 drivers/gpu/drm/i915/i915_utils.h  |    3 +--
 2 files changed, 1 insertion(+), 3 deletions(-)

--- a/drivers/gpu/drm/i915/Kconfig.debug
+++ b/drivers/gpu/drm/i915/Kconfig.debug
@@ -20,7 +20,6 @@ config DRM_I915_DEBUG
 	bool "Enable additional driver debugging"
 	depends on DRM_I915
 	select DEBUG_FS
-	select PREEMPT_COUNT
 	select I2C_CHARDEV
 	select STACKDEPOT
 	select DRM_DP_AUX_CHARDEV
--- a/drivers/gpu/drm/i915/i915_utils.h
+++ b/drivers/gpu/drm/i915/i915_utils.h
@@ -337,8 +337,7 @@ wait_remaining_ms_from_jiffies(unsigned
 						   (Wmax))
 #define wait_for(COND, MS)		_wait_for((COND), (MS) * 1000, 10, 1000)
 
-/* If CONFIG_PREEMPT_COUNT is disabled, in_atomic() always reports false. */
-#if defined(CONFIG_DRM_I915_DEBUG) && defined(CONFIG_PREEMPT_COUNT)
+#ifdef CONFIG_DRM_I915_DEBUG
 # define _WAIT_FOR_ATOMIC_CHECK(ATOMIC) WARN_ON_ONCE((ATOMIC) && !in_atomic())
 #else
 # define _WAIT_FOR_ATOMIC_CHECK(ATOMIC) do { } while (0)



^ permalink raw reply	[flat|nested] 282+ messages in thread

* [patch 11/13] drm/i915: Clenaup PREEMPT_COUNT leftovers
@ 2020-09-14 20:42   ` Thomas Gleixner
  0 siblings, 0 replies; 282+ messages in thread
From: Thomas Gleixner @ 2020-09-14 20:42 UTC (permalink / raw)
  To: LKML
  Cc: Juri Lelli, David Airlie, Sebastian Andrzej Siewior,
	Joonas Lahtinen, Lai Jiangshan, dri-devel, Ben Segall, linux-mm,
	linux-kselftest, linux-hexagon, Will Deacon, Ingo Molnar,
	Anton Ivanov, linux-arch, Vincent Guittot, Brian Cain,
	Richard Weinberger, Russell King, Peter Zijlstra, Ingo Molnar,
	Geert Uytterhoeven, Mel Gorman, Matt Turner, intel-gfx,
	linux-xtensa, Shuah Khan, Paul E. McKenney, Jeff Dike, linux-um,
	Josh Triplett, Jani Nikula, rcu, Linus Torvalds, Ivan Kokshaysky,
	Steven Rostedt, Rodrigo Vivi, Dietmar Eggemann, linux-arm-kernel,
	Richard Henderson, Chris Zankel, Max Filippov, linux-m68k,
	Valentin Schneider, Daniel Vetter, linux-alpha,
	Mathieu Desnoyers, Andrew Morton, Daniel Bristot de Oliveira

CONFIG_PREEMPT_COUNT is now unconditionally enabled and will be
removed. Cleanup the leftovers before doing so.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Jani Nikula <jani.nikula@linux.intel.com>
Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
Cc: David Airlie <airlied@linux.ie>
Cc: Daniel Vetter <daniel@ffwll.ch>
Cc: intel-gfx@lists.freedesktop.org
Cc: dri-devel@lists.freedesktop.org
---
 drivers/gpu/drm/i915/Kconfig.debug |    1 -
 drivers/gpu/drm/i915/i915_utils.h  |    3 +--
 2 files changed, 1 insertion(+), 3 deletions(-)

--- a/drivers/gpu/drm/i915/Kconfig.debug
+++ b/drivers/gpu/drm/i915/Kconfig.debug
@@ -20,7 +20,6 @@ config DRM_I915_DEBUG
 	bool "Enable additional driver debugging"
 	depends on DRM_I915
 	select DEBUG_FS
-	select PREEMPT_COUNT
 	select I2C_CHARDEV
 	select STACKDEPOT
 	select DRM_DP_AUX_CHARDEV
--- a/drivers/gpu/drm/i915/i915_utils.h
+++ b/drivers/gpu/drm/i915/i915_utils.h
@@ -337,8 +337,7 @@ wait_remaining_ms_from_jiffies(unsigned
 						   (Wmax))
 #define wait_for(COND, MS)		_wait_for((COND), (MS) * 1000, 10, 1000)
 
-/* If CONFIG_PREEMPT_COUNT is disabled, in_atomic() always reports false. */
-#if defined(CONFIG_DRM_I915_DEBUG) && defined(CONFIG_PREEMPT_COUNT)
+#ifdef CONFIG_DRM_I915_DEBUG
 # define _WAIT_FOR_ATOMIC_CHECK(ATOMIC) WARN_ON_ONCE((ATOMIC) && !in_atomic())
 #else
 # define _WAIT_FOR_ATOMIC_CHECK(ATOMIC) do { } while (0)


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 282+ messages in thread

* [patch 11/13] drm/i915: Clenaup PREEMPT_COUNT leftovers
@ 2020-09-14 20:42   ` Thomas Gleixner
  0 siblings, 0 replies; 282+ messages in thread
From: Thomas Gleixner @ 2020-09-14 20:42 UTC (permalink / raw)
  To: LKML
  Cc: Juri Lelli, David Airlie, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, linux-mm, linux-kselftest,
	linux-hexagon, Will Deacon, Ingo Molnar, Anton Ivanov,
	linux-arch, Vincent Guittot, Brian Cain, Richard Weinberger,
	Russell King, Peter Zijlstra, Ingo Molnar, Geert Uytterhoeven,
	Mel Gorman, Matt Turner, intel-gfx, linux-xtensa, Shuah Khan,
	Paul E. McKenney, Jeff Dike, linux-um, Josh Triplett, rcu,
	Linus Torvalds, Ivan Kokshaysky, Steven Rostedt, Rodrigo Vivi,
	Dietmar Eggemann, linux-arm-kernel, Richard Henderson,
	Chris Zankel, Max Filippov, linux-m68k, Valentin Schneider,
	linux-alpha, Mathieu Desnoyers, Andrew Morton,
	Daniel Bristot de Oliveira

CONFIG_PREEMPT_COUNT is now unconditionally enabled and will be
removed. Cleanup the leftovers before doing so.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Jani Nikula <jani.nikula@linux.intel.com>
Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
Cc: David Airlie <airlied@linux.ie>
Cc: Daniel Vetter <daniel@ffwll.ch>
Cc: intel-gfx@lists.freedesktop.org
Cc: dri-devel@lists.freedesktop.org
---
 drivers/gpu/drm/i915/Kconfig.debug |    1 -
 drivers/gpu/drm/i915/i915_utils.h  |    3 +--
 2 files changed, 1 insertion(+), 3 deletions(-)

--- a/drivers/gpu/drm/i915/Kconfig.debug
+++ b/drivers/gpu/drm/i915/Kconfig.debug
@@ -20,7 +20,6 @@ config DRM_I915_DEBUG
 	bool "Enable additional driver debugging"
 	depends on DRM_I915
 	select DEBUG_FS
-	select PREEMPT_COUNT
 	select I2C_CHARDEV
 	select STACKDEPOT
 	select DRM_DP_AUX_CHARDEV
--- a/drivers/gpu/drm/i915/i915_utils.h
+++ b/drivers/gpu/drm/i915/i915_utils.h
@@ -337,8 +337,7 @@ wait_remaining_ms_from_jiffies(unsigned
 						   (Wmax))
 #define wait_for(COND, MS)		_wait_for((COND), (MS) * 1000, 10, 1000)
 
-/* If CONFIG_PREEMPT_COUNT is disabled, in_atomic() always reports false. */
-#if defined(CONFIG_DRM_I915_DEBUG) && defined(CONFIG_PREEMPT_COUNT)
+#ifdef CONFIG_DRM_I915_DEBUG
 # define _WAIT_FOR_ATOMIC_CHECK(ATOMIC) WARN_ON_ONCE((ATOMIC) && !in_atomic())
 #else
 # define _WAIT_FOR_ATOMIC_CHECK(ATOMIC) do { } while (0)

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply	[flat|nested] 282+ messages in thread

* [Intel-gfx] [patch 11/13] drm/i915: Clenaup PREEMPT_COUNT leftovers
@ 2020-09-14 20:42   ` Thomas Gleixner
  0 siblings, 0 replies; 282+ messages in thread
From: Thomas Gleixner @ 2020-09-14 20:42 UTC (permalink / raw)
  To: LKML
  Cc: Juri Lelli, David Airlie, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, linux-mm, linux-kselftest,
	linux-hexagon, Will Deacon, Ingo Molnar, Anton Ivanov,
	linux-arch, Brian Cain, Richard Weinberger, Russell King,
	Peter Zijlstra, Ingo Molnar, Geert Uytterhoeven, Mel Gorman,
	Matt Turner, intel-gfx, linux-xtensa, Shuah Khan,
	Paul E. McKenney, Jeff Dike, linux-um, Josh Triplett, rcu,
	Linus Torvalds, Ivan Kokshaysky, Steven Rostedt,
	Dietmar Eggemann, linux-arm-kernel, Richard Henderson,
	Chris Zankel, Max Filippov, linux-m68k, Valentin Schneider,
	linux-alpha, Mathieu Desnoyers, Andrew Morton,
	Daniel Bristot de Oliveira

CONFIG_PREEMPT_COUNT is now unconditionally enabled and will be
removed. Cleanup the leftovers before doing so.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Jani Nikula <jani.nikula@linux.intel.com>
Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
Cc: David Airlie <airlied@linux.ie>
Cc: Daniel Vetter <daniel@ffwll.ch>
Cc: intel-gfx@lists.freedesktop.org
Cc: dri-devel@lists.freedesktop.org
---
 drivers/gpu/drm/i915/Kconfig.debug |    1 -
 drivers/gpu/drm/i915/i915_utils.h  |    3 +--
 2 files changed, 1 insertion(+), 3 deletions(-)

--- a/drivers/gpu/drm/i915/Kconfig.debug
+++ b/drivers/gpu/drm/i915/Kconfig.debug
@@ -20,7 +20,6 @@ config DRM_I915_DEBUG
 	bool "Enable additional driver debugging"
 	depends on DRM_I915
 	select DEBUG_FS
-	select PREEMPT_COUNT
 	select I2C_CHARDEV
 	select STACKDEPOT
 	select DRM_DP_AUX_CHARDEV
--- a/drivers/gpu/drm/i915/i915_utils.h
+++ b/drivers/gpu/drm/i915/i915_utils.h
@@ -337,8 +337,7 @@ wait_remaining_ms_from_jiffies(unsigned
 						   (Wmax))
 #define wait_for(COND, MS)		_wait_for((COND), (MS) * 1000, 10, 1000)
 
-/* If CONFIG_PREEMPT_COUNT is disabled, in_atomic() always reports false. */
-#if defined(CONFIG_DRM_I915_DEBUG) && defined(CONFIG_PREEMPT_COUNT)
+#ifdef CONFIG_DRM_I915_DEBUG
 # define _WAIT_FOR_ATOMIC_CHECK(ATOMIC) WARN_ON_ONCE((ATOMIC) && !in_atomic())
 #else
 # define _WAIT_FOR_ATOMIC_CHECK(ATOMIC) do { } while (0)

_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

^ permalink raw reply	[flat|nested] 282+ messages in thread

* [patch 11/13] drm/i915: Clenaup PREEMPT_COUNT leftovers
@ 2020-09-14 20:42   ` Thomas Gleixner
  0 siblings, 0 replies; 282+ messages in thread
From: Thomas Gleixner @ 2020-09-14 20:42 UTC (permalink / raw)
  To: LKML
  Cc: Juri Lelli, David Airlie, Sebastian Andrzej Siewior,
	Joonas Lahtinen, Lai Jiangshan, dri-devel, Ben Segall, linux-mm,
	linux-kselftest, linux-hexagon, Will Deacon, Ingo Molnar,
	Anton Ivanov, linux-arch, Vincent Guittot, Brian Cain,
	Richard Weinberger, Russell King, Peter Zijlstra, Ingo Molnar,
	Geert Uytterhoeven, Mel Gorman, Matt Turner, intel-gfx,
	linux-xtensa

CONFIG_PREEMPT_COUNT is now unconditionally enabled and will be
removed. Cleanup the leftovers before doing so.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Jani Nikula <jani.nikula@linux.intel.com>
Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
Cc: David Airlie <airlied@linux.ie>
Cc: Daniel Vetter <daniel@ffwll.ch>
Cc: intel-gfx@lists.freedesktop.org
Cc: dri-devel@lists.freedesktop.org
---
 drivers/gpu/drm/i915/Kconfig.debug |    1 -
 drivers/gpu/drm/i915/i915_utils.h  |    3 +--
 2 files changed, 1 insertion(+), 3 deletions(-)

--- a/drivers/gpu/drm/i915/Kconfig.debug
+++ b/drivers/gpu/drm/i915/Kconfig.debug
@@ -20,7 +20,6 @@ config DRM_I915_DEBUG
 	bool "Enable additional driver debugging"
 	depends on DRM_I915
 	select DEBUG_FS
-	select PREEMPT_COUNT
 	select I2C_CHARDEV
 	select STACKDEPOT
 	select DRM_DP_AUX_CHARDEV
--- a/drivers/gpu/drm/i915/i915_utils.h
+++ b/drivers/gpu/drm/i915/i915_utils.h
@@ -337,8 +337,7 @@ wait_remaining_ms_from_jiffies(unsigned
 						   (Wmax))
 #define wait_for(COND, MS)		_wait_for((COND), (MS) * 1000, 10, 1000)
 
-/* If CONFIG_PREEMPT_COUNT is disabled, in_atomic() always reports false. */
-#if defined(CONFIG_DRM_I915_DEBUG) && defined(CONFIG_PREEMPT_COUNT)
+#ifdef CONFIG_DRM_I915_DEBUG
 # define _WAIT_FOR_ATOMIC_CHECK(ATOMIC) WARN_ON_ONCE((ATOMIC) && !in_atomic())
 #else
 # define _WAIT_FOR_ATOMIC_CHECK(ATOMIC) do { } while (0)

^ permalink raw reply	[flat|nested] 282+ messages in thread

* [patch 12/13] rcutorture: Clenaup PREEMPT_COUNT leftovers
  2020-09-14 20:42 ` Thomas Gleixner
                     ` (2 preceding siblings ...)
  (?)
@ 2020-09-14 20:42   ` Thomas Gleixner
  -1 siblings, 0 replies; 282+ messages in thread
From: Thomas Gleixner @ 2020-09-14 20:42 UTC (permalink / raw)
  To: LKML
  Cc: linux-arch, Linus Torvalds, Sebastian Andrzej Siewior,
	Valentin Schneider, Paul E. McKenney, Josh Triplett,
	Steven Rostedt, Mathieu Desnoyers, Lai Jiangshan, Shuah Khan,
	rcu, linux-kselftest, Richard Henderson, Ivan Kokshaysky,
	Matt Turner, linux-alpha, Jeff Dike, Richard Weinberger,
	Anton Ivanov, linux-um, Brian Cain, linux-hexagon,
	Geert Uytterhoeven, linux-m68k, Ingo Molnar, Peter Zijlstra,
	Juri Lelli, Vincent Guittot, Dietmar Eggemann, Ben Segall,
	Mel Gorman, Daniel Bristot de Oliveira, Will Deacon,
	Andrew Morton, linux-mm, Ingo Molnar, Russell King,
	linux-arm-kernel, Chris Zankel, Max Filippov, linux-xtensa,
	Jani Nikula, Joonas Lahtinen, Rodrigo Vivi, David Airlie,
	Daniel Vetter, intel-gfx, dri-devel

CONFIG_PREEMPT_COUNT is now unconditionally enabled and will be
removed. Cleanup the leftovers before doing so.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: "Paul E. McKenney" <paulmck@kernel.org>
Cc: Josh Triplett <josh@joshtriplett.org>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Cc: Lai Jiangshan <jiangshanlai@gmail.com>
Cc: Shuah Khan <shuah@kernel.org>
Cc: rcu@vger.kernel.org
Cc: linux-kselftest@vger.kernel.org
---
 tools/testing/selftests/rcutorture/configs/rcu/SRCU-t            |    1 -
 tools/testing/selftests/rcutorture/configs/rcu/SRCU-u            |    1 -
 tools/testing/selftests/rcutorture/configs/rcu/TINY01            |    1 -
 tools/testing/selftests/rcutorture/doc/TINY_RCU.txt              |    5 ++---
 tools/testing/selftests/rcutorture/doc/TREE_RCU-kconfig.txt      |    1 -
 tools/testing/selftests/rcutorture/formal/srcu-cbmc/src/config.h |    1 -
 6 files changed, 2 insertions(+), 8 deletions(-)

--- a/tools/testing/selftests/rcutorture/configs/rcu/SRCU-t
+++ b/tools/testing/selftests/rcutorture/configs/rcu/SRCU-t
@@ -7,4 +7,3 @@ CONFIG_RCU_TRACE=n
 CONFIG_DEBUG_LOCK_ALLOC=n
 CONFIG_DEBUG_OBJECTS_RCU_HEAD=n
 CONFIG_DEBUG_ATOMIC_SLEEP=y
-#CHECK#CONFIG_PREEMPT_COUNT=y
--- a/tools/testing/selftests/rcutorture/configs/rcu/SRCU-u
+++ b/tools/testing/selftests/rcutorture/configs/rcu/SRCU-u
@@ -7,4 +7,3 @@ CONFIG_RCU_TRACE=n
 CONFIG_DEBUG_LOCK_ALLOC=y
 CONFIG_PROVE_LOCKING=y
 CONFIG_DEBUG_OBJECTS_RCU_HEAD=n
-CONFIG_PREEMPT_COUNT=n
--- a/tools/testing/selftests/rcutorture/configs/rcu/TINY01
+++ b/tools/testing/selftests/rcutorture/configs/rcu/TINY01
@@ -10,4 +10,3 @@ CONFIG_RCU_TRACE=n
 #CHECK#CONFIG_RCU_STALL_COMMON=n
 CONFIG_DEBUG_LOCK_ALLOC=n
 CONFIG_DEBUG_OBJECTS_RCU_HEAD=n
-CONFIG_PREEMPT_COUNT=n
--- a/tools/testing/selftests/rcutorture/doc/TINY_RCU.txt
+++ b/tools/testing/selftests/rcutorture/doc/TINY_RCU.txt
@@ -3,11 +3,10 @@ This document gives a brief rationale fo
 
 Kconfig Parameters:
 
-CONFIG_DEBUG_LOCK_ALLOC -- Do all three and none of the three.
-CONFIG_PREEMPT_COUNT
+CONFIG_DEBUG_LOCK_ALLOC -- Do both and none of the two.
 CONFIG_RCU_TRACE
 
-The theory here is that randconfig testing will hit the other six possible
+The theory here is that randconfig testing will hit the other two possible
 combinations of these parameters.
 
 
--- a/tools/testing/selftests/rcutorture/doc/TREE_RCU-kconfig.txt
+++ b/tools/testing/selftests/rcutorture/doc/TREE_RCU-kconfig.txt
@@ -43,7 +43,6 @@ CONFIG_64BIT
 
 	Used only to check CONFIG_RCU_FANOUT value, inspection suffices.
 
-CONFIG_PREEMPT_COUNT
 CONFIG_PREEMPT_RCU
 
 	Redundant with CONFIG_PREEMPT, ignore.
--- a/tools/testing/selftests/rcutorture/formal/srcu-cbmc/src/config.h
+++ b/tools/testing/selftests/rcutorture/formal/srcu-cbmc/src/config.h
@@ -8,7 +8,6 @@
 #undef CONFIG_HOTPLUG_CPU
 #undef CONFIG_MODULES
 #undef CONFIG_NO_HZ_FULL_SYSIDLE
-#undef CONFIG_PREEMPT_COUNT
 #undef CONFIG_PREEMPT_RCU
 #undef CONFIG_PROVE_RCU
 #undef CONFIG_RCU_NOCB_CPU



^ permalink raw reply	[flat|nested] 282+ messages in thread

* [patch 12/13] rcutorture: Clenaup PREEMPT_COUNT leftovers
@ 2020-09-14 20:42   ` Thomas Gleixner
  0 siblings, 0 replies; 282+ messages in thread
From: Thomas Gleixner @ 2020-09-14 20:42 UTC (permalink / raw)
  To: LKML
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Joonas Lahtinen, Lai Jiangshan, dri-devel, Ben Segall, linux-mm,
	Will Deacon, linux-kselftest, Shuah Khan, Ingo Molnar,
	Anton Ivanov, linux-arch, Vincent Guittot, Brian Cain,
	Richard Weinberger, Russell King, David Airlie, Ingo Molnar,
	Geert Uytterhoeven, Mel Gorman, intel-gfx, Matt Turner,
	Valentin Schneider, linux-xtensa, Paul E. McKenney, Jeff Dike,
	linux-alpha, linux-um, Josh Triplett, Steven Rostedt, rcu,
	Linus Torvalds, Mathieu Desnoyers, Jani Nikula, Rodrigo Vivi,
	Dietmar Eggemann, linux-arm-kernel, Richard Henderson,
	Chris Zankel, Max Filippov, linux-m68k, Daniel Vetter,
	linux-hexagon, Ivan Kokshaysky, Andrew Morton,
	Daniel Bristot de Oliveira

CONFIG_PREEMPT_COUNT is now unconditionally enabled and will be
removed. Cleanup the leftovers before doing so.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: "Paul E. McKenney" <paulmck@kernel.org>
Cc: Josh Triplett <josh@joshtriplett.org>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Cc: Lai Jiangshan <jiangshanlai@gmail.com>
Cc: Shuah Khan <shuah@kernel.org>
Cc: rcu@vger.kernel.org
Cc: linux-kselftest@vger.kernel.org
---
 tools/testing/selftests/rcutorture/configs/rcu/SRCU-t            |    1 -
 tools/testing/selftests/rcutorture/configs/rcu/SRCU-u            |    1 -
 tools/testing/selftests/rcutorture/configs/rcu/TINY01            |    1 -
 tools/testing/selftests/rcutorture/doc/TINY_RCU.txt              |    5 ++---
 tools/testing/selftests/rcutorture/doc/TREE_RCU-kconfig.txt      |    1 -
 tools/testing/selftests/rcutorture/formal/srcu-cbmc/src/config.h |    1 -
 6 files changed, 2 insertions(+), 8 deletions(-)

--- a/tools/testing/selftests/rcutorture/configs/rcu/SRCU-t
+++ b/tools/testing/selftests/rcutorture/configs/rcu/SRCU-t
@@ -7,4 +7,3 @@ CONFIG_RCU_TRACE=n
 CONFIG_DEBUG_LOCK_ALLOC=n
 CONFIG_DEBUG_OBJECTS_RCU_HEAD=n
 CONFIG_DEBUG_ATOMIC_SLEEP=y
-#CHECK#CONFIG_PREEMPT_COUNT=y
--- a/tools/testing/selftests/rcutorture/configs/rcu/SRCU-u
+++ b/tools/testing/selftests/rcutorture/configs/rcu/SRCU-u
@@ -7,4 +7,3 @@ CONFIG_RCU_TRACE=n
 CONFIG_DEBUG_LOCK_ALLOC=y
 CONFIG_PROVE_LOCKING=y
 CONFIG_DEBUG_OBJECTS_RCU_HEAD=n
-CONFIG_PREEMPT_COUNT=n
--- a/tools/testing/selftests/rcutorture/configs/rcu/TINY01
+++ b/tools/testing/selftests/rcutorture/configs/rcu/TINY01
@@ -10,4 +10,3 @@ CONFIG_RCU_TRACE=n
 #CHECK#CONFIG_RCU_STALL_COMMON=n
 CONFIG_DEBUG_LOCK_ALLOC=n
 CONFIG_DEBUG_OBJECTS_RCU_HEAD=n
-CONFIG_PREEMPT_COUNT=n
--- a/tools/testing/selftests/rcutorture/doc/TINY_RCU.txt
+++ b/tools/testing/selftests/rcutorture/doc/TINY_RCU.txt
@@ -3,11 +3,10 @@ This document gives a brief rationale fo
 
 Kconfig Parameters:
 
-CONFIG_DEBUG_LOCK_ALLOC -- Do all three and none of the three.
-CONFIG_PREEMPT_COUNT
+CONFIG_DEBUG_LOCK_ALLOC -- Do both and none of the two.
 CONFIG_RCU_TRACE
 
-The theory here is that randconfig testing will hit the other six possible
+The theory here is that randconfig testing will hit the other two possible
 combinations of these parameters.
 
 
--- a/tools/testing/selftests/rcutorture/doc/TREE_RCU-kconfig.txt
+++ b/tools/testing/selftests/rcutorture/doc/TREE_RCU-kconfig.txt
@@ -43,7 +43,6 @@ CONFIG_64BIT
 
 	Used only to check CONFIG_RCU_FANOUT value, inspection suffices.
 
-CONFIG_PREEMPT_COUNT
 CONFIG_PREEMPT_RCU
 
 	Redundant with CONFIG_PREEMPT, ignore.
--- a/tools/testing/selftests/rcutorture/formal/srcu-cbmc/src/config.h
+++ b/tools/testing/selftests/rcutorture/formal/srcu-cbmc/src/config.h
@@ -8,7 +8,6 @@
 #undef CONFIG_HOTPLUG_CPU
 #undef CONFIG_MODULES
 #undef CONFIG_NO_HZ_FULL_SYSIDLE
-#undef CONFIG_PREEMPT_COUNT
 #undef CONFIG_PREEMPT_RCU
 #undef CONFIG_PROVE_RCU
 #undef CONFIG_RCU_NOCB_CPU


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 282+ messages in thread

* [patch 12/13] rcutorture: Clenaup PREEMPT_COUNT leftovers
@ 2020-09-14 20:42   ` Thomas Gleixner
  0 siblings, 0 replies; 282+ messages in thread
From: Thomas Gleixner @ 2020-09-14 20:42 UTC (permalink / raw)
  To: LKML
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, linux-mm, Will Deacon,
	linux-kselftest, Shuah Khan, Ingo Molnar, Anton Ivanov,
	linux-arch, Vincent Guittot, Brian Cain, Richard Weinberger,
	Russell King, David Airlie, Ingo Molnar, Geert Uytterhoeven,
	Mel Gorman, intel-gfx, Matt Turner, Valentin Schneider,
	linux-xtensa, Paul E. McKenney, Jeff Dike, linux-alpha, linux-um,
	Josh Triplett, Steven Rostedt, rcu, Linus Torvalds,
	Mathieu Desnoyers, Rodrigo Vivi, Dietmar Eggemann,
	linux-arm-kernel, Richard Henderson, Chris Zankel, Max Filippov,
	linux-m68k, linux-hexagon, Ivan Kokshaysky, Andrew Morton,
	Daniel Bristot de Oliveira

CONFIG_PREEMPT_COUNT is now unconditionally enabled and will be
removed. Cleanup the leftovers before doing so.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: "Paul E. McKenney" <paulmck@kernel.org>
Cc: Josh Triplett <josh@joshtriplett.org>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Cc: Lai Jiangshan <jiangshanlai@gmail.com>
Cc: Shuah Khan <shuah@kernel.org>
Cc: rcu@vger.kernel.org
Cc: linux-kselftest@vger.kernel.org
---
 tools/testing/selftests/rcutorture/configs/rcu/SRCU-t            |    1 -
 tools/testing/selftests/rcutorture/configs/rcu/SRCU-u            |    1 -
 tools/testing/selftests/rcutorture/configs/rcu/TINY01            |    1 -
 tools/testing/selftests/rcutorture/doc/TINY_RCU.txt              |    5 ++---
 tools/testing/selftests/rcutorture/doc/TREE_RCU-kconfig.txt      |    1 -
 tools/testing/selftests/rcutorture/formal/srcu-cbmc/src/config.h |    1 -
 6 files changed, 2 insertions(+), 8 deletions(-)

--- a/tools/testing/selftests/rcutorture/configs/rcu/SRCU-t
+++ b/tools/testing/selftests/rcutorture/configs/rcu/SRCU-t
@@ -7,4 +7,3 @@ CONFIG_RCU_TRACE=n
 CONFIG_DEBUG_LOCK_ALLOC=n
 CONFIG_DEBUG_OBJECTS_RCU_HEAD=n
 CONFIG_DEBUG_ATOMIC_SLEEP=y
-#CHECK#CONFIG_PREEMPT_COUNT=y
--- a/tools/testing/selftests/rcutorture/configs/rcu/SRCU-u
+++ b/tools/testing/selftests/rcutorture/configs/rcu/SRCU-u
@@ -7,4 +7,3 @@ CONFIG_RCU_TRACE=n
 CONFIG_DEBUG_LOCK_ALLOC=y
 CONFIG_PROVE_LOCKING=y
 CONFIG_DEBUG_OBJECTS_RCU_HEAD=n
-CONFIG_PREEMPT_COUNT=n
--- a/tools/testing/selftests/rcutorture/configs/rcu/TINY01
+++ b/tools/testing/selftests/rcutorture/configs/rcu/TINY01
@@ -10,4 +10,3 @@ CONFIG_RCU_TRACE=n
 #CHECK#CONFIG_RCU_STALL_COMMON=n
 CONFIG_DEBUG_LOCK_ALLOC=n
 CONFIG_DEBUG_OBJECTS_RCU_HEAD=n
-CONFIG_PREEMPT_COUNT=n
--- a/tools/testing/selftests/rcutorture/doc/TINY_RCU.txt
+++ b/tools/testing/selftests/rcutorture/doc/TINY_RCU.txt
@@ -3,11 +3,10 @@ This document gives a brief rationale fo
 
 Kconfig Parameters:
 
-CONFIG_DEBUG_LOCK_ALLOC -- Do all three and none of the three.
-CONFIG_PREEMPT_COUNT
+CONFIG_DEBUG_LOCK_ALLOC -- Do both and none of the two.
 CONFIG_RCU_TRACE
 
-The theory here is that randconfig testing will hit the other six possible
+The theory here is that randconfig testing will hit the other two possible
 combinations of these parameters.
 
 
--- a/tools/testing/selftests/rcutorture/doc/TREE_RCU-kconfig.txt
+++ b/tools/testing/selftests/rcutorture/doc/TREE_RCU-kconfig.txt
@@ -43,7 +43,6 @@ CONFIG_64BIT
 
 	Used only to check CONFIG_RCU_FANOUT value, inspection suffices.
 
-CONFIG_PREEMPT_COUNT
 CONFIG_PREEMPT_RCU
 
 	Redundant with CONFIG_PREEMPT, ignore.
--- a/tools/testing/selftests/rcutorture/formal/srcu-cbmc/src/config.h
+++ b/tools/testing/selftests/rcutorture/formal/srcu-cbmc/src/config.h
@@ -8,7 +8,6 @@
 #undef CONFIG_HOTPLUG_CPU
 #undef CONFIG_MODULES
 #undef CONFIG_NO_HZ_FULL_SYSIDLE
-#undef CONFIG_PREEMPT_COUNT
 #undef CONFIG_PREEMPT_RCU
 #undef CONFIG_PROVE_RCU
 #undef CONFIG_RCU_NOCB_CPU

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply	[flat|nested] 282+ messages in thread

* [Intel-gfx] [patch 12/13] rcutorture: Clenaup PREEMPT_COUNT leftovers
@ 2020-09-14 20:42   ` Thomas Gleixner
  0 siblings, 0 replies; 282+ messages in thread
From: Thomas Gleixner @ 2020-09-14 20:42 UTC (permalink / raw)
  To: LKML
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, linux-mm, Will Deacon,
	linux-kselftest, Shuah Khan, Ingo Molnar, Anton Ivanov,
	linux-arch, Brian Cain, Richard Weinberger, Russell King,
	David Airlie, Ingo Molnar, Geert Uytterhoeven, Mel Gorman,
	intel-gfx, Matt Turner, Valentin Schneider, linux-xtensa,
	Paul E. McKenney, Jeff Dike, linux-alpha, linux-um,
	Josh Triplett, Steven Rostedt, rcu, Linus Torvalds,
	Mathieu Desnoyers, Dietmar Eggemann, linux-arm-kernel,
	Richard Henderson, Chris Zankel, Max Filippov, linux-m68k,
	linux-hexagon, Ivan Kokshaysky, Andrew Morton,
	Daniel Bristot de Oliveira

CONFIG_PREEMPT_COUNT is now unconditionally enabled and will be
removed. Cleanup the leftovers before doing so.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: "Paul E. McKenney" <paulmck@kernel.org>
Cc: Josh Triplett <josh@joshtriplett.org>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Cc: Lai Jiangshan <jiangshanlai@gmail.com>
Cc: Shuah Khan <shuah@kernel.org>
Cc: rcu@vger.kernel.org
Cc: linux-kselftest@vger.kernel.org
---
 tools/testing/selftests/rcutorture/configs/rcu/SRCU-t            |    1 -
 tools/testing/selftests/rcutorture/configs/rcu/SRCU-u            |    1 -
 tools/testing/selftests/rcutorture/configs/rcu/TINY01            |    1 -
 tools/testing/selftests/rcutorture/doc/TINY_RCU.txt              |    5 ++---
 tools/testing/selftests/rcutorture/doc/TREE_RCU-kconfig.txt      |    1 -
 tools/testing/selftests/rcutorture/formal/srcu-cbmc/src/config.h |    1 -
 6 files changed, 2 insertions(+), 8 deletions(-)

--- a/tools/testing/selftests/rcutorture/configs/rcu/SRCU-t
+++ b/tools/testing/selftests/rcutorture/configs/rcu/SRCU-t
@@ -7,4 +7,3 @@ CONFIG_RCU_TRACE=n
 CONFIG_DEBUG_LOCK_ALLOC=n
 CONFIG_DEBUG_OBJECTS_RCU_HEAD=n
 CONFIG_DEBUG_ATOMIC_SLEEP=y
-#CHECK#CONFIG_PREEMPT_COUNT=y
--- a/tools/testing/selftests/rcutorture/configs/rcu/SRCU-u
+++ b/tools/testing/selftests/rcutorture/configs/rcu/SRCU-u
@@ -7,4 +7,3 @@ CONFIG_RCU_TRACE=n
 CONFIG_DEBUG_LOCK_ALLOC=y
 CONFIG_PROVE_LOCKING=y
 CONFIG_DEBUG_OBJECTS_RCU_HEAD=n
-CONFIG_PREEMPT_COUNT=n
--- a/tools/testing/selftests/rcutorture/configs/rcu/TINY01
+++ b/tools/testing/selftests/rcutorture/configs/rcu/TINY01
@@ -10,4 +10,3 @@ CONFIG_RCU_TRACE=n
 #CHECK#CONFIG_RCU_STALL_COMMON=n
 CONFIG_DEBUG_LOCK_ALLOC=n
 CONFIG_DEBUG_OBJECTS_RCU_HEAD=n
-CONFIG_PREEMPT_COUNT=n
--- a/tools/testing/selftests/rcutorture/doc/TINY_RCU.txt
+++ b/tools/testing/selftests/rcutorture/doc/TINY_RCU.txt
@@ -3,11 +3,10 @@ This document gives a brief rationale fo
 
 Kconfig Parameters:
 
-CONFIG_DEBUG_LOCK_ALLOC -- Do all three and none of the three.
-CONFIG_PREEMPT_COUNT
+CONFIG_DEBUG_LOCK_ALLOC -- Do both and none of the two.
 CONFIG_RCU_TRACE
 
-The theory here is that randconfig testing will hit the other six possible
+The theory here is that randconfig testing will hit the other two possible
 combinations of these parameters.
 
 
--- a/tools/testing/selftests/rcutorture/doc/TREE_RCU-kconfig.txt
+++ b/tools/testing/selftests/rcutorture/doc/TREE_RCU-kconfig.txt
@@ -43,7 +43,6 @@ CONFIG_64BIT
 
 	Used only to check CONFIG_RCU_FANOUT value, inspection suffices.
 
-CONFIG_PREEMPT_COUNT
 CONFIG_PREEMPT_RCU
 
 	Redundant with CONFIG_PREEMPT, ignore.
--- a/tools/testing/selftests/rcutorture/formal/srcu-cbmc/src/config.h
+++ b/tools/testing/selftests/rcutorture/formal/srcu-cbmc/src/config.h
@@ -8,7 +8,6 @@
 #undef CONFIG_HOTPLUG_CPU
 #undef CONFIG_MODULES
 #undef CONFIG_NO_HZ_FULL_SYSIDLE
-#undef CONFIG_PREEMPT_COUNT
 #undef CONFIG_PREEMPT_RCU
 #undef CONFIG_PROVE_RCU
 #undef CONFIG_RCU_NOCB_CPU

_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

^ permalink raw reply	[flat|nested] 282+ messages in thread

* [patch 12/13] rcutorture: Clenaup PREEMPT_COUNT leftovers
@ 2020-09-14 20:42   ` Thomas Gleixner
  0 siblings, 0 replies; 282+ messages in thread
From: Thomas Gleixner @ 2020-09-14 20:42 UTC (permalink / raw)
  To: LKML
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Joonas Lahtinen, Lai Jiangshan, dri-devel, Ben Segall, linux-mm,
	Will Deacon, linux-kselftest, Shuah Khan, Ingo Molnar,
	Anton Ivanov, linux-arch, Vincent Guittot, Brian Cain,
	Richard Weinberger, Russell King, David Airlie, Ingo Molnar,
	Geert Uytterhoeven, Mel Gorman, intel-gfx, Matt Turner, Valentin

CONFIG_PREEMPT_COUNT is now unconditionally enabled and will be
removed. Cleanup the leftovers before doing so.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: "Paul E. McKenney" <paulmck@kernel.org>
Cc: Josh Triplett <josh@joshtriplett.org>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Cc: Lai Jiangshan <jiangshanlai@gmail.com>
Cc: Shuah Khan <shuah@kernel.org>
Cc: rcu@vger.kernel.org
Cc: linux-kselftest@vger.kernel.org
---
 tools/testing/selftests/rcutorture/configs/rcu/SRCU-t            |    1 -
 tools/testing/selftests/rcutorture/configs/rcu/SRCU-u            |    1 -
 tools/testing/selftests/rcutorture/configs/rcu/TINY01            |    1 -
 tools/testing/selftests/rcutorture/doc/TINY_RCU.txt              |    5 ++---
 tools/testing/selftests/rcutorture/doc/TREE_RCU-kconfig.txt      |    1 -
 tools/testing/selftests/rcutorture/formal/srcu-cbmc/src/config.h |    1 -
 6 files changed, 2 insertions(+), 8 deletions(-)

--- a/tools/testing/selftests/rcutorture/configs/rcu/SRCU-t
+++ b/tools/testing/selftests/rcutorture/configs/rcu/SRCU-t
@@ -7,4 +7,3 @@ CONFIG_RCU_TRACE=n
 CONFIG_DEBUG_LOCK_ALLOC=n
 CONFIG_DEBUG_OBJECTS_RCU_HEAD=n
 CONFIG_DEBUG_ATOMIC_SLEEP=y
-#CHECK#CONFIG_PREEMPT_COUNT=y
--- a/tools/testing/selftests/rcutorture/configs/rcu/SRCU-u
+++ b/tools/testing/selftests/rcutorture/configs/rcu/SRCU-u
@@ -7,4 +7,3 @@ CONFIG_RCU_TRACE=n
 CONFIG_DEBUG_LOCK_ALLOC=y
 CONFIG_PROVE_LOCKING=y
 CONFIG_DEBUG_OBJECTS_RCU_HEAD=n
-CONFIG_PREEMPT_COUNT=n
--- a/tools/testing/selftests/rcutorture/configs/rcu/TINY01
+++ b/tools/testing/selftests/rcutorture/configs/rcu/TINY01
@@ -10,4 +10,3 @@ CONFIG_RCU_TRACE=n
 #CHECK#CONFIG_RCU_STALL_COMMON=n
 CONFIG_DEBUG_LOCK_ALLOC=n
 CONFIG_DEBUG_OBJECTS_RCU_HEAD=n
-CONFIG_PREEMPT_COUNT=n
--- a/tools/testing/selftests/rcutorture/doc/TINY_RCU.txt
+++ b/tools/testing/selftests/rcutorture/doc/TINY_RCU.txt
@@ -3,11 +3,10 @@ This document gives a brief rationale fo
 
 Kconfig Parameters:
 
-CONFIG_DEBUG_LOCK_ALLOC -- Do all three and none of the three.
-CONFIG_PREEMPT_COUNT
+CONFIG_DEBUG_LOCK_ALLOC -- Do both and none of the two.
 CONFIG_RCU_TRACE
 
-The theory here is that randconfig testing will hit the other six possible
+The theory here is that randconfig testing will hit the other two possible
 combinations of these parameters.
 
 
--- a/tools/testing/selftests/rcutorture/doc/TREE_RCU-kconfig.txt
+++ b/tools/testing/selftests/rcutorture/doc/TREE_RCU-kconfig.txt
@@ -43,7 +43,6 @@ CONFIG_64BIT
 
 	Used only to check CONFIG_RCU_FANOUT value, inspection suffices.
 
-CONFIG_PREEMPT_COUNT
 CONFIG_PREEMPT_RCU
 
 	Redundant with CONFIG_PREEMPT, ignore.
--- a/tools/testing/selftests/rcutorture/formal/srcu-cbmc/src/config.h
+++ b/tools/testing/selftests/rcutorture/formal/srcu-cbmc/src/config.h
@@ -8,7 +8,6 @@
 #undef CONFIG_HOTPLUG_CPU
 #undef CONFIG_MODULES
 #undef CONFIG_NO_HZ_FULL_SYSIDLE
-#undef CONFIG_PREEMPT_COUNT
 #undef CONFIG_PREEMPT_RCU
 #undef CONFIG_PROVE_RCU
 #undef CONFIG_RCU_NOCB_CPU

^ permalink raw reply	[flat|nested] 282+ messages in thread

* [patch 13/13] preempt: Remove PREEMPT_COUNT from Kconfig
  2020-09-14 20:42 ` Thomas Gleixner
                     ` (2 preceding siblings ...)
  (?)
@ 2020-09-14 20:42   ` Thomas Gleixner
  -1 siblings, 0 replies; 282+ messages in thread
From: Thomas Gleixner @ 2020-09-14 20:42 UTC (permalink / raw)
  To: LKML
  Cc: linux-arch, Linus Torvalds, Sebastian Andrzej Siewior,
	Valentin Schneider, Richard Henderson, Ivan Kokshaysky,
	Matt Turner, linux-alpha, Jeff Dike, Richard Weinberger,
	Anton Ivanov, linux-um, Brian Cain, linux-hexagon,
	Geert Uytterhoeven, linux-m68k, Ingo Molnar, Peter Zijlstra,
	Juri Lelli, Vincent Guittot, Dietmar Eggemann, Steven Rostedt,
	Ben Segall, Mel Gorman, Daniel Bristot de Oliveira, Will Deacon,
	Andrew Morton, linux-mm, Ingo Molnar, Russell King,
	linux-arm-kernel, Chris Zankel, Max Filippov, linux-xtensa,
	Jani Nikula, Joonas Lahtinen, Rodrigo Vivi, David Airlie,
	Daniel Vetter, intel-gfx, dri-devel, Paul E. McKenney,
	Josh Triplett, Mathieu Desnoyers, Lai Jiangshan, Shuah Khan, rcu,
	linux-kselftest

All conditionals and irritations are gone.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 kernel/Kconfig.preempt |    3 ---
 1 file changed, 3 deletions(-)

--- a/kernel/Kconfig.preempt
+++ b/kernel/Kconfig.preempt
@@ -74,8 +74,5 @@ config PREEMPT_RT
 
 endchoice
 
-config PREEMPT_COUNT
-       def_bool y
-
 config PREEMPTION
        bool



^ permalink raw reply	[flat|nested] 282+ messages in thread

* [patch 13/13] preempt: Remove PREEMPT_COUNT from Kconfig
@ 2020-09-14 20:42   ` Thomas Gleixner
  0 siblings, 0 replies; 282+ messages in thread
From: Thomas Gleixner @ 2020-09-14 20:42 UTC (permalink / raw)
  To: LKML
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Joonas Lahtinen, Lai Jiangshan, dri-devel, Ben Segall, linux-mm,
	linux-kselftest, linux-hexagon, Will Deacon, Ingo Molnar,
	Anton Ivanov, linux-arch, Vincent Guittot, Brian Cain,
	Richard Weinberger, Russell King, David Airlie, Ingo Molnar,
	Geert Uytterhoeven, Mel Gorman, intel-gfx, Matt Turner,
	Valentin Schneider, linux-xtensa, Shuah Khan, Paul E. McKenney,
	Jeff Dike, linux-um, Josh Triplett, Steven Rostedt, rcu,
	linux-m68k, Ivan Kokshaysky, Jani Nikula, Rodrigo Vivi,
	Dietmar Eggemann, linux-arm-kernel, Richard Henderson,
	Chris Zankel, Max Filippov, Linus Torvalds, Daniel Vetter,
	linux-alpha, Mathieu Desnoyers, Andrew Morton,
	Daniel Bristot de Oliveira

All conditionals and irritations are gone.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 kernel/Kconfig.preempt |    3 ---
 1 file changed, 3 deletions(-)

--- a/kernel/Kconfig.preempt
+++ b/kernel/Kconfig.preempt
@@ -74,8 +74,5 @@ config PREEMPT_RT
 
 endchoice
 
-config PREEMPT_COUNT
-       def_bool y
-
 config PREEMPTION
        bool


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 282+ messages in thread

* [patch 13/13] preempt: Remove PREEMPT_COUNT from Kconfig
@ 2020-09-14 20:42   ` Thomas Gleixner
  0 siblings, 0 replies; 282+ messages in thread
From: Thomas Gleixner @ 2020-09-14 20:42 UTC (permalink / raw)
  To: LKML
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, linux-mm, linux-kselftest,
	linux-hexagon, Will Deacon, Ingo Molnar, Anton Ivanov,
	linux-arch, Vincent Guittot, Brian Cain, Richard Weinberger,
	Russell King, David Airlie, Ingo Molnar, Geert Uytterhoeven,
	Mel Gorman, intel-gfx, Matt Turner, Valentin Schneider,
	linux-xtensa, Shuah Khan, Paul E. McKenney, Jeff Dike, linux-um,
	Josh Triplett, Steven Rostedt, rcu, linux-m68k, Ivan Kokshaysky,
	Rodrigo Vivi, Dietmar Eggemann, linux-arm-kernel,
	Richard Henderson, Chris Zankel, Max Filippov, Linus Torvalds,
	linux-alpha, Mathieu Desnoyers, Andrew Morton,
	Daniel Bristot de Oliveira

All conditionals and irritations are gone.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 kernel/Kconfig.preempt |    3 ---
 1 file changed, 3 deletions(-)

--- a/kernel/Kconfig.preempt
+++ b/kernel/Kconfig.preempt
@@ -74,8 +74,5 @@ config PREEMPT_RT
 
 endchoice
 
-config PREEMPT_COUNT
-       def_bool y
-
 config PREEMPTION
        bool

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply	[flat|nested] 282+ messages in thread

* [Intel-gfx] [patch 13/13] preempt: Remove PREEMPT_COUNT from Kconfig
@ 2020-09-14 20:42   ` Thomas Gleixner
  0 siblings, 0 replies; 282+ messages in thread
From: Thomas Gleixner @ 2020-09-14 20:42 UTC (permalink / raw)
  To: LKML
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, linux-mm, linux-kselftest,
	linux-hexagon, Will Deacon, Ingo Molnar, Anton Ivanov,
	linux-arch, Brian Cain, Richard Weinberger, Russell King,
	David Airlie, Ingo Molnar, Geert Uytterhoeven, Mel Gorman,
	intel-gfx, Matt Turner, Valentin Schneider, linux-xtensa,
	Shuah Khan, Paul E. McKenney, Jeff Dike, linux-um, Josh Triplett,
	Steven Rostedt, rcu, linux-m68k, Ivan Kokshaysky,
	Dietmar Eggemann, linux-arm-kernel, Richard Henderson,
	Chris Zankel, Max Filippov, Linus Torvalds, linux-alpha,
	Mathieu Desnoyers, Andrew Morton, Daniel Bristot de Oliveira

All conditionals and irritations are gone.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 kernel/Kconfig.preempt |    3 ---
 1 file changed, 3 deletions(-)

--- a/kernel/Kconfig.preempt
+++ b/kernel/Kconfig.preempt
@@ -74,8 +74,5 @@ config PREEMPT_RT
 
 endchoice
 
-config PREEMPT_COUNT
-       def_bool y
-
 config PREEMPTION
        bool

_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

^ permalink raw reply	[flat|nested] 282+ messages in thread

* [patch 13/13] preempt: Remove PREEMPT_COUNT from Kconfig
@ 2020-09-14 20:42   ` Thomas Gleixner
  0 siblings, 0 replies; 282+ messages in thread
From: Thomas Gleixner @ 2020-09-14 20:42 UTC (permalink / raw)
  To: LKML
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Joonas Lahtinen, Lai Jiangshan, dri-devel, Ben Segall, linux-mm,
	linux-kselftest, linux-hexagon, Will Deacon, Ingo Molnar,
	Anton Ivanov, linux-arch, Vincent Guittot, Brian Cain,
	Richard Weinberger, Russell King, David Airlie, Ingo Molnar,
	Geert Uytterhoeven, Mel Gorman, intel-gfx, Matt Turner, Valentin

All conditionals and irritations are gone.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 kernel/Kconfig.preempt |    3 ---
 1 file changed, 3 deletions(-)

--- a/kernel/Kconfig.preempt
+++ b/kernel/Kconfig.preempt
@@ -74,8 +74,5 @@ config PREEMPT_RT
 
 endchoice
 
-config PREEMPT_COUNT
-       def_bool y
-
 config PREEMPTION
        bool

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
  2020-09-14 20:42 ` Thomas Gleixner
                     ` (2 preceding siblings ...)
  (?)
@ 2020-09-14 20:54   ` Steven Rostedt
  -1 siblings, 0 replies; 282+ messages in thread
From: Steven Rostedt @ 2020-09-14 20:54 UTC (permalink / raw)
  To: Thomas Gleixner
  Cc: LKML, linux-arch, Linus Torvalds, Sebastian Andrzej Siewior,
	Valentin Schneider, Richard Henderson, Ivan Kokshaysky,
	Matt Turner, linux-alpha, Jeff Dike, Richard Weinberger,
	Anton Ivanov, linux-um, Brian Cain, linux-hexagon,
	Geert Uytterhoeven, linux-m68k, Ingo Molnar, Peter Zijlstra,
	Juri Lelli, Vincent Guittot, Dietmar Eggemann, Ben Segall,
	Mel Gorman, Daniel Bristot de Oliveira, Will Deacon,
	Andrew Morton, linux-mm, Ingo Molnar, Russell King,
	linux-arm-kernel, Chris Zankel, Max Filippov, linux-xtensa,
	Jani Nikula, Joonas Lahtinen, Rodrigo Vivi, David Airlie,
	Daniel Vetter, intel-gfx, dri-devel, Paul E. McKenney,
	Josh Triplett, Mathieu Desnoyers, Lai Jiangshan, Shuah Khan, rcu,
	linux-kselftest

On Mon, 14 Sep 2020 22:42:09 +0200
Thomas Gleixner <tglx@linutronix.de> wrote:

> 21 files changed, 23 insertions(+), 92 deletions(-)

This alone makes it look promising, and hopefully acceptable by Linus :-)

-- Steve


^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-14 20:54   ` Steven Rostedt
  0 siblings, 0 replies; 282+ messages in thread
From: Steven Rostedt @ 2020-09-14 20:54 UTC (permalink / raw)
  To: Thomas Gleixner
  Cc: Juri Lelli, Peter Zijlstra, Linus Torvalds,
	Sebastian Andrzej Siewior, Joonas Lahtinen, Lai Jiangshan,
	dri-devel, Ben Segall, linux-mm, linux-kselftest, linux-hexagon,
	Will Deacon, Ingo Molnar, Anton Ivanov, linux-arch,
	Vincent Guittot, Brian Cain, Richard Weinberger, Russell King,
	David Airlie, Ingo Molnar, Geert Uytterhoeven, Mel Gorman,
	intel-gfx, Matt Turner, Valentin Schneider, linux-xtensa,
	Shuah Khan, Paul E. McKenney, Jeff Dike, linux-um, Josh Triplett,
	Jani Nikula, rcu, linux-m68k, Ivan Kokshaysky, Rodrigo Vivi,
	Dietmar Eggemann, linux-arm-kernel, Richard Henderson,
	Chris Zankel, Max Filippov, LKML, Daniel Vetter, linux-alpha,
	Mathieu Desnoyers, Andrew Morton, Daniel Bristot de Oliveira

On Mon, 14 Sep 2020 22:42:09 +0200
Thomas Gleixner <tglx@linutronix.de> wrote:

> 21 files changed, 23 insertions(+), 92 deletions(-)

This alone makes it look promising, and hopefully acceptable by Linus :-)

-- Steve

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-14 20:54   ` Steven Rostedt
  0 siblings, 0 replies; 282+ messages in thread
From: Steven Rostedt @ 2020-09-14 20:54 UTC (permalink / raw)
  To: Thomas Gleixner
  Cc: Juri Lelli, Peter Zijlstra, Linus Torvalds,
	Sebastian Andrzej Siewior, Lai Jiangshan, dri-devel, Ben Segall,
	linux-mm, linux-kselftest, linux-hexagon, Will Deacon,
	Ingo Molnar, Anton Ivanov, linux-arch, Vincent Guittot,
	Brian Cain, Richard Weinberger, Russell King, David Airlie,
	Ingo Molnar, Geert Uytterhoeven, Mel Gorman, intel-gfx,
	Matt Turner, Valentin Schneider, linux-xtensa, Shuah Khan,
	Paul E. McKenney, Jeff Dike, linux-um, Josh Triplett, rcu,
	linux-m68k, Ivan Kokshaysky, Rodrigo Vivi, Dietmar Eggemann,
	linux-arm-kernel, Richard Henderson, Chris Zankel, Max Filippov,
	LKML, linux-alpha, Mathieu Desnoyers, Andrew Morton,
	Daniel Bristot de Oliveira

On Mon, 14 Sep 2020 22:42:09 +0200
Thomas Gleixner <tglx@linutronix.de> wrote:

> 21 files changed, 23 insertions(+), 92 deletions(-)

This alone makes it look promising, and hopefully acceptable by Linus :-)

-- Steve
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [Intel-gfx] [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-14 20:54   ` Steven Rostedt
  0 siblings, 0 replies; 282+ messages in thread
From: Steven Rostedt @ 2020-09-14 20:54 UTC (permalink / raw)
  To: Thomas Gleixner
  Cc: Juri Lelli, Peter Zijlstra, Linus Torvalds,
	Sebastian Andrzej Siewior, Lai Jiangshan, dri-devel, Ben Segall,
	linux-mm, linux-kselftest, linux-hexagon, Will Deacon,
	Ingo Molnar, Anton Ivanov, linux-arch, Brian Cain,
	Richard Weinberger, Russell King, David Airlie, Ingo Molnar,
	Geert Uytterhoeven, Mel Gorman, intel-gfx, Matt Turner,
	Valentin Schneider, linux-xtensa, Shuah Khan, Paul E. McKenney,
	Jeff Dike, linux-um, Josh Triplett, rcu, linux-m68k,
	Ivan Kokshaysky, Dietmar Eggemann, linux-arm-kernel,
	Richard Henderson, Chris Zankel, Max Filippov, LKML, linux-alpha,
	Mathieu Desnoyers, Andrew Morton, Daniel Bristot de Oliveira

On Mon, 14 Sep 2020 22:42:09 +0200
Thomas Gleixner <tglx@linutronix.de> wrote:

> 21 files changed, 23 insertions(+), 92 deletions(-)

This alone makes it look promising, and hopefully acceptable by Linus :-)

-- Steve
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-14 20:54   ` Steven Rostedt
  0 siblings, 0 replies; 282+ messages in thread
From: Steven Rostedt @ 2020-09-14 20:54 UTC (permalink / raw)
  To: Thomas Gleixner
  Cc: Juri Lelli, Peter Zijlstra, Linus Torvalds,
	Sebastian Andrzej Siewior, Lai Jiangshan, dri-devel, Ben Segall,
	linux-mm, linux-kselftest, linux-hexagon, Will Deacon,
	Ingo Molnar, Anton Ivanov, linux-arch, Brian Cain,
	Richard Weinberger, Russell King, David Airlie, Ingo Molnar,
	Geert Uytterhoeven, Mel Gorman, intel-gfx, Matt Turner,
	Valentin Schneider, linux-xtensa

On Mon, 14 Sep 2020 22:42:09 +0200
Thomas Gleixner <tglx@linutronix.de> wrote:

> 21 files changed, 23 insertions(+), 92 deletions(-)

This alone makes it look promising, and hopefully acceptable by Linus :-)

-- Steve

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
  2020-09-14 20:42 ` Thomas Gleixner
                     ` (2 preceding siblings ...)
  (?)
@ 2020-09-14 20:59   ` Linus Torvalds
  -1 siblings, 0 replies; 282+ messages in thread
From: Linus Torvalds @ 2020-09-14 20:59 UTC (permalink / raw)
  To: Thomas Gleixner
  Cc: LKML, linux-arch, Sebastian Andrzej Siewior, Valentin Schneider,
	Richard Henderson, Ivan Kokshaysky, Matt Turner, alpha,
	Jeff Dike, Richard Weinberger, Anton Ivanov, linux-um,
	Brian Cain, linux-hexagon, Geert Uytterhoeven, linux-m68k,
	Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
	Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman,
	Daniel Bristot de Oliveira, Will Deacon, Andrew Morton, Linux-MM,
	Ingo Molnar, Russell King, Linux ARM, Chris Zankel, Max Filippov,
	linux-xtensa, Jani Nikula, Joonas Lahtinen, Rodrigo Vivi,
	David Airlie, Daniel Vetter, intel-gfx, dri-devel,
	Paul E. McKenney, Josh Triplett, Mathieu Desnoyers,
	Lai Jiangshan, Shuah Khan, rcu,
	open list:KERNEL SELFTEST FRAMEWORK

On Mon, Sep 14, 2020 at 1:45 PM Thomas Gleixner <tglx@linutronix.de> wrote:
>
> Recently merged code does:
>
>          gfp = preemptible() ? GFP_KERNEL : GFP_ATOMIC;
>
> Looks obviously correct, except for the fact that preemptible() is
> unconditionally false for CONFIF_PREEMPT_COUNT=n, i.e. all allocations in
> that code use GFP_ATOMIC on such kernels.

I don't think this is a good reason to entirely get rid of the no-preempt thing.

The above is just garbage. It's bogus. You can't do it.

Blaming the no-preempt code for this bug is extremely unfair, imho.

And the no-preempt code does help make for much better code generation
for simple spinlocks.

Where is that horribly buggy recent code? It's not in that exact
format, certainly, since 'grep' doesn't find it.

             Linus


^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-14 20:59   ` Linus Torvalds
  0 siblings, 0 replies; 282+ messages in thread
From: Linus Torvalds @ 2020-09-14 20:59 UTC (permalink / raw)
  To: Thomas Gleixner
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Joonas Lahtinen, Lai Jiangshan, dri-devel, Ben Segall, Linux-MM,
	open list:KERNEL SELFTEST FRAMEWORK, linux-hexagon, Will Deacon,
	Ingo Molnar, Anton Ivanov, linux-arch, Vincent Guittot,
	Brian Cain, Richard Weinberger, Russell King, David Airlie,
	Ingo Molnar, Geert Uytterhoeven, Mel Gorman, intel-gfx,
	Matt Turner, Valentin Schneider, linux-xtensa, Shuah Khan,
	Paul E. McKenney, Jeff Dike, linux-um, Josh Triplett,
	Steven Rostedt, rcu, linux-m68k, Ivan Kokshaysky, Jani Nikula,
	Rodrigo Vivi, Dietmar Eggemann, Linux ARM, Richard Henderson,
	Chris Zankel, Max Filippov, LKML, Daniel Vetter, alpha,
	Mathieu Desnoyers, Andrew Morton, Daniel Bristot de Oliveira

On Mon, Sep 14, 2020 at 1:45 PM Thomas Gleixner <tglx@linutronix.de> wrote:
>
> Recently merged code does:
>
>          gfp = preemptible() ? GFP_KERNEL : GFP_ATOMIC;
>
> Looks obviously correct, except for the fact that preemptible() is
> unconditionally false for CONFIF_PREEMPT_COUNT=n, i.e. all allocations in
> that code use GFP_ATOMIC on such kernels.

I don't think this is a good reason to entirely get rid of the no-preempt thing.

The above is just garbage. It's bogus. You can't do it.

Blaming the no-preempt code for this bug is extremely unfair, imho.

And the no-preempt code does help make for much better code generation
for simple spinlocks.

Where is that horribly buggy recent code? It's not in that exact
format, certainly, since 'grep' doesn't find it.

             Linus

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-14 20:59   ` Linus Torvalds
  0 siblings, 0 replies; 282+ messages in thread
From: Linus Torvalds @ 2020-09-14 20:59 UTC (permalink / raw)
  To: Thomas Gleixner
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, Linux-MM,
	open list:KERNEL SELFTEST FRAMEWORK, linux-hexagon, Will Deacon,
	Ingo Molnar, Anton Ivanov, linux-arch, Vincent Guittot,
	Brian Cain, Richard Weinberger, Russell King, David Airlie,
	Ingo Molnar, Geert Uytterhoeven, Mel Gorman, intel-gfx,
	Matt Turner, Valentin Schneider, linux-xtensa, Shuah Khan,
	Paul E. McKenney, Jeff Dike, linux-um, Josh Triplett,
	Steven Rostedt, rcu, linux-m68k, Ivan Kokshaysky, Rodrigo Vivi,
	Dietmar Eggemann, Linux ARM, Richard Henderson, Chris Zankel,
	Max Filippov, LKML, alpha, Mathieu Desnoyers, Andrew Morton,
	Daniel Bristot de Oliveira

On Mon, Sep 14, 2020 at 1:45 PM Thomas Gleixner <tglx@linutronix.de> wrote:
>
> Recently merged code does:
>
>          gfp = preemptible() ? GFP_KERNEL : GFP_ATOMIC;
>
> Looks obviously correct, except for the fact that preemptible() is
> unconditionally false for CONFIF_PREEMPT_COUNT=n, i.e. all allocations in
> that code use GFP_ATOMIC on such kernels.

I don't think this is a good reason to entirely get rid of the no-preempt thing.

The above is just garbage. It's bogus. You can't do it.

Blaming the no-preempt code for this bug is extremely unfair, imho.

And the no-preempt code does help make for much better code generation
for simple spinlocks.

Where is that horribly buggy recent code? It's not in that exact
format, certainly, since 'grep' doesn't find it.

             Linus
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [Intel-gfx] [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-14 20:59   ` Linus Torvalds
  0 siblings, 0 replies; 282+ messages in thread
From: Linus Torvalds @ 2020-09-14 20:59 UTC (permalink / raw)
  To: Thomas Gleixner
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, Linux-MM,
	open list:KERNEL SELFTEST FRAMEWORK, linux-hexagon, Will Deacon,
	Ingo Molnar, Anton Ivanov, linux-arch, Brian Cain,
	Richard Weinberger, Russell King, David Airlie, Ingo Molnar,
	Geert Uytterhoeven, Mel Gorman, intel-gfx, Matt Turner,
	Valentin Schneider, linux-xtensa, Shuah Khan, Paul E. McKenney,
	Jeff Dike, linux-um, Josh Triplett, Steven Rostedt, rcu,
	linux-m68k, Ivan Kokshaysky, Dietmar Eggemann, Linux ARM,
	Richard Henderson, Chris Zankel, Max Filippov, LKML, alpha,
	Mathieu Desnoyers, Andrew Morton, Daniel Bristot de Oliveira

On Mon, Sep 14, 2020 at 1:45 PM Thomas Gleixner <tglx@linutronix.de> wrote:
>
> Recently merged code does:
>
>          gfp = preemptible() ? GFP_KERNEL : GFP_ATOMIC;
>
> Looks obviously correct, except for the fact that preemptible() is
> unconditionally false for CONFIF_PREEMPT_COUNT=n, i.e. all allocations in
> that code use GFP_ATOMIC on such kernels.

I don't think this is a good reason to entirely get rid of the no-preempt thing.

The above is just garbage. It's bogus. You can't do it.

Blaming the no-preempt code for this bug is extremely unfair, imho.

And the no-preempt code does help make for much better code generation
for simple spinlocks.

Where is that horribly buggy recent code? It's not in that exact
format, certainly, since 'grep' doesn't find it.

             Linus
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-14 20:59   ` Linus Torvalds
  0 siblings, 0 replies; 282+ messages in thread
From: Linus Torvalds @ 2020-09-14 20:59 UTC (permalink / raw)
  To: Thomas Gleixner
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, Linux-MM,
	open list:KERNEL SELFTEST FRAMEWORK, linux-hexagon, Will Deacon,
	Ingo Molnar, Anton Ivanov, linux-arch, Brian Cain,
	Richard Weinberger, Russell King, David Airlie, Ingo Molnar,
	Geert Uytterhoeven, Mel Gorman, intel-gfx, Matt Turner,
	Valentin Schneider

On Mon, Sep 14, 2020 at 1:45 PM Thomas Gleixner <tglx@linutronix.de> wrote:
>
> Recently merged code does:
>
>          gfp = preemptible() ? GFP_KERNEL : GFP_ATOMIC;
>
> Looks obviously correct, except for the fact that preemptible() is
> unconditionally false for CONFIF_PREEMPT_COUNT=n, i.e. all allocations in
> that code use GFP_ATOMIC on such kernels.

I don't think this is a good reason to entirely get rid of the no-preempt thing.

The above is just garbage. It's bogus. You can't do it.

Blaming the no-preempt code for this bug is extremely unfair, imho.

And the no-preempt code does help make for much better code generation
for simple spinlocks.

Where is that horribly buggy recent code? It's not in that exact
format, certainly, since 'grep' doesn't find it.

             Linus

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
  2020-09-14 20:59   ` Linus Torvalds
                       ` (3 preceding siblings ...)
  (?)
@ 2020-09-14 21:55     ` Thomas Gleixner
  -1 siblings, 0 replies; 282+ messages in thread
From: Thomas Gleixner @ 2020-09-14 21:55 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: LKML, linux-arch, Sebastian Andrzej Siewior, Valentin Schneider,
	Richard Henderson, Ivan Kokshaysky, Matt Turner, alpha,
	Jeff Dike, Richard Weinberger, Anton Ivanov, linux-um,
	Brian Cain, linux-hexagon, Geert Uytterhoeven, linux-m68k,
	Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
	Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman,
	Daniel Bristot de Oliveira, Will Deacon, Andrew Morton, Linux-MM,
	Ingo Molnar, Russell King, Linux ARM, Chris Zankel, Max Filippov,
	linux-xtensa, Jani Nikula, Joonas Lahtinen, Rodrigo Vivi,
	David Airlie, Daniel Vetter, intel-gfx, dri-devel,
	Paul E. McKenney, Josh Triplett, Mathieu Desnoyers,
	Lai Jiangshan, Shuah Khan, rcu,
	open list:KERNEL SELFTEST FRAMEWORK

On Mon, Sep 14 2020 at 13:59, Linus Torvalds wrote:
> On Mon, Sep 14, 2020 at 1:45 PM Thomas Gleixner <tglx@linutronix.de> wrote:
>>
>> Recently merged code does:
>>
>>          gfp = preemptible() ? GFP_KERNEL : GFP_ATOMIC;
>>
>> Looks obviously correct, except for the fact that preemptible() is
>> unconditionally false for CONFIF_PREEMPT_COUNT=n, i.e. all allocations in
>> that code use GFP_ATOMIC on such kernels.
>
> I don't think this is a good reason to entirely get rid of the
> no-preempt thing.

I did not say that this is a good reason. It just illustrates the
problem.

> The above is just garbage. It's bogus. You can't do it.
>
> Blaming the no-preempt code for this bug is extremely unfair, imho.

I'm not blaming the no-preempt code. I'm blaming inconsistency and there
is no real good argument for inconsistent behaviour, TBH.

> And the no-preempt code does help make for much better code generation
> for simple spinlocks.

Yes it does generate better code, but I tried hard to spot a difference
in various metrics exposed by perf. It's all in the noise and I only
can spot a difference when the actual preemption check after the
decrement which still depends on CONFIG_PREEMPT is in place, but that's
not the case for PREEMPT_NONE or PREEMPT_VOLUNTARY kernels where the
decrement is just a decrement w/o any conditional behind it.

> Where is that horribly buggy recent code? It's not in that exact
> format, certainly, since 'grep' doesn't find it.

Bah, that was stuff in next which got dropped again.

But just look at any check which uses preemptible(), especially those
which check !preemptible():

In the X86 #GP handler we have:

	/*
	 * To be potentially processing a kprobe fault and to trust the result
	 * from kprobe_running(), we have to be non-preemptible.
	 */
	if (!preemptible() &&
	    kprobe_running() &&
	    kprobe_fault_handler(regs, X86_TRAP_GP))
		goto exit;

and a similar check in the S390 code in kprobe_exceptions_notify(). That
all magically 'works' because that code might have been actually tested
with lockdep enabled which enforces PREEMPT_COUNT...

The SG code has some interesting usage as well:

		if (miter->__flags & SG_MITER_ATOMIC) {
			WARN_ON_ONCE(preemptible());
			kunmap_atomic(miter->addr);

How is that WARN_ON_ONCE() supposed to catch anything? Especially as
calling code does:

	flags = SG_MITER_TO_SG;
	if (!preemptible())
		flags |= SG_MITER_ATOMIC;

which is equally useless on kernels which have PREEMPT_COUNT=n.

There are bugs which are related to in_atomic() or other in_***() usage
all over the place as well.

Inconsistency at the core level is a clear recipe for disaster and at
some point we have to bite the bullet and accept that consistency is
more important than the non measurable 3 cycles?

Thanks,

        tglx


^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-14 21:55     ` Thomas Gleixner
  0 siblings, 0 replies; 282+ messages in thread
From: Thomas Gleixner @ 2020-09-14 21:55 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Joonas Lahtinen, Lai Jiangshan, dri-devel, Ben Segall, Linux-MM,
	open list:KERNEL SELFTEST FRAMEWORK, linux-hexagon, Will Deacon,
	Ingo Molnar, Anton Ivanov, linux-arch, Vincent Guittot,
	Brian Cain, Richard Weinberger, Russell King, David Airlie,
	Ingo Molnar, Geert Uytterhoeven, Mel Gorman, intel-gfx,
	Matt Turner, Valentin Schneider, linux-xtensa, Shuah Khan,
	Paul E. McKenney, Jeff Dike, linux-um, Josh Triplett,
	Steven Rostedt, rcu, linux-m68k, Ivan Kokshaysky, Jani Nikula,
	Rodrigo Vivi, Dietmar Eggemann, Linux ARM, Richard Henderson,
	Chris Zankel, Max Filippov, LKML, Daniel Vetter, alpha,
	Mathieu Desnoyers, Andrew Morton, Daniel Bristot de Oliveira

On Mon, Sep 14 2020 at 13:59, Linus Torvalds wrote:
> On Mon, Sep 14, 2020 at 1:45 PM Thomas Gleixner <tglx@linutronix.de> wrote:
>>
>> Recently merged code does:
>>
>>          gfp = preemptible() ? GFP_KERNEL : GFP_ATOMIC;
>>
>> Looks obviously correct, except for the fact that preemptible() is
>> unconditionally false for CONFIF_PREEMPT_COUNT=n, i.e. all allocations in
>> that code use GFP_ATOMIC on such kernels.
>
> I don't think this is a good reason to entirely get rid of the
> no-preempt thing.

I did not say that this is a good reason. It just illustrates the
problem.

> The above is just garbage. It's bogus. You can't do it.
>
> Blaming the no-preempt code for this bug is extremely unfair, imho.

I'm not blaming the no-preempt code. I'm blaming inconsistency and there
is no real good argument for inconsistent behaviour, TBH.

> And the no-preempt code does help make for much better code generation
> for simple spinlocks.

Yes it does generate better code, but I tried hard to spot a difference
in various metrics exposed by perf. It's all in the noise and I only
can spot a difference when the actual preemption check after the
decrement which still depends on CONFIG_PREEMPT is in place, but that's
not the case for PREEMPT_NONE or PREEMPT_VOLUNTARY kernels where the
decrement is just a decrement w/o any conditional behind it.

> Where is that horribly buggy recent code? It's not in that exact
> format, certainly, since 'grep' doesn't find it.

Bah, that was stuff in next which got dropped again.

But just look at any check which uses preemptible(), especially those
which check !preemptible():

In the X86 #GP handler we have:

	/*
	 * To be potentially processing a kprobe fault and to trust the result
	 * from kprobe_running(), we have to be non-preemptible.
	 */
	if (!preemptible() &&
	    kprobe_running() &&
	    kprobe_fault_handler(regs, X86_TRAP_GP))
		goto exit;

and a similar check in the S390 code in kprobe_exceptions_notify(). That
all magically 'works' because that code might have been actually tested
with lockdep enabled which enforces PREEMPT_COUNT...

The SG code has some interesting usage as well:

		if (miter->__flags & SG_MITER_ATOMIC) {
			WARN_ON_ONCE(preemptible());
			kunmap_atomic(miter->addr);

How is that WARN_ON_ONCE() supposed to catch anything? Especially as
calling code does:

	flags = SG_MITER_TO_SG;
	if (!preemptible())
		flags |= SG_MITER_ATOMIC;

which is equally useless on kernels which have PREEMPT_COUNT=n.

There are bugs which are related to in_atomic() or other in_***() usage
all over the place as well.

Inconsistency at the core level is a clear recipe for disaster and at
some point we have to bite the bullet and accept that consistency is
more important than the non measurable 3 cycles?

Thanks,

        tglx

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-14 21:55     ` Thomas Gleixner
  0 siblings, 0 replies; 282+ messages in thread
From: Thomas Gleixner @ 2020-09-14 21:55 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, Linux-MM,
	open list:KERNEL SELFTEST FRAMEWORK, linux-hexagon, Will Deacon,
	Ingo Molnar, Anton Ivanov, linux-arch, Vincent Guittot,
	Brian Cain, Richard Weinberger, Russell King, David Airlie,
	Ingo Molnar, Geert Uytterhoeven, Mel Gorman, intel-gfx,
	Matt Turner, Valentin Schneider, linux-xtensa, Shuah Khan,
	Paul E. McKenney, Jeff Dike, linux-um, Josh Triplett,
	Steven Rostedt, rcu, linux-m68k, Ivan Kokshaysky, Rodrigo Vivi,
	Dietmar Eggemann, Linux ARM, Richard Henderson, Chris Zankel,
	Max Filippov, LKML, alpha, Mathieu Desnoyers, Andrew Morton,
	Daniel Bristot de Oliveira

On Mon, Sep 14 2020 at 13:59, Linus Torvalds wrote:
> On Mon, Sep 14, 2020 at 1:45 PM Thomas Gleixner <tglx@linutronix.de> wrote:
>>
>> Recently merged code does:
>>
>>          gfp = preemptible() ? GFP_KERNEL : GFP_ATOMIC;
>>
>> Looks obviously correct, except for the fact that preemptible() is
>> unconditionally false for CONFIF_PREEMPT_COUNT=n, i.e. all allocations in
>> that code use GFP_ATOMIC on such kernels.
>
> I don't think this is a good reason to entirely get rid of the
> no-preempt thing.

I did not say that this is a good reason. It just illustrates the
problem.

> The above is just garbage. It's bogus. You can't do it.
>
> Blaming the no-preempt code for this bug is extremely unfair, imho.

I'm not blaming the no-preempt code. I'm blaming inconsistency and there
is no real good argument for inconsistent behaviour, TBH.

> And the no-preempt code does help make for much better code generation
> for simple spinlocks.

Yes it does generate better code, but I tried hard to spot a difference
in various metrics exposed by perf. It's all in the noise and I only
can spot a difference when the actual preemption check after the
decrement which still depends on CONFIG_PREEMPT is in place, but that's
not the case for PREEMPT_NONE or PREEMPT_VOLUNTARY kernels where the
decrement is just a decrement w/o any conditional behind it.

> Where is that horribly buggy recent code? It's not in that exact
> format, certainly, since 'grep' doesn't find it.

Bah, that was stuff in next which got dropped again.

But just look at any check which uses preemptible(), especially those
which check !preemptible():

In the X86 #GP handler we have:

	/*
	 * To be potentially processing a kprobe fault and to trust the result
	 * from kprobe_running(), we have to be non-preemptible.
	 */
	if (!preemptible() &&
	    kprobe_running() &&
	    kprobe_fault_handler(regs, X86_TRAP_GP))
		goto exit;

and a similar check in the S390 code in kprobe_exceptions_notify(). That
all magically 'works' because that code might have been actually tested
with lockdep enabled which enforces PREEMPT_COUNT...

The SG code has some interesting usage as well:

		if (miter->__flags & SG_MITER_ATOMIC) {
			WARN_ON_ONCE(preemptible());
			kunmap_atomic(miter->addr);

How is that WARN_ON_ONCE() supposed to catch anything? Especially as
calling code does:

	flags = SG_MITER_TO_SG;
	if (!preemptible())
		flags |= SG_MITER_ATOMIC;

which is equally useless on kernels which have PREEMPT_COUNT=n.

There are bugs which are related to in_atomic() or other in_***() usage
all over the place as well.

Inconsistency at the core level is a clear recipe for disaster and at
some point we have to bite the bullet and accept that consistency is
more important than the non measurable 3 cycles?

Thanks,

        tglx
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [Intel-gfx] [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-14 21:55     ` Thomas Gleixner
  0 siblings, 0 replies; 282+ messages in thread
From: Thomas Gleixner @ 2020-09-14 21:55 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, Linux-MM,
	open list:KERNEL SELFTEST FRAMEWORK, linux-hexagon, Will Deacon,
	Ingo Molnar, Anton Ivanov, linux-arch, Brian Cain,
	Richard Weinberger, Russell King, David Airlie, Ingo Molnar,
	Geert Uytterhoeven, Mel Gorman, intel-gfx, Matt Turner,
	Valentin Schneider, linux-xtensa, Shuah Khan, Paul E. McKenney,
	Jeff Dike, linux-um, Josh Triplett, Steven Rostedt, rcu,
	linux-m68k, Ivan Kokshaysky, Dietmar Eggemann, Linux ARM,
	Richard Henderson, Chris Zankel, Max Filippov, LKML, alpha,
	Mathieu Desnoyers, Andrew Morton, Daniel Bristot de Oliveira

On Mon, Sep 14 2020 at 13:59, Linus Torvalds wrote:
> On Mon, Sep 14, 2020 at 1:45 PM Thomas Gleixner <tglx@linutronix.de> wrote:
>>
>> Recently merged code does:
>>
>>          gfp = preemptible() ? GFP_KERNEL : GFP_ATOMIC;
>>
>> Looks obviously correct, except for the fact that preemptible() is
>> unconditionally false for CONFIF_PREEMPT_COUNT=n, i.e. all allocations in
>> that code use GFP_ATOMIC on such kernels.
>
> I don't think this is a good reason to entirely get rid of the
> no-preempt thing.

I did not say that this is a good reason. It just illustrates the
problem.

> The above is just garbage. It's bogus. You can't do it.
>
> Blaming the no-preempt code for this bug is extremely unfair, imho.

I'm not blaming the no-preempt code. I'm blaming inconsistency and there
is no real good argument for inconsistent behaviour, TBH.

> And the no-preempt code does help make for much better code generation
> for simple spinlocks.

Yes it does generate better code, but I tried hard to spot a difference
in various metrics exposed by perf. It's all in the noise and I only
can spot a difference when the actual preemption check after the
decrement which still depends on CONFIG_PREEMPT is in place, but that's
not the case for PREEMPT_NONE or PREEMPT_VOLUNTARY kernels where the
decrement is just a decrement w/o any conditional behind it.

> Where is that horribly buggy recent code? It's not in that exact
> format, certainly, since 'grep' doesn't find it.

Bah, that was stuff in next which got dropped again.

But just look at any check which uses preemptible(), especially those
which check !preemptible():

In the X86 #GP handler we have:

	/*
	 * To be potentially processing a kprobe fault and to trust the result
	 * from kprobe_running(), we have to be non-preemptible.
	 */
	if (!preemptible() &&
	    kprobe_running() &&
	    kprobe_fault_handler(regs, X86_TRAP_GP))
		goto exit;

and a similar check in the S390 code in kprobe_exceptions_notify(). That
all magically 'works' because that code might have been actually tested
with lockdep enabled which enforces PREEMPT_COUNT...

The SG code has some interesting usage as well:

		if (miter->__flags & SG_MITER_ATOMIC) {
			WARN_ON_ONCE(preemptible());
			kunmap_atomic(miter->addr);

How is that WARN_ON_ONCE() supposed to catch anything? Especially as
calling code does:

	flags = SG_MITER_TO_SG;
	if (!preemptible())
		flags |= SG_MITER_ATOMIC;

which is equally useless on kernels which have PREEMPT_COUNT=n.

There are bugs which are related to in_atomic() or other in_***() usage
all over the place as well.

Inconsistency at the core level is a clear recipe for disaster and at
some point we have to bite the bullet and accept that consistency is
more important than the non measurable 3 cycles?

Thanks,

        tglx
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-14 21:55     ` Thomas Gleixner
  0 siblings, 0 replies; 282+ messages in thread
From: Thomas Gleixner @ 2020-09-14 21:55 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: LKML, linux-arch, Sebastian Andrzej Siewior, Valentin Schneider,
	Richard Henderson, Ivan Kokshaysky, Matt Turner, alpha,
	Jeff Dike, Richard Weinberger, Anton Ivanov, linux-um,
	Brian Cain, linux-hexagon, Geert Uytterhoeven, linux-m68k,
	Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
	Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman,
	Daniel Bristot de Oliveira, Will Deacon, Andrew Morton, Linux-MM,
	Ingo Molnar, Russell King, Linux ARM, Chris Zankel, Max Filippov,
	linux-xtensa, Jani Nikula, Joonas Lahtinen, Rodrigo Vivi,
	David Airlie, Daniel Vetter, intel-gfx, dri-devel,
	Paul E. McKenney, Josh Triplett, Mathieu Desnoyers,
	Lai Jiangshan, Shuah Khan, rcu,
	open list:KERNEL SELFTEST FRAMEWORK

On Mon, Sep 14 2020 at 13:59, Linus Torvalds wrote:
> On Mon, Sep 14, 2020 at 1:45 PM Thomas Gleixner <tglx@linutronix.de> wrote:
>>
>> Recently merged code does:
>>
>>          gfp = preemptible() ? GFP_KERNEL : GFP_ATOMIC;
>>
>> Looks obviously correct, except for the fact that preemptible() is
>> unconditionally false for CONFIF_PREEMPT_COUNT=n, i.e. all allocations in
>> that code use GFP_ATOMIC on such kernels.
>
> I don't think this is a good reason to entirely get rid of the
> no-preempt thing.

I did not say that this is a good reason. It just illustrates the
problem.

> The above is just garbage. It's bogus. You can't do it.
>
> Blaming the no-preempt code for this bug is extremely unfair, imho.

I'm not blaming the no-preempt code. I'm blaming inconsistency and there
is no real good argument for inconsistent behaviour, TBH.

> And the no-preempt code does help make for much better code generation
> for simple spinlocks.

Yes it does generate better code, but I tried hard to spot a difference
in various metrics exposed by perf. It's all in the noise and I only
can spot a difference when the actual preemption check after the
decrement which still depends on CONFIG_PREEMPT is in place, but that's
not the case for PREEMPT_NONE or PREEMPT_VOLUNTARY kernels where the
decrement is just a decrement w/o any conditional behind it.

> Where is that horribly buggy recent code? It's not in that exact
> format, certainly, since 'grep' doesn't find it.

Bah, that was stuff in next which got dropped again.

But just look at any check which uses preemptible(), especially those
which check !preemptible():

In the X86 #GP handler we have:

	/*
	 * To be potentially processing a kprobe fault and to trust the result
	 * from kprobe_running(), we have to be non-preemptible.
	 */
	if (!preemptible() &&
	    kprobe_running() &&
	    kprobe_fault_handler(regs, X86_TRAP_GP))
		goto exit;

and a similar check in the S390 code in kprobe_exceptions_notify(). That
all magically 'works' because that code might have been actually tested
with lockdep enabled which enforces PREEMPT_COUNT...

The SG code has some interesting usage as well:

		if (miter->__flags & SG_MITER_ATOMIC) {
			WARN_ON_ONCE(preemptible());
			kunmap_atomic(miter->addr);

How is that WARN_ON_ONCE() supposed to catch anything? Especially as
calling code does:

	flags = SG_MITER_TO_SG;
	if (!preemptible())
		flags |= SG_MITER_ATOMIC;

which is equally useless on kernels which have PREEMPT_COUNT=n.

There are bugs which are related to in_atomic() or other in_***() usage
all over the place as well.

Inconsistency at the core level is a clear recipe for disaster and at
some point we have to bite the bullet and accept that consistency is
more important than the non measurable 3 cycles?

Thanks,

        tglx


^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-14 21:55     ` Thomas Gleixner
  0 siblings, 0 replies; 282+ messages in thread
From: Thomas Gleixner @ 2020-09-14 21:55 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, Linux-MM,
	open list:KERNEL SELFTEST FRAMEWORK, linux-hexagon, Will Deacon,
	Ingo Molnar, Anton Ivanov, linux-arch, Brian Cain,
	Richard Weinberger, Russell King, David Airlie, Ingo Molnar,
	Geert Uytterhoeven, Mel Gorman, intel-gfx, Matt Turner,
	Valentin Schneider

On Mon, Sep 14 2020 at 13:59, Linus Torvalds wrote:
> On Mon, Sep 14, 2020 at 1:45 PM Thomas Gleixner <tglx@linutronix.de> wrote:
>>
>> Recently merged code does:
>>
>>          gfp = preemptible() ? GFP_KERNEL : GFP_ATOMIC;
>>
>> Looks obviously correct, except for the fact that preemptible() is
>> unconditionally false for CONFIF_PREEMPT_COUNT=n, i.e. all allocations in
>> that code use GFP_ATOMIC on such kernels.
>
> I don't think this is a good reason to entirely get rid of the
> no-preempt thing.

I did not say that this is a good reason. It just illustrates the
problem.

> The above is just garbage. It's bogus. You can't do it.
>
> Blaming the no-preempt code for this bug is extremely unfair, imho.

I'm not blaming the no-preempt code. I'm blaming inconsistency and there
is no real good argument for inconsistent behaviour, TBH.

> And the no-preempt code does help make for much better code generation
> for simple spinlocks.

Yes it does generate better code, but I tried hard to spot a difference
in various metrics exposed by perf. It's all in the noise and I only
can spot a difference when the actual preemption check after the
decrement which still depends on CONFIG_PREEMPT is in place, but that's
not the case for PREEMPT_NONE or PREEMPT_VOLUNTARY kernels where the
decrement is just a decrement w/o any conditional behind it.

> Where is that horribly buggy recent code? It's not in that exact
> format, certainly, since 'grep' doesn't find it.

Bah, that was stuff in next which got dropped again.

But just look at any check which uses preemptible(), especially those
which check !preemptible():

In the X86 #GP handler we have:

	/*
	 * To be potentially processing a kprobe fault and to trust the result
	 * from kprobe_running(), we have to be non-preemptible.
	 */
	if (!preemptible() &&
	    kprobe_running() &&
	    kprobe_fault_handler(regs, X86_TRAP_GP))
		goto exit;

and a similar check in the S390 code in kprobe_exceptions_notify(). That
all magically 'works' because that code might have been actually tested
with lockdep enabled which enforces PREEMPT_COUNT...

The SG code has some interesting usage as well:

		if (miter->__flags & SG_MITER_ATOMIC) {
			WARN_ON_ONCE(preemptible());
			kunmap_atomic(miter->addr);

How is that WARN_ON_ONCE() supposed to catch anything? Especially as
calling code does:

	flags = SG_MITER_TO_SG;
	if (!preemptible())
		flags |= SG_MITER_ATOMIC;

which is equally useless on kernels which have PREEMPT_COUNT=n.

There are bugs which are related to in_atomic() or other in_***() usage
all over the place as well.

Inconsistency at the core level is a clear recipe for disaster and at
some point we have to bite the bullet and accept that consistency is
more important than the non measurable 3 cycles?

Thanks,

        tglx

^ permalink raw reply	[flat|nested] 282+ messages in thread

* [Intel-gfx] ✗ Fi.CI.BUILD: failure for preempt: Make preempt count unconditional
  2020-09-14 20:42 ` Thomas Gleixner
                   ` (18 preceding siblings ...)
  (?)
@ 2020-09-14 22:01 ` Patchwork
  -1 siblings, 0 replies; 282+ messages in thread
From: Patchwork @ 2020-09-14 22:01 UTC (permalink / raw)
  To: Thomas Gleixner; +Cc: intel-gfx

== Series Details ==

Series: preempt: Make preempt count unconditional
URL   : https://patchwork.freedesktop.org/series/81661/
State : failure

== Summary ==

Applying: lib/debug: Remove pointless ARCH_NO_PREEMPT dependencies
Applying: preempt: Make preempt count unconditional
Applying: preempt: Clenaup PREEMPT_COUNT leftovers
Applying: lockdep: Clenaup PREEMPT_COUNT leftovers
Applying: mm/pagemap: Clenaup PREEMPT_COUNT leftovers
Applying: locking/bitspinlock: Clenaup PREEMPT_COUNT leftovers
Applying: uaccess: Clenaup PREEMPT_COUNT leftovers
Applying: sched: Clenaup PREEMPT_COUNT leftovers
Applying: ARM: Clenaup PREEMPT_COUNT leftovers
Applying: xtensa: Clenaup PREEMPT_COUNT leftovers
Applying: drm/i915: Clenaup PREEMPT_COUNT leftovers
error: sha1 information is lacking or useless (drivers/gpu/drm/i915/Kconfig.debug).
error: could not build fake ancestor
hint: Use 'git am --show-current-patch=diff' to see the failed patch
Patch failed at 0011 drm/i915: Clenaup PREEMPT_COUNT leftovers
When you have resolved this problem, run "git am --continue".
If you prefer to skip this patch, run "git am --skip" instead.
To restore the original branch and stop patching, run "git am --abort".


_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
  2020-09-14 21:55     ` Thomas Gleixner
  (?)
  (?)
@ 2020-09-14 22:24       ` Linus Torvalds
  -1 siblings, 0 replies; 282+ messages in thread
From: Linus Torvalds @ 2020-09-14 22:24 UTC (permalink / raw)
  To: Thomas Gleixner, Ard Biesheuvel, Herbert Xu
  Cc: LKML, linux-arch, Sebastian Andrzej Siewior, Valentin Schneider,
	Richard Henderson, Ivan Kokshaysky, Matt Turner, alpha,
	Jeff Dike, Richard Weinberger, Anton Ivanov, linux-um,
	Brian Cain, linux-hexagon, Geert Uytterhoeven, linux-m68k,
	Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
	Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman,
	Daniel Bristot de Oliveira, Will Deacon, Andrew Morton, Linux-MM,
	Ingo Molnar, Russell King, Linux ARM, Chris Zankel, Max Filippov,
	linux-xtensa, Jani Nikula, Joonas Lahtinen, Rodrigo Vivi,
	David Airlie, Daniel Vetter, intel-gfx, dri-devel,
	Paul E. McKenney, Josh Triplett, Mathieu Desnoyers,
	Lai Jiangshan, Shuah Khan, rcu,
	open list:KERNEL SELFTEST FRAMEWORK

On Mon, Sep 14, 2020 at 2:55 PM Thomas Gleixner <tglx@linutronix.de> wrote:
>
> Yes it does generate better code, but I tried hard to spot a difference
> in various metrics exposed by perf. It's all in the noise and I only
> can spot a difference when the actual preemption check after the
> decrement

I'm somewhat more worried about the small-device case.

That said, the diffstat certainly has its very clear charm, and I do
agree that it makes things simpler.

I'm just not convinced people should ever EVER do things like that "if
(preemptible())" garbage. It sounds like somebody is doing seriously
bad things.

The chacha20poly1305 code does look fundamentally broken. But no, the
fix is not to make "preemptible" work with spinlocks, the fix is to
not *do* that kind of broken things.

Those things would be broken even if you changed the semantics of
preemptible. There's no way that it's valid to say "use this debug
flag to decide if we should do atomic allocations or not".

It smells like "I got a debug failure, so I'm papering it over by
checking the thing the debug code checks for".

The debug check is to catch the obvious bad cases. It's not the _only_
bad cases, so copying the debug check test is just completely wrong.

Ard and Herbert added to participants: see
chacha20poly1305_crypt_sg_inplace(), which does

        flags = SG_MITER_TO_SG;
        if (!preemptible())
                flags |= SG_MITER_ATOMIC;

introduced in commit d95312a3ccc0 ("crypto: lib/chacha20poly1305 -
reimplement crypt_from_sg() routine").

You *fundamentally* cannot do that. Seriously. It's completely wrong.
Pick one or the other, or have the caller *tell* you what the context
is.

                Linus


^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-14 22:24       ` Linus Torvalds
  0 siblings, 0 replies; 282+ messages in thread
From: Linus Torvalds @ 2020-09-14 22:24 UTC (permalink / raw)
  To: Thomas Gleixner, Ard Biesheuvel, Herbert Xu
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, Linux-MM,
	open list:KERNEL SELFTEST FRAMEWORK, linux-hexagon, Will Deacon,
	Ingo Molnar, Anton Ivanov, linux-arch, Vincent Guittot,
	Brian Cain, Richard Weinberger, Russell King, David Airlie,
	Ingo Molnar, Geert Uytterhoeven, Mel Gorman, intel-gfx,
	Matt Turner, Valentin Schneider, linux-xtensa, Shuah Khan,
	Paul E. McKenney, Jeff Dike, linux-um, Josh Triplett,
	Steven Rostedt, rcu, linux-m68k, Ivan Kokshaysky, Rodrigo Vivi,
	Dietmar Eggemann, Linux ARM, Richard Henderson, Chris Zankel,
	Max Filippov, LKML, alpha, Mathieu Desnoyers, Andrew Morton,
	Daniel Bristot de Oliveira

On Mon, Sep 14, 2020 at 2:55 PM Thomas Gleixner <tglx@linutronix.de> wrote:
>
> Yes it does generate better code, but I tried hard to spot a difference
> in various metrics exposed by perf. It's all in the noise and I only
> can spot a difference when the actual preemption check after the
> decrement

I'm somewhat more worried about the small-device case.

That said, the diffstat certainly has its very clear charm, and I do
agree that it makes things simpler.

I'm just not convinced people should ever EVER do things like that "if
(preemptible())" garbage. It sounds like somebody is doing seriously
bad things.

The chacha20poly1305 code does look fundamentally broken. But no, the
fix is not to make "preemptible" work with spinlocks, the fix is to
not *do* that kind of broken things.

Those things would be broken even if you changed the semantics of
preemptible. There's no way that it's valid to say "use this debug
flag to decide if we should do atomic allocations or not".

It smells like "I got a debug failure, so I'm papering it over by
checking the thing the debug code checks for".

The debug check is to catch the obvious bad cases. It's not the _only_
bad cases, so copying the debug check test is just completely wrong.

Ard and Herbert added to participants: see
chacha20poly1305_crypt_sg_inplace(), which does

        flags = SG_MITER_TO_SG;
        if (!preemptible())
                flags |= SG_MITER_ATOMIC;

introduced in commit d95312a3ccc0 ("crypto: lib/chacha20poly1305 -
reimplement crypt_from_sg() routine").

You *fundamentally* cannot do that. Seriously. It's completely wrong.
Pick one or the other, or have the caller *tell* you what the context
is.

                Linus
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [Intel-gfx] [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-14 22:24       ` Linus Torvalds
  0 siblings, 0 replies; 282+ messages in thread
From: Linus Torvalds @ 2020-09-14 22:24 UTC (permalink / raw)
  To: Thomas Gleixner, Ard Biesheuvel, Herbert Xu
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, Linux-MM,
	open list:KERNEL SELFTEST FRAMEWORK, linux-hexagon, Will Deacon,
	Ingo Molnar, Anton Ivanov, linux-arch, Brian Cain,
	Richard Weinberger, Russell King, David Airlie, Ingo Molnar,
	Geert Uytterhoeven, Mel Gorman, intel-gfx, Matt Turner,
	Valentin Schneider, linux-xtensa, Shuah Khan, Paul E. McKenney,
	Jeff Dike, linux-um, Josh Triplett, Steven Rostedt, rcu,
	linux-m68k, Ivan Kokshaysky, Dietmar Eggemann, Linux ARM,
	Richard Henderson, Chris Zankel, Max Filippov, LKML, alpha,
	Mathieu Desnoyers, Andrew Morton, Daniel Bristot de Oliveira

On Mon, Sep 14, 2020 at 2:55 PM Thomas Gleixner <tglx@linutronix.de> wrote:
>
> Yes it does generate better code, but I tried hard to spot a difference
> in various metrics exposed by perf. It's all in the noise and I only
> can spot a difference when the actual preemption check after the
> decrement

I'm somewhat more worried about the small-device case.

That said, the diffstat certainly has its very clear charm, and I do
agree that it makes things simpler.

I'm just not convinced people should ever EVER do things like that "if
(preemptible())" garbage. It sounds like somebody is doing seriously
bad things.

The chacha20poly1305 code does look fundamentally broken. But no, the
fix is not to make "preemptible" work with spinlocks, the fix is to
not *do* that kind of broken things.

Those things would be broken even if you changed the semantics of
preemptible. There's no way that it's valid to say "use this debug
flag to decide if we should do atomic allocations or not".

It smells like "I got a debug failure, so I'm papering it over by
checking the thing the debug code checks for".

The debug check is to catch the obvious bad cases. It's not the _only_
bad cases, so copying the debug check test is just completely wrong.

Ard and Herbert added to participants: see
chacha20poly1305_crypt_sg_inplace(), which does

        flags = SG_MITER_TO_SG;
        if (!preemptible())
                flags |= SG_MITER_ATOMIC;

introduced in commit d95312a3ccc0 ("crypto: lib/chacha20poly1305 -
reimplement crypt_from_sg() routine").

You *fundamentally* cannot do that. Seriously. It's completely wrong.
Pick one or the other, or have the caller *tell* you what the context
is.

                Linus
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-14 22:24       ` Linus Torvalds
  0 siblings, 0 replies; 282+ messages in thread
From: Linus Torvalds @ 2020-09-14 22:24 UTC (permalink / raw)
  To: Thomas Gleixner, Ard Biesheuvel, Herbert Xu
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, Linux-MM,
	open list:KERNEL SELFTEST FRAMEWORK, linux-hexagon, Will Deacon,
	Ingo Molnar, Anton Ivanov, linux-arch, Brian Cain,
	Richard Weinberger, Russell King, David Airlie, Ingo Molnar,
	Geert Uytterhoeven, Mel Gorman, intel-gfx, Matt Turner,
	Valentin Schneider

On Mon, Sep 14, 2020 at 2:55 PM Thomas Gleixner <tglx@linutronix.de> wrote:
>
> Yes it does generate better code, but I tried hard to spot a difference
> in various metrics exposed by perf. It's all in the noise and I only
> can spot a difference when the actual preemption check after the
> decrement

I'm somewhat more worried about the small-device case.

That said, the diffstat certainly has its very clear charm, and I do
agree that it makes things simpler.

I'm just not convinced people should ever EVER do things like that "if
(preemptible())" garbage. It sounds like somebody is doing seriously
bad things.

The chacha20poly1305 code does look fundamentally broken. But no, the
fix is not to make "preemptible" work with spinlocks, the fix is to
not *do* that kind of broken things.

Those things would be broken even if you changed the semantics of
preemptible. There's no way that it's valid to say "use this debug
flag to decide if we should do atomic allocations or not".

It smells like "I got a debug failure, so I'm papering it over by
checking the thing the debug code checks for".

The debug check is to catch the obvious bad cases. It's not the _only_
bad cases, so copying the debug check test is just completely wrong.

Ard and Herbert added to participants: see
chacha20poly1305_crypt_sg_inplace(), which does

        flags = SG_MITER_TO_SG;
        if (!preemptible())
                flags |= SG_MITER_ATOMIC;

introduced in commit d95312a3ccc0 ("crypto: lib/chacha20poly1305 -
reimplement crypt_from_sg() routine").

You *fundamentally* cannot do that. Seriously. It's completely wrong.
Pick one or the other, or have the caller *tell* you what the context
is.

                Linus

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
  2020-09-14 22:24       ` Linus Torvalds
  (?)
  (?)
@ 2020-09-14 22:37         ` Linus Torvalds
  -1 siblings, 0 replies; 282+ messages in thread
From: Linus Torvalds @ 2020-09-14 22:37 UTC (permalink / raw)
  To: Thomas Gleixner, Ard Biesheuvel, Herbert Xu
  Cc: LKML, linux-arch, Sebastian Andrzej Siewior, Valentin Schneider,
	Richard Henderson, Ivan Kokshaysky, Matt Turner, alpha,
	Jeff Dike, Richard Weinberger, Anton Ivanov, linux-um,
	Brian Cain, linux-hexagon, Geert Uytterhoeven, linux-m68k,
	Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
	Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman,
	Daniel Bristot de Oliveira, Will Deacon, Andrew Morton, Linux-MM,
	Ingo Molnar, Russell King, Linux ARM, Chris Zankel, Max Filippov,
	linux-xtensa, Jani Nikula, Joonas Lahtinen, Rodrigo Vivi,
	David Airlie, Daniel Vetter, intel-gfx, dri-devel,
	Paul E. McKenney, Josh Triplett, Mathieu Desnoyers,
	Lai Jiangshan, Shuah Khan, rcu,
	open list:KERNEL SELFTEST FRAMEWORK

On Mon, Sep 14, 2020 at 3:24 PM Linus Torvalds
<torvalds@linux-foundation.org> wrote:
>
> Ard and Herbert added to participants: see
> chacha20poly1305_crypt_sg_inplace(), which does
>
>         flags = SG_MITER_TO_SG;
>         if (!preemptible())
>                 flags |= SG_MITER_ATOMIC;
>
> introduced in commit d95312a3ccc0 ("crypto: lib/chacha20poly1305 -
> reimplement crypt_from_sg() routine").

As far as I can tell, the only reason for this all is to try to use
"kmap()" rather than "kmap_atomic()".

And kmap() actually has the much more complex "might_sleep()" tests,
and apparently the "preemptible()" check wasn't even the proper full
debug check, it was just a complete hack to catch the one that
triggered.

From a quick look, that code should probably just get rid of
SG_MITER_ATOMIC entirely, and alwayse use kmap_atomic().

kmap_atomic() is actually the faster and proper interface to use
anyway (never mind that any of this matters on any sane hardware). The
old kmap() and kunmap() interfaces should generally be avoided like
the plague - yes, they allow sleeping in the middle and that is
sometimes required, but if you don't need that, you should never ever
use them.

We used to have a very nasty kmap_atomic() that required people to be
very careful and know exactly which atomic entry to use, and that was
admitedly quite nasty.

So it _looks_ like this code started using kmap() - probably back when
kmap_atomic() was so cumbersome to use - and was then converted
(conditionally) to kmap_atomic() rather than just changed whole-sale.
Is there actually something that wants to use those sg_miter functions
and sleep?

Because if there is, that choice should come from the outside, not
from inside lib/scatterlist.c trying to make some bad guess based on
the wrong thing entirely.

                 Linus


^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-14 22:37         ` Linus Torvalds
  0 siblings, 0 replies; 282+ messages in thread
From: Linus Torvalds @ 2020-09-14 22:37 UTC (permalink / raw)
  To: Thomas Gleixner, Ard Biesheuvel, Herbert Xu
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, Linux-MM,
	open list:KERNEL SELFTEST FRAMEWORK, linux-hexagon, Will Deacon,
	Ingo Molnar, Anton Ivanov, linux-arch, Vincent Guittot,
	Brian Cain, Richard Weinberger, Russell King, David Airlie,
	Ingo Molnar, Geert Uytterhoeven, Mel Gorman, intel-gfx,
	Matt Turner, Valentin Schneider, linux-xtensa, Shuah Khan,
	Paul E. McKenney, Jeff Dike, linux-um, Josh Triplett,
	Steven Rostedt, rcu, linux-m68k, Ivan Kokshaysky, Rodrigo Vivi,
	Dietmar Eggemann, Linux ARM, Richard Henderson, Chris Zankel,
	Max Filippov, LKML, alpha, Mathieu Desnoyers, Andrew Morton,
	Daniel Bristot de Oliveira

On Mon, Sep 14, 2020 at 3:24 PM Linus Torvalds
<torvalds@linux-foundation.org> wrote:
>
> Ard and Herbert added to participants: see
> chacha20poly1305_crypt_sg_inplace(), which does
>
>         flags = SG_MITER_TO_SG;
>         if (!preemptible())
>                 flags |= SG_MITER_ATOMIC;
>
> introduced in commit d95312a3ccc0 ("crypto: lib/chacha20poly1305 -
> reimplement crypt_from_sg() routine").

As far as I can tell, the only reason for this all is to try to use
"kmap()" rather than "kmap_atomic()".

And kmap() actually has the much more complex "might_sleep()" tests,
and apparently the "preemptible()" check wasn't even the proper full
debug check, it was just a complete hack to catch the one that
triggered.

From a quick look, that code should probably just get rid of
SG_MITER_ATOMIC entirely, and alwayse use kmap_atomic().

kmap_atomic() is actually the faster and proper interface to use
anyway (never mind that any of this matters on any sane hardware). The
old kmap() and kunmap() interfaces should generally be avoided like
the plague - yes, they allow sleeping in the middle and that is
sometimes required, but if you don't need that, you should never ever
use them.

We used to have a very nasty kmap_atomic() that required people to be
very careful and know exactly which atomic entry to use, and that was
admitedly quite nasty.

So it _looks_ like this code started using kmap() - probably back when
kmap_atomic() was so cumbersome to use - and was then converted
(conditionally) to kmap_atomic() rather than just changed whole-sale.
Is there actually something that wants to use those sg_miter functions
and sleep?

Because if there is, that choice should come from the outside, not
from inside lib/scatterlist.c trying to make some bad guess based on
the wrong thing entirely.

                 Linus
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [Intel-gfx] [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-14 22:37         ` Linus Torvalds
  0 siblings, 0 replies; 282+ messages in thread
From: Linus Torvalds @ 2020-09-14 22:37 UTC (permalink / raw)
  To: Thomas Gleixner, Ard Biesheuvel, Herbert Xu
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, Linux-MM,
	open list:KERNEL SELFTEST FRAMEWORK, linux-hexagon, Will Deacon,
	Ingo Molnar, Anton Ivanov, linux-arch, Brian Cain,
	Richard Weinberger, Russell King, David Airlie, Ingo Molnar,
	Geert Uytterhoeven, Mel Gorman, intel-gfx, Matt Turner,
	Valentin Schneider, linux-xtensa, Shuah Khan, Paul E. McKenney,
	Jeff Dike, linux-um, Josh Triplett, Steven Rostedt, rcu,
	linux-m68k, Ivan Kokshaysky, Dietmar Eggemann, Linux ARM,
	Richard Henderson, Chris Zankel, Max Filippov, LKML, alpha,
	Mathieu Desnoyers, Andrew Morton, Daniel Bristot de Oliveira

On Mon, Sep 14, 2020 at 3:24 PM Linus Torvalds
<torvalds@linux-foundation.org> wrote:
>
> Ard and Herbert added to participants: see
> chacha20poly1305_crypt_sg_inplace(), which does
>
>         flags = SG_MITER_TO_SG;
>         if (!preemptible())
>                 flags |= SG_MITER_ATOMIC;
>
> introduced in commit d95312a3ccc0 ("crypto: lib/chacha20poly1305 -
> reimplement crypt_from_sg() routine").

As far as I can tell, the only reason for this all is to try to use
"kmap()" rather than "kmap_atomic()".

And kmap() actually has the much more complex "might_sleep()" tests,
and apparently the "preemptible()" check wasn't even the proper full
debug check, it was just a complete hack to catch the one that
triggered.

From a quick look, that code should probably just get rid of
SG_MITER_ATOMIC entirely, and alwayse use kmap_atomic().

kmap_atomic() is actually the faster and proper interface to use
anyway (never mind that any of this matters on any sane hardware). The
old kmap() and kunmap() interfaces should generally be avoided like
the plague - yes, they allow sleeping in the middle and that is
sometimes required, but if you don't need that, you should never ever
use them.

We used to have a very nasty kmap_atomic() that required people to be
very careful and know exactly which atomic entry to use, and that was
admitedly quite nasty.

So it _looks_ like this code started using kmap() - probably back when
kmap_atomic() was so cumbersome to use - and was then converted
(conditionally) to kmap_atomic() rather than just changed whole-sale.
Is there actually something that wants to use those sg_miter functions
and sleep?

Because if there is, that choice should come from the outside, not
from inside lib/scatterlist.c trying to make some bad guess based on
the wrong thing entirely.

                 Linus
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-14 22:37         ` Linus Torvalds
  0 siblings, 0 replies; 282+ messages in thread
From: Linus Torvalds @ 2020-09-14 22:37 UTC (permalink / raw)
  To: Thomas Gleixner, Ard Biesheuvel, Herbert Xu
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, Linux-MM,
	open list:KERNEL SELFTEST FRAMEWORK, linux-hexagon, Will Deacon,
	Ingo Molnar, Anton Ivanov, linux-arch, Brian Cain,
	Richard Weinberger, Russell King, David Airlie, Ingo Molnar,
	Geert Uytterhoeven, Mel Gorman, intel-gfx, Matt Turner,
	Valentin Schneider

On Mon, Sep 14, 2020 at 3:24 PM Linus Torvalds
<torvalds@linux-foundation.org> wrote:
>
> Ard and Herbert added to participants: see
> chacha20poly1305_crypt_sg_inplace(), which does
>
>         flags = SG_MITER_TO_SG;
>         if (!preemptible())
>                 flags |= SG_MITER_ATOMIC;
>
> introduced in commit d95312a3ccc0 ("crypto: lib/chacha20poly1305 -
> reimplement crypt_from_sg() routine").

As far as I can tell, the only reason for this all is to try to use
"kmap()" rather than "kmap_atomic()".

And kmap() actually has the much more complex "might_sleep()" tests,
and apparently the "preemptible()" check wasn't even the proper full
debug check, it was just a complete hack to catch the one that
triggered.

From a quick look, that code should probably just get rid of
SG_MITER_ATOMIC entirely, and alwayse use kmap_atomic().

kmap_atomic() is actually the faster and proper interface to use
anyway (never mind that any of this matters on any sane hardware). The
old kmap() and kunmap() interfaces should generally be avoided like
the plague - yes, they allow sleeping in the middle and that is
sometimes required, but if you don't need that, you should never ever
use them.

We used to have a very nasty kmap_atomic() that required people to be
very careful and know exactly which atomic entry to use, and that was
admitedly quite nasty.

So it _looks_ like this code started using kmap() - probably back when
kmap_atomic() was so cumbersome to use - and was then converted
(conditionally) to kmap_atomic() rather than just changed whole-sale.
Is there actually something that wants to use those sg_miter functions
and sleep?

Because if there is, that choice should come from the outside, not
from inside lib/scatterlist.c trying to make some bad guess based on
the wrong thing entirely.

                 Linus

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 05/13] mm/pagemap: Clenaup PREEMPT_COUNT leftovers
  2020-09-14 20:42   ` Thomas Gleixner
@ 2020-09-15  0:52     ` kernel test robot
  -1 siblings, 0 replies; 282+ messages in thread
From: kernel test robot @ 2020-09-15  0:52 UTC (permalink / raw)
  To: Thomas Gleixner, LKML
  Cc: kbuild-all, clang-built-linux, linux-arch,
	Sebastian Andrzej Siewior, Valentin Schneider, Andrew Morton,
	Linux Memory Management List, Richard Henderson, Ivan Kokshaysky,
	Matt Turner, linux-alpha

[-- Attachment #1: Type: text/plain, Size: 18744 bytes --]

Hi Thomas,

I love your patch! Yet something to improve:

[auto build test ERROR on drm-intel/for-linux-next]
[also build test ERROR on linus/master v5.9-rc5 next-20200914]
[cannot apply to rcu/dev arm/for-next tip/sched/core linux/master]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]

url:    https://github.com/0day-ci/linux/commits/Thomas-Gleixner/preempt-Make-preempt-count-unconditional/20200915-044640
base:   git://anongit.freedesktop.org/drm-intel for-linux-next
config: arm-randconfig-r002-20200914 (attached as .config)
compiler: clang version 12.0.0 (https://github.com/llvm/llvm-project b2c32c90bab09a6e2c1f370429db26017a182143)
reproduce (this is a W=1 build):
        wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
        chmod +x ~/bin/make.cross
        # install arm cross compiling tool for clang build
        # apt-get install binutils-arm-linux-gnueabi
        # save the attached .config to linux build tree
        COMPILER_INSTALL_PATH=$HOME/0day COMPILER=clang make.cross ARCH=arm 

If you fix the issue, kindly add following tag as appropriate
Reported-by: kernel test robot <lkp@intel.com>

All error/warnings (new ones prefixed by >>):

   In file included from drivers/scsi/scsi.c:47:
   In file included from include/linux/blkdev.h:13:
>> include/linux/pagemap.h:181:2: error: called object type 'void' is not a function or function pointer
           VM_BUG_ON_PAGE(page_count(page) == 0, page);
           ^
   include/linux/mmdebug.h:46:36: note: expanded from macro 'VM_BUG_ON_PAGE'
   #define VM_BUG_ON_PAGE(cond, page) VM_BUG_ON(cond)
                                      ^
   include/linux/mmdebug.h:45:25: note: expanded from macro 'VM_BUG_ON'
   #define VM_BUG_ON(cond) BUILD_BUG_ON_INVALID(cond)
                           ^
   include/linux/build_bug.h:30:33: note: expanded from macro 'BUILD_BUG_ON_INVALID'
   #define BUILD_BUG_ON_INVALID(e) ((void)(sizeof((__force long)(e))))
                                   ^
   In file included from drivers/scsi/scsi.c:62:
   In file included from include/scsi/scsi_cmnd.h:5:
   include/linux/dma-mapping.h:632:9: warning: implicit conversion from 'unsigned long long' to 'unsigned long' changes value from 18446744073709551615 to 4294967295 [-Wconstant-conversion]
           return DMA_BIT_MASK(32);
           ~~~~~~ ^~~~~~~~~~~~~~~~
   include/linux/dma-mapping.h:139:40: note: expanded from macro 'DMA_BIT_MASK'
   #define DMA_BIT_MASK(n) (((n) == 64) ? ~0ULL : ((1ULL<<(n))-1))
                                          ^~~~~
   1 warning and 1 error generated.
--
   In file included from drivers/scsi/scsicam.c:19:
   In file included from include/linux/blkdev.h:13:
>> include/linux/pagemap.h:181:2: error: called object type 'void' is not a function or function pointer
           VM_BUG_ON_PAGE(page_count(page) == 0, page);
           ^
   include/linux/mmdebug.h:46:36: note: expanded from macro 'VM_BUG_ON_PAGE'
   #define VM_BUG_ON_PAGE(cond, page) VM_BUG_ON(cond)
                                      ^
   include/linux/mmdebug.h:45:25: note: expanded from macro 'VM_BUG_ON'
   #define VM_BUG_ON(cond) BUILD_BUG_ON_INVALID(cond)
                           ^
   include/linux/build_bug.h:30:33: note: expanded from macro 'BUILD_BUG_ON_INVALID'
   #define BUILD_BUG_ON_INVALID(e) ((void)(sizeof((__force long)(e))))
                                   ^
   1 error generated.
--
   In file included from drivers/scsi/scsi_sysfs.c:13:
   In file included from include/linux/blkdev.h:13:
>> include/linux/pagemap.h:181:2: error: called object type 'void' is not a function or function pointer
           VM_BUG_ON_PAGE(page_count(page) == 0, page);
           ^
   include/linux/mmdebug.h:46:36: note: expanded from macro 'VM_BUG_ON_PAGE'
   #define VM_BUG_ON_PAGE(cond, page) VM_BUG_ON(cond)
                                      ^
   include/linux/mmdebug.h:45:25: note: expanded from macro 'VM_BUG_ON'
   #define VM_BUG_ON(cond) BUILD_BUG_ON_INVALID(cond)
                           ^
   include/linux/build_bug.h:30:33: note: expanded from macro 'BUILD_BUG_ON_INVALID'
   #define BUILD_BUG_ON_INVALID(e) ((void)(sizeof((__force long)(e))))
                                   ^
   In file included from drivers/scsi/scsi_sysfs.c:20:
   In file included from include/scsi/scsi_tcq.h:6:
   In file included from include/scsi/scsi_cmnd.h:5:
   include/linux/dma-mapping.h:632:9: warning: implicit conversion from 'unsigned long long' to 'unsigned long' changes value from 18446744073709551615 to 4294967295 [-Wconstant-conversion]
           return DMA_BIT_MASK(32);
           ~~~~~~ ^~~~~~~~~~~~~~~~
   include/linux/dma-mapping.h:139:40: note: expanded from macro 'DMA_BIT_MASK'
   #define DMA_BIT_MASK(n) (((n) == 64) ? ~0ULL : ((1ULL<<(n))-1))
                                          ^~~~~
   drivers/scsi/scsi_sysfs.c:373:1: warning: format specifies type 'short' but the argument has type 'int' [-Wformat]
   shost_rd_attr(can_queue, "%hd\n");
   ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
                             %d
   drivers/scsi/scsi_sysfs.c:176:45: note: expanded from macro 'shost_rd_attr'
   #define shost_rd_attr(field, format_string) \
                                               ^
   drivers/scsi/scsi_sysfs.c:173:2: note: expanded from macro '\
   shost_rd_attr2'
           shost_show_function(name, field, format_string)                 \
           ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
   drivers/scsi/scsi_sysfs.c:165:43: note: expanded from macro 'shost_show_function'
           return snprintf (buf, 20, format_string, shost->field);         \
                                     ~~~~~~~~~~~~~  ^~~~~~~~~~~~
   2 warnings and 1 error generated.
--
   In file included from drivers/scsi/ufs/ufshcd.c:18:
   In file included from drivers/scsi/ufs/ufshcd.h:32:
   In file included from include/linux/regulator/consumer.h:35:
   In file included from include/linux/suspend.h:5:
   In file included from include/linux/swap.h:9:
   In file included from include/linux/memcontrol.h:22:
   In file included from include/linux/writeback.h:14:
   In file included from include/linux/blk-cgroup.h:23:
   In file included from include/linux/blkdev.h:13:
>> include/linux/pagemap.h:181:2: error: called object type 'void' is not a function or function pointer
           VM_BUG_ON_PAGE(page_count(page) == 0, page);
           ^
   include/linux/mmdebug.h:46:36: note: expanded from macro 'VM_BUG_ON_PAGE'
   #define VM_BUG_ON_PAGE(cond, page) VM_BUG_ON(cond)
                                      ^
   include/linux/mmdebug.h:45:25: note: expanded from macro 'VM_BUG_ON'
   #define VM_BUG_ON(cond) BUILD_BUG_ON_INVALID(cond)
                           ^
   include/linux/build_bug.h:30:33: note: expanded from macro 'BUILD_BUG_ON_INVALID'
   #define BUILD_BUG_ON_INVALID(e) ((void)(sizeof((__force long)(e))))
                                   ^
   In file included from drivers/scsi/ufs/ufshcd.c:18:
   In file included from drivers/scsi/ufs/ufshcd.h:41:
   In file included from include/scsi/scsi_cmnd.h:5:
   include/linux/dma-mapping.h:632:9: warning: implicit conversion from 'unsigned long long' to 'unsigned long' changes value from 18446744073709551615 to 4294967295 [-Wconstant-conversion]
           return DMA_BIT_MASK(32);
           ~~~~~~ ^~~~~~~~~~~~~~~~
   include/linux/dma-mapping.h:139:40: note: expanded from macro 'DMA_BIT_MASK'
   #define DMA_BIT_MASK(n) (((n) == 64) ? ~0ULL : ((1ULL<<(n))-1))
                                          ^~~~~
>> drivers/scsi/ufs/ufshcd.c:8716:44: warning: shift count >= width of type [-Wshift-count-overflow]
                   if (!dma_set_mask_and_coherent(hba->dev, DMA_BIT_MASK(64)))
                                                            ^~~~~~~~~~~~~~~~
   include/linux/dma-mapping.h:139:54: note: expanded from macro 'DMA_BIT_MASK'
   #define DMA_BIT_MASK(n) (((n) == 64) ? ~0ULL : ((1ULL<<(n))-1))
                                                        ^ ~~~
   2 warnings and 1 error generated.
--
   In file included from drivers/scsi/hisi_sas/hisi_sas_main.c:7:
   In file included from drivers/scsi/hisi_sas/hisi_sas.h:16:
   In file included from include/linux/libata.h:21:
   In file included from include/scsi/scsi_host.h:11:
   In file included from include/linux/blk-mq.h:5:
   In file included from include/linux/blkdev.h:13:
>> include/linux/pagemap.h:181:2: error: called object type 'void' is not a function or function pointer
           VM_BUG_ON_PAGE(page_count(page) == 0, page);
           ^
   include/linux/mmdebug.h:46:36: note: expanded from macro 'VM_BUG_ON_PAGE'
   #define VM_BUG_ON_PAGE(cond, page) VM_BUG_ON(cond)
                                      ^
   include/linux/mmdebug.h:45:25: note: expanded from macro 'VM_BUG_ON'
   #define VM_BUG_ON(cond) BUILD_BUG_ON_INVALID(cond)
                           ^
   include/linux/build_bug.h:30:33: note: expanded from macro 'BUILD_BUG_ON_INVALID'
   #define BUILD_BUG_ON_INVALID(e) ((void)(sizeof((__force long)(e))))
                                   ^
>> drivers/scsi/hisi_sas/hisi_sas_main.c:2579:41: warning: shift count >= width of type [-Wshift-count-overflow]
           error = dma_set_mask_and_coherent(dev, DMA_BIT_MASK(64));
                                                  ^~~~~~~~~~~~~~~~
   include/linux/dma-mapping.h:139:54: note: expanded from macro 'DMA_BIT_MASK'
   #define DMA_BIT_MASK(n) (((n) == 64) ? ~0ULL : ((1ULL<<(n))-1))
                                                        ^ ~~~
   1 warning and 1 error generated.
--
   In file included from drivers/scsi/scsi_sysfs.c:13:
   In file included from include/linux/blkdev.h:13:
>> include/linux/pagemap.h:181:2: error: called object type 'void' is not a function or function pointer
           VM_BUG_ON_PAGE(page_count(page) == 0, page);
           ^
   include/linux/mmdebug.h:46:36: note: expanded from macro 'VM_BUG_ON_PAGE'
   #define VM_BUG_ON_PAGE(cond, page) VM_BUG_ON(cond)
                                      ^
   include/linux/mmdebug.h:45:25: note: expanded from macro 'VM_BUG_ON'
   #define VM_BUG_ON(cond) BUILD_BUG_ON_INVALID(cond)
                           ^
   include/linux/build_bug.h:30:33: note: expanded from macro 'BUILD_BUG_ON_INVALID'
   #define BUILD_BUG_ON_INVALID(e) ((void)(sizeof((__force long)(e))))
                                   ^
   In file included from drivers/scsi/scsi_sysfs.c:20:
   In file included from include/scsi/scsi_tcq.h:6:
   In file included from include/scsi/scsi_cmnd.h:5:
   include/linux/dma-mapping.h:632:9: warning: implicit conversion from 'unsigned long long' to 'unsigned long' changes value from 18446744073709551615 to 4294967295 [-Wconstant-conversion]
           return DMA_BIT_MASK(32);
           ~~~~~~ ^~~~~~~~~~~~~~~~
   include/linux/dma-mapping.h:139:40: note: expanded from macro 'DMA_BIT_MASK'
   #define DMA_BIT_MASK(n) (((n) == 64) ? ~0ULL : ((1ULL<<(n))-1))
                                          ^~~~~
   drivers/scsi/scsi_sysfs.c:373:1: warning: format specifies type 'short' but the argument has type 'int' [-Wformat]
   shost_rd_attr(can_queue, "%hd\n");
   ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
                             %d
   drivers/scsi/scsi_sysfs.c:176:45: note: expanded from macro 'shost_rd_attr'
   #define shost_rd_attr(field, format_string) \
                                               ^
   drivers/scsi/scsi_sysfs.c:173:2: note: expanded from macro '\
   shost_rd_attr2'
           shost_show_function(name, field, format_string)                 \
           ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
   drivers/scsi/scsi_sysfs.c:165:43: note: expanded from macro 'shost_show_function'
           return snprintf (buf, 20, format_string, shost->field);         \
                                     ~~~~~~~~~~~~~  ^~~~~~~~~~~~
   drivers/scsi/scsi_sysfs.c:1027:10: fatal error: 'scsi_devinfo_tbl.c' file not found
   #include "scsi_devinfo_tbl.c"
            ^~~~~~~~~~~~~~~~~~~~
   2 warnings and 2 errors generated.

# https://github.com/0day-ci/linux/commit/a4a0f54fdd08d95dfe20d684b405db8a47fb61d8
git remote add linux-review https://github.com/0day-ci/linux
git fetch --no-tags linux-review Thomas-Gleixner/preempt-Make-preempt-count-unconditional/20200915-044640
git checkout a4a0f54fdd08d95dfe20d684b405db8a47fb61d8
vim +/void +181 include/linux/pagemap.h

^1da177e4c3f41 Linus Torvalds         2005-04-16  123  
e286781d5f2e9c Nick Piggin            2008-07-25  124  /*
e286781d5f2e9c Nick Piggin            2008-07-25  125   * speculatively take a reference to a page.
0139aa7b7fa12c Joonsoo Kim            2016-05-19  126   * If the page is free (_refcount == 0), then _refcount is untouched, and 0
0139aa7b7fa12c Joonsoo Kim            2016-05-19  127   * is returned. Otherwise, _refcount is incremented by 1 and 1 is returned.
e286781d5f2e9c Nick Piggin            2008-07-25  128   *
e286781d5f2e9c Nick Piggin            2008-07-25  129   * This function must be called inside the same rcu_read_lock() section as has
e286781d5f2e9c Nick Piggin            2008-07-25  130   * been used to lookup the page in the pagecache radix-tree (or page table):
0139aa7b7fa12c Joonsoo Kim            2016-05-19  131   * this allows allocators to use a synchronize_rcu() to stabilize _refcount.
e286781d5f2e9c Nick Piggin            2008-07-25  132   *
e286781d5f2e9c Nick Piggin            2008-07-25  133   * Unless an RCU grace period has passed, the count of all pages coming out
e286781d5f2e9c Nick Piggin            2008-07-25  134   * of the allocator must be considered unstable. page_count may return higher
e286781d5f2e9c Nick Piggin            2008-07-25  135   * than expected, and put_page must be able to do the right thing when the
e286781d5f2e9c Nick Piggin            2008-07-25  136   * page has been finished with, no matter what it is subsequently allocated
e286781d5f2e9c Nick Piggin            2008-07-25  137   * for (because put_page is what is used here to drop an invalid speculative
e286781d5f2e9c Nick Piggin            2008-07-25  138   * reference).
e286781d5f2e9c Nick Piggin            2008-07-25  139   *
e286781d5f2e9c Nick Piggin            2008-07-25  140   * This is the interesting part of the lockless pagecache (and lockless
e286781d5f2e9c Nick Piggin            2008-07-25  141   * get_user_pages) locking protocol, where the lookup-side (eg. find_get_page)
e286781d5f2e9c Nick Piggin            2008-07-25  142   * has the following pattern:
e286781d5f2e9c Nick Piggin            2008-07-25  143   * 1. find page in radix tree
e286781d5f2e9c Nick Piggin            2008-07-25  144   * 2. conditionally increment refcount
e286781d5f2e9c Nick Piggin            2008-07-25  145   * 3. check the page is still in pagecache (if no, goto 1)
e286781d5f2e9c Nick Piggin            2008-07-25  146   *
0139aa7b7fa12c Joonsoo Kim            2016-05-19  147   * Remove-side that cares about stability of _refcount (eg. reclaim) has the
b93b016313b3ba Matthew Wilcox         2018-04-10  148   * following (with the i_pages lock held):
e286781d5f2e9c Nick Piggin            2008-07-25  149   * A. atomically check refcount is correct and set it to 0 (atomic_cmpxchg)
e286781d5f2e9c Nick Piggin            2008-07-25  150   * B. remove page from pagecache
e286781d5f2e9c Nick Piggin            2008-07-25  151   * C. free the page
e286781d5f2e9c Nick Piggin            2008-07-25  152   *
e286781d5f2e9c Nick Piggin            2008-07-25  153   * There are 2 critical interleavings that matter:
e286781d5f2e9c Nick Piggin            2008-07-25  154   * - 2 runs before A: in this case, A sees elevated refcount and bails out
e286781d5f2e9c Nick Piggin            2008-07-25  155   * - A runs before 2: in this case, 2 sees zero refcount and retries;
e286781d5f2e9c Nick Piggin            2008-07-25  156   *   subsequently, B will complete and 1 will find no page, causing the
e286781d5f2e9c Nick Piggin            2008-07-25  157   *   lookup to return NULL.
e286781d5f2e9c Nick Piggin            2008-07-25  158   *
e286781d5f2e9c Nick Piggin            2008-07-25  159   * It is possible that between 1 and 2, the page is removed then the exact same
e286781d5f2e9c Nick Piggin            2008-07-25  160   * page is inserted into the same position in pagecache. That's OK: the
b93b016313b3ba Matthew Wilcox         2018-04-10  161   * old find_get_page using a lock could equally have run before or after
e286781d5f2e9c Nick Piggin            2008-07-25  162   * such a re-insertion, depending on order that locks are granted.
e286781d5f2e9c Nick Piggin            2008-07-25  163   *
e286781d5f2e9c Nick Piggin            2008-07-25  164   * Lookups racing against pagecache insertion isn't a big problem: either 1
e286781d5f2e9c Nick Piggin            2008-07-25  165   * will find the page or it will not. Likewise, the old find_get_page could run
e286781d5f2e9c Nick Piggin            2008-07-25  166   * either before the insertion or afterwards, depending on timing.
e286781d5f2e9c Nick Piggin            2008-07-25  167   */
494eec70f05496 john.hubbard@gmail.com 2019-03-05  168  static inline int __page_cache_add_speculative(struct page *page, int count)
e286781d5f2e9c Nick Piggin            2008-07-25  169  {
8375ad98cc1def Paul E. McKenney       2013-04-29  170  #ifdef CONFIG_TINY_RCU
a4a0f54fdd08d9 Thomas Gleixner        2020-09-14  171  	VM_BUG_ON(preemptible())
e286781d5f2e9c Nick Piggin            2008-07-25  172  	/*
e286781d5f2e9c Nick Piggin            2008-07-25  173  	 * Preempt must be disabled here - we rely on rcu_read_lock doing
e286781d5f2e9c Nick Piggin            2008-07-25  174  	 * this for us.
e286781d5f2e9c Nick Piggin            2008-07-25  175  	 *
e286781d5f2e9c Nick Piggin            2008-07-25  176  	 * Pagecache won't be truncated from interrupt context, so if we have
e286781d5f2e9c Nick Piggin            2008-07-25  177  	 * found a page in the radix tree here, we have pinned its refcount by
e286781d5f2e9c Nick Piggin            2008-07-25  178  	 * disabling preempt, and hence no need for the "speculative get" that
e286781d5f2e9c Nick Piggin            2008-07-25  179  	 * SMP requires.
e286781d5f2e9c Nick Piggin            2008-07-25  180  	 */
309381feaee564 Sasha Levin            2014-01-23 @181  	VM_BUG_ON_PAGE(page_count(page) == 0, page);
494eec70f05496 john.hubbard@gmail.com 2019-03-05  182  	page_ref_add(page, count);
e286781d5f2e9c Nick Piggin            2008-07-25  183  

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all@lists.01.org

[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 26917 bytes --]

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 05/13] mm/pagemap: Clenaup PREEMPT_COUNT leftovers
@ 2020-09-15  0:52     ` kernel test robot
  0 siblings, 0 replies; 282+ messages in thread
From: kernel test robot @ 2020-09-15  0:52 UTC (permalink / raw)
  To: kbuild-all

[-- Attachment #1: Type: text/plain, Size: 19034 bytes --]

Hi Thomas,

I love your patch! Yet something to improve:

[auto build test ERROR on drm-intel/for-linux-next]
[also build test ERROR on linus/master v5.9-rc5 next-20200914]
[cannot apply to rcu/dev arm/for-next tip/sched/core linux/master]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]

url:    https://github.com/0day-ci/linux/commits/Thomas-Gleixner/preempt-Make-preempt-count-unconditional/20200915-044640
base:   git://anongit.freedesktop.org/drm-intel for-linux-next
config: arm-randconfig-r002-20200914 (attached as .config)
compiler: clang version 12.0.0 (https://github.com/llvm/llvm-project b2c32c90bab09a6e2c1f370429db26017a182143)
reproduce (this is a W=1 build):
        wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
        chmod +x ~/bin/make.cross
        # install arm cross compiling tool for clang build
        # apt-get install binutils-arm-linux-gnueabi
        # save the attached .config to linux build tree
        COMPILER_INSTALL_PATH=$HOME/0day COMPILER=clang make.cross ARCH=arm 

If you fix the issue, kindly add following tag as appropriate
Reported-by: kernel test robot <lkp@intel.com>

All error/warnings (new ones prefixed by >>):

   In file included from drivers/scsi/scsi.c:47:
   In file included from include/linux/blkdev.h:13:
>> include/linux/pagemap.h:181:2: error: called object type 'void' is not a function or function pointer
           VM_BUG_ON_PAGE(page_count(page) == 0, page);
           ^
   include/linux/mmdebug.h:46:36: note: expanded from macro 'VM_BUG_ON_PAGE'
   #define VM_BUG_ON_PAGE(cond, page) VM_BUG_ON(cond)
                                      ^
   include/linux/mmdebug.h:45:25: note: expanded from macro 'VM_BUG_ON'
   #define VM_BUG_ON(cond) BUILD_BUG_ON_INVALID(cond)
                           ^
   include/linux/build_bug.h:30:33: note: expanded from macro 'BUILD_BUG_ON_INVALID'
   #define BUILD_BUG_ON_INVALID(e) ((void)(sizeof((__force long)(e))))
                                   ^
   In file included from drivers/scsi/scsi.c:62:
   In file included from include/scsi/scsi_cmnd.h:5:
   include/linux/dma-mapping.h:632:9: warning: implicit conversion from 'unsigned long long' to 'unsigned long' changes value from 18446744073709551615 to 4294967295 [-Wconstant-conversion]
           return DMA_BIT_MASK(32);
           ~~~~~~ ^~~~~~~~~~~~~~~~
   include/linux/dma-mapping.h:139:40: note: expanded from macro 'DMA_BIT_MASK'
   #define DMA_BIT_MASK(n) (((n) == 64) ? ~0ULL : ((1ULL<<(n))-1))
                                          ^~~~~
   1 warning and 1 error generated.
--
   In file included from drivers/scsi/scsicam.c:19:
   In file included from include/linux/blkdev.h:13:
>> include/linux/pagemap.h:181:2: error: called object type 'void' is not a function or function pointer
           VM_BUG_ON_PAGE(page_count(page) == 0, page);
           ^
   include/linux/mmdebug.h:46:36: note: expanded from macro 'VM_BUG_ON_PAGE'
   #define VM_BUG_ON_PAGE(cond, page) VM_BUG_ON(cond)
                                      ^
   include/linux/mmdebug.h:45:25: note: expanded from macro 'VM_BUG_ON'
   #define VM_BUG_ON(cond) BUILD_BUG_ON_INVALID(cond)
                           ^
   include/linux/build_bug.h:30:33: note: expanded from macro 'BUILD_BUG_ON_INVALID'
   #define BUILD_BUG_ON_INVALID(e) ((void)(sizeof((__force long)(e))))
                                   ^
   1 error generated.
--
   In file included from drivers/scsi/scsi_sysfs.c:13:
   In file included from include/linux/blkdev.h:13:
>> include/linux/pagemap.h:181:2: error: called object type 'void' is not a function or function pointer
           VM_BUG_ON_PAGE(page_count(page) == 0, page);
           ^
   include/linux/mmdebug.h:46:36: note: expanded from macro 'VM_BUG_ON_PAGE'
   #define VM_BUG_ON_PAGE(cond, page) VM_BUG_ON(cond)
                                      ^
   include/linux/mmdebug.h:45:25: note: expanded from macro 'VM_BUG_ON'
   #define VM_BUG_ON(cond) BUILD_BUG_ON_INVALID(cond)
                           ^
   include/linux/build_bug.h:30:33: note: expanded from macro 'BUILD_BUG_ON_INVALID'
   #define BUILD_BUG_ON_INVALID(e) ((void)(sizeof((__force long)(e))))
                                   ^
   In file included from drivers/scsi/scsi_sysfs.c:20:
   In file included from include/scsi/scsi_tcq.h:6:
   In file included from include/scsi/scsi_cmnd.h:5:
   include/linux/dma-mapping.h:632:9: warning: implicit conversion from 'unsigned long long' to 'unsigned long' changes value from 18446744073709551615 to 4294967295 [-Wconstant-conversion]
           return DMA_BIT_MASK(32);
           ~~~~~~ ^~~~~~~~~~~~~~~~
   include/linux/dma-mapping.h:139:40: note: expanded from macro 'DMA_BIT_MASK'
   #define DMA_BIT_MASK(n) (((n) == 64) ? ~0ULL : ((1ULL<<(n))-1))
                                          ^~~~~
   drivers/scsi/scsi_sysfs.c:373:1: warning: format specifies type 'short' but the argument has type 'int' [-Wformat]
   shost_rd_attr(can_queue, "%hd\n");
   ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
                             %d
   drivers/scsi/scsi_sysfs.c:176:45: note: expanded from macro 'shost_rd_attr'
   #define shost_rd_attr(field, format_string) \
                                               ^
   drivers/scsi/scsi_sysfs.c:173:2: note: expanded from macro '\
   shost_rd_attr2'
           shost_show_function(name, field, format_string)                 \
           ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
   drivers/scsi/scsi_sysfs.c:165:43: note: expanded from macro 'shost_show_function'
           return snprintf (buf, 20, format_string, shost->field);         \
                                     ~~~~~~~~~~~~~  ^~~~~~~~~~~~
   2 warnings and 1 error generated.
--
   In file included from drivers/scsi/ufs/ufshcd.c:18:
   In file included from drivers/scsi/ufs/ufshcd.h:32:
   In file included from include/linux/regulator/consumer.h:35:
   In file included from include/linux/suspend.h:5:
   In file included from include/linux/swap.h:9:
   In file included from include/linux/memcontrol.h:22:
   In file included from include/linux/writeback.h:14:
   In file included from include/linux/blk-cgroup.h:23:
   In file included from include/linux/blkdev.h:13:
>> include/linux/pagemap.h:181:2: error: called object type 'void' is not a function or function pointer
           VM_BUG_ON_PAGE(page_count(page) == 0, page);
           ^
   include/linux/mmdebug.h:46:36: note: expanded from macro 'VM_BUG_ON_PAGE'
   #define VM_BUG_ON_PAGE(cond, page) VM_BUG_ON(cond)
                                      ^
   include/linux/mmdebug.h:45:25: note: expanded from macro 'VM_BUG_ON'
   #define VM_BUG_ON(cond) BUILD_BUG_ON_INVALID(cond)
                           ^
   include/linux/build_bug.h:30:33: note: expanded from macro 'BUILD_BUG_ON_INVALID'
   #define BUILD_BUG_ON_INVALID(e) ((void)(sizeof((__force long)(e))))
                                   ^
   In file included from drivers/scsi/ufs/ufshcd.c:18:
   In file included from drivers/scsi/ufs/ufshcd.h:41:
   In file included from include/scsi/scsi_cmnd.h:5:
   include/linux/dma-mapping.h:632:9: warning: implicit conversion from 'unsigned long long' to 'unsigned long' changes value from 18446744073709551615 to 4294967295 [-Wconstant-conversion]
           return DMA_BIT_MASK(32);
           ~~~~~~ ^~~~~~~~~~~~~~~~
   include/linux/dma-mapping.h:139:40: note: expanded from macro 'DMA_BIT_MASK'
   #define DMA_BIT_MASK(n) (((n) == 64) ? ~0ULL : ((1ULL<<(n))-1))
                                          ^~~~~
>> drivers/scsi/ufs/ufshcd.c:8716:44: warning: shift count >= width of type [-Wshift-count-overflow]
                   if (!dma_set_mask_and_coherent(hba->dev, DMA_BIT_MASK(64)))
                                                            ^~~~~~~~~~~~~~~~
   include/linux/dma-mapping.h:139:54: note: expanded from macro 'DMA_BIT_MASK'
   #define DMA_BIT_MASK(n) (((n) == 64) ? ~0ULL : ((1ULL<<(n))-1))
                                                        ^ ~~~
   2 warnings and 1 error generated.
--
   In file included from drivers/scsi/hisi_sas/hisi_sas_main.c:7:
   In file included from drivers/scsi/hisi_sas/hisi_sas.h:16:
   In file included from include/linux/libata.h:21:
   In file included from include/scsi/scsi_host.h:11:
   In file included from include/linux/blk-mq.h:5:
   In file included from include/linux/blkdev.h:13:
>> include/linux/pagemap.h:181:2: error: called object type 'void' is not a function or function pointer
           VM_BUG_ON_PAGE(page_count(page) == 0, page);
           ^
   include/linux/mmdebug.h:46:36: note: expanded from macro 'VM_BUG_ON_PAGE'
   #define VM_BUG_ON_PAGE(cond, page) VM_BUG_ON(cond)
                                      ^
   include/linux/mmdebug.h:45:25: note: expanded from macro 'VM_BUG_ON'
   #define VM_BUG_ON(cond) BUILD_BUG_ON_INVALID(cond)
                           ^
   include/linux/build_bug.h:30:33: note: expanded from macro 'BUILD_BUG_ON_INVALID'
   #define BUILD_BUG_ON_INVALID(e) ((void)(sizeof((__force long)(e))))
                                   ^
>> drivers/scsi/hisi_sas/hisi_sas_main.c:2579:41: warning: shift count >= width of type [-Wshift-count-overflow]
           error = dma_set_mask_and_coherent(dev, DMA_BIT_MASK(64));
                                                  ^~~~~~~~~~~~~~~~
   include/linux/dma-mapping.h:139:54: note: expanded from macro 'DMA_BIT_MASK'
   #define DMA_BIT_MASK(n) (((n) == 64) ? ~0ULL : ((1ULL<<(n))-1))
                                                        ^ ~~~
   1 warning and 1 error generated.
--
   In file included from drivers/scsi/scsi_sysfs.c:13:
   In file included from include/linux/blkdev.h:13:
>> include/linux/pagemap.h:181:2: error: called object type 'void' is not a function or function pointer
           VM_BUG_ON_PAGE(page_count(page) == 0, page);
           ^
   include/linux/mmdebug.h:46:36: note: expanded from macro 'VM_BUG_ON_PAGE'
   #define VM_BUG_ON_PAGE(cond, page) VM_BUG_ON(cond)
                                      ^
   include/linux/mmdebug.h:45:25: note: expanded from macro 'VM_BUG_ON'
   #define VM_BUG_ON(cond) BUILD_BUG_ON_INVALID(cond)
                           ^
   include/linux/build_bug.h:30:33: note: expanded from macro 'BUILD_BUG_ON_INVALID'
   #define BUILD_BUG_ON_INVALID(e) ((void)(sizeof((__force long)(e))))
                                   ^
   In file included from drivers/scsi/scsi_sysfs.c:20:
   In file included from include/scsi/scsi_tcq.h:6:
   In file included from include/scsi/scsi_cmnd.h:5:
   include/linux/dma-mapping.h:632:9: warning: implicit conversion from 'unsigned long long' to 'unsigned long' changes value from 18446744073709551615 to 4294967295 [-Wconstant-conversion]
           return DMA_BIT_MASK(32);
           ~~~~~~ ^~~~~~~~~~~~~~~~
   include/linux/dma-mapping.h:139:40: note: expanded from macro 'DMA_BIT_MASK'
   #define DMA_BIT_MASK(n) (((n) == 64) ? ~0ULL : ((1ULL<<(n))-1))
                                          ^~~~~
   drivers/scsi/scsi_sysfs.c:373:1: warning: format specifies type 'short' but the argument has type 'int' [-Wformat]
   shost_rd_attr(can_queue, "%hd\n");
   ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
                             %d
   drivers/scsi/scsi_sysfs.c:176:45: note: expanded from macro 'shost_rd_attr'
   #define shost_rd_attr(field, format_string) \
                                               ^
   drivers/scsi/scsi_sysfs.c:173:2: note: expanded from macro '\
   shost_rd_attr2'
           shost_show_function(name, field, format_string)                 \
           ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
   drivers/scsi/scsi_sysfs.c:165:43: note: expanded from macro 'shost_show_function'
           return snprintf (buf, 20, format_string, shost->field);         \
                                     ~~~~~~~~~~~~~  ^~~~~~~~~~~~
   drivers/scsi/scsi_sysfs.c:1027:10: fatal error: 'scsi_devinfo_tbl.c' file not found
   #include "scsi_devinfo_tbl.c"
            ^~~~~~~~~~~~~~~~~~~~
   2 warnings and 2 errors generated.

# https://github.com/0day-ci/linux/commit/a4a0f54fdd08d95dfe20d684b405db8a47fb61d8
git remote add linux-review https://github.com/0day-ci/linux
git fetch --no-tags linux-review Thomas-Gleixner/preempt-Make-preempt-count-unconditional/20200915-044640
git checkout a4a0f54fdd08d95dfe20d684b405db8a47fb61d8
vim +/void +181 include/linux/pagemap.h

^1da177e4c3f41 Linus Torvalds         2005-04-16  123  
e286781d5f2e9c Nick Piggin            2008-07-25  124  /*
e286781d5f2e9c Nick Piggin            2008-07-25  125   * speculatively take a reference to a page.
0139aa7b7fa12c Joonsoo Kim            2016-05-19  126   * If the page is free (_refcount == 0), then _refcount is untouched, and 0
0139aa7b7fa12c Joonsoo Kim            2016-05-19  127   * is returned. Otherwise, _refcount is incremented by 1 and 1 is returned.
e286781d5f2e9c Nick Piggin            2008-07-25  128   *
e286781d5f2e9c Nick Piggin            2008-07-25  129   * This function must be called inside the same rcu_read_lock() section as has
e286781d5f2e9c Nick Piggin            2008-07-25  130   * been used to lookup the page in the pagecache radix-tree (or page table):
0139aa7b7fa12c Joonsoo Kim            2016-05-19  131   * this allows allocators to use a synchronize_rcu() to stabilize _refcount.
e286781d5f2e9c Nick Piggin            2008-07-25  132   *
e286781d5f2e9c Nick Piggin            2008-07-25  133   * Unless an RCU grace period has passed, the count of all pages coming out
e286781d5f2e9c Nick Piggin            2008-07-25  134   * of the allocator must be considered unstable. page_count may return higher
e286781d5f2e9c Nick Piggin            2008-07-25  135   * than expected, and put_page must be able to do the right thing when the
e286781d5f2e9c Nick Piggin            2008-07-25  136   * page has been finished with, no matter what it is subsequently allocated
e286781d5f2e9c Nick Piggin            2008-07-25  137   * for (because put_page is what is used here to drop an invalid speculative
e286781d5f2e9c Nick Piggin            2008-07-25  138   * reference).
e286781d5f2e9c Nick Piggin            2008-07-25  139   *
e286781d5f2e9c Nick Piggin            2008-07-25  140   * This is the interesting part of the lockless pagecache (and lockless
e286781d5f2e9c Nick Piggin            2008-07-25  141   * get_user_pages) locking protocol, where the lookup-side (eg. find_get_page)
e286781d5f2e9c Nick Piggin            2008-07-25  142   * has the following pattern:
e286781d5f2e9c Nick Piggin            2008-07-25  143   * 1. find page in radix tree
e286781d5f2e9c Nick Piggin            2008-07-25  144   * 2. conditionally increment refcount
e286781d5f2e9c Nick Piggin            2008-07-25  145   * 3. check the page is still in pagecache (if no, goto 1)
e286781d5f2e9c Nick Piggin            2008-07-25  146   *
0139aa7b7fa12c Joonsoo Kim            2016-05-19  147   * Remove-side that cares about stability of _refcount (eg. reclaim) has the
b93b016313b3ba Matthew Wilcox         2018-04-10  148   * following (with the i_pages lock held):
e286781d5f2e9c Nick Piggin            2008-07-25  149   * A. atomically check refcount is correct and set it to 0 (atomic_cmpxchg)
e286781d5f2e9c Nick Piggin            2008-07-25  150   * B. remove page from pagecache
e286781d5f2e9c Nick Piggin            2008-07-25  151   * C. free the page
e286781d5f2e9c Nick Piggin            2008-07-25  152   *
e286781d5f2e9c Nick Piggin            2008-07-25  153   * There are 2 critical interleavings that matter:
e286781d5f2e9c Nick Piggin            2008-07-25  154   * - 2 runs before A: in this case, A sees elevated refcount and bails out
e286781d5f2e9c Nick Piggin            2008-07-25  155   * - A runs before 2: in this case, 2 sees zero refcount and retries;
e286781d5f2e9c Nick Piggin            2008-07-25  156   *   subsequently, B will complete and 1 will find no page, causing the
e286781d5f2e9c Nick Piggin            2008-07-25  157   *   lookup to return NULL.
e286781d5f2e9c Nick Piggin            2008-07-25  158   *
e286781d5f2e9c Nick Piggin            2008-07-25  159   * It is possible that between 1 and 2, the page is removed then the exact same
e286781d5f2e9c Nick Piggin            2008-07-25  160   * page is inserted into the same position in pagecache. That's OK: the
b93b016313b3ba Matthew Wilcox         2018-04-10  161   * old find_get_page using a lock could equally have run before or after
e286781d5f2e9c Nick Piggin            2008-07-25  162   * such a re-insertion, depending on order that locks are granted.
e286781d5f2e9c Nick Piggin            2008-07-25  163   *
e286781d5f2e9c Nick Piggin            2008-07-25  164   * Lookups racing against pagecache insertion isn't a big problem: either 1
e286781d5f2e9c Nick Piggin            2008-07-25  165   * will find the page or it will not. Likewise, the old find_get_page could run
e286781d5f2e9c Nick Piggin            2008-07-25  166   * either before the insertion or afterwards, depending on timing.
e286781d5f2e9c Nick Piggin            2008-07-25  167   */
494eec70f05496 john.hubbard(a)gmail.com 2019-03-05  168  static inline int __page_cache_add_speculative(struct page *page, int count)
e286781d5f2e9c Nick Piggin            2008-07-25  169  {
8375ad98cc1def Paul E. McKenney       2013-04-29  170  #ifdef CONFIG_TINY_RCU
a4a0f54fdd08d9 Thomas Gleixner        2020-09-14  171  	VM_BUG_ON(preemptible())
e286781d5f2e9c Nick Piggin            2008-07-25  172  	/*
e286781d5f2e9c Nick Piggin            2008-07-25  173  	 * Preempt must be disabled here - we rely on rcu_read_lock doing
e286781d5f2e9c Nick Piggin            2008-07-25  174  	 * this for us.
e286781d5f2e9c Nick Piggin            2008-07-25  175  	 *
e286781d5f2e9c Nick Piggin            2008-07-25  176  	 * Pagecache won't be truncated from interrupt context, so if we have
e286781d5f2e9c Nick Piggin            2008-07-25  177  	 * found a page in the radix tree here, we have pinned its refcount by
e286781d5f2e9c Nick Piggin            2008-07-25  178  	 * disabling preempt, and hence no need for the "speculative get" that
e286781d5f2e9c Nick Piggin            2008-07-25  179  	 * SMP requires.
e286781d5f2e9c Nick Piggin            2008-07-25  180  	 */
309381feaee564 Sasha Levin            2014-01-23 @181  	VM_BUG_ON_PAGE(page_count(page) == 0, page);
494eec70f05496 john.hubbard(a)gmail.com 2019-03-05  182  	page_ref_add(page, count);
e286781d5f2e9c Nick Piggin            2008-07-25  183  

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org

[-- Attachment #2: config.gz --]
[-- Type: application/gzip, Size: 26917 bytes --]

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 05/13] mm/pagemap: Clenaup PREEMPT_COUNT leftovers
  2020-09-14 20:42   ` Thomas Gleixner
                     ` (4 preceding siblings ...)
  (?)
@ 2020-09-15  2:40   ` kernel test robot
  -1 siblings, 0 replies; 282+ messages in thread
From: kernel test robot @ 2020-09-15  2:40 UTC (permalink / raw)
  To: kbuild-all

[-- Attachment #1: Type: text/plain, Size: 5715 bytes --]

Hi Thomas,

I love your patch! Yet something to improve:

[auto build test ERROR on drm-intel/for-linux-next]
[also build test ERROR on linus/master v5.9-rc5 next-20200914]
[cannot apply to rcu/dev arm/for-next tip/sched/core linux/master]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]

url:    https://github.com/0day-ci/linux/commits/Thomas-Gleixner/preempt-Make-preempt-count-unconditional/20200915-044640
base:   git://anongit.freedesktop.org/drm-intel for-linux-next
config: i386-randconfig-r035-20200913 (attached as .config)
compiler: gcc-9 (Debian 9.3.0-15) 9.3.0
reproduce (this is a W=1 build):
        # save the attached .config to linux build tree
        make W=1 ARCH=i386 

If you fix the issue, kindly add following tag as appropriate
Reported-by: kernel test robot <lkp@intel.com>

All errors (new ones prefixed by >>):

   In file included from include/linux/gfp.h:5,
                    from include/linux/slab.h:15,
                    from include/linux/crypto.h:20,
                    from arch/x86/kernel/asm-offsets.c:9:
   include/linux/pagemap.h: In function '__page_cache_add_speculative':
>> include/linux/mmdebug.h:20:2: error: expected ';' before 'do'
      20 |  do {        \
         |  ^~
   include/linux/pagemap.h:181:2: note: in expansion of macro 'VM_BUG_ON_PAGE'
     181 |  VM_BUG_ON_PAGE(page_count(page) == 0, page);
         |  ^~~~~~~~~~~~~~
   make[2]: *** [scripts/Makefile.build:117: arch/x86/kernel/asm-offsets.s] Error 1
   make[2]: Target '__build' not remade because of errors.
   make[1]: *** [Makefile:1198: prepare0] Error 2
   make[1]: Target 'prepare' not remade because of errors.
   make: *** [Makefile:185: __sub-make] Error 2
   make: Target 'prepare' not remade because of errors.

# https://github.com/0day-ci/linux/commit/a4a0f54fdd08d95dfe20d684b405db8a47fb61d8
git remote add linux-review https://github.com/0day-ci/linux
git fetch --no-tags linux-review Thomas-Gleixner/preempt-Make-preempt-count-unconditional/20200915-044640
git checkout a4a0f54fdd08d95dfe20d684b405db8a47fb61d8
vim +20 include/linux/mmdebug.h

309381feaee564 Sasha Levin           2014-01-23  16  
59ea746337c69f Jiri Slaby            2008-06-12  17  #ifdef CONFIG_DEBUG_VM
59ea746337c69f Jiri Slaby            2008-06-12  18  #define VM_BUG_ON(cond) BUG_ON(cond)
309381feaee564 Sasha Levin           2014-01-23  19  #define VM_BUG_ON_PAGE(cond, page)					\
e4f674229ce63d Dave Hansen           2014-06-04 @20  	do {								\
e4f674229ce63d Dave Hansen           2014-06-04  21  		if (unlikely(cond)) {					\
e4f674229ce63d Dave Hansen           2014-06-04  22  			dump_page(page, "VM_BUG_ON_PAGE(" __stringify(cond)")");\
e4f674229ce63d Dave Hansen           2014-06-04  23  			BUG();						\
e4f674229ce63d Dave Hansen           2014-06-04  24  		}							\
e4f674229ce63d Dave Hansen           2014-06-04  25  	} while (0)
fa3759ccd5651c Sasha Levin           2014-10-09  26  #define VM_BUG_ON_VMA(cond, vma)					\
fa3759ccd5651c Sasha Levin           2014-10-09  27  	do {								\
fa3759ccd5651c Sasha Levin           2014-10-09  28  		if (unlikely(cond)) {					\
fa3759ccd5651c Sasha Levin           2014-10-09  29  			dump_vma(vma);					\
fa3759ccd5651c Sasha Levin           2014-10-09  30  			BUG();						\
fa3759ccd5651c Sasha Levin           2014-10-09  31  		}							\
fa3759ccd5651c Sasha Levin           2014-10-09  32  	} while (0)
31c9afa6db122a Sasha Levin           2014-10-09  33  #define VM_BUG_ON_MM(cond, mm)						\
31c9afa6db122a Sasha Levin           2014-10-09  34  	do {								\
31c9afa6db122a Sasha Levin           2014-10-09  35  		if (unlikely(cond)) {					\
31c9afa6db122a Sasha Levin           2014-10-09  36  			dump_mm(mm);					\
31c9afa6db122a Sasha Levin           2014-10-09  37  			BUG();						\
31c9afa6db122a Sasha Levin           2014-10-09  38  		}							\
31c9afa6db122a Sasha Levin           2014-10-09  39  	} while (0)
91241681c62a5a Michal Hocko          2018-04-05  40  #define VM_WARN_ON(cond) (void)WARN_ON(cond)
91241681c62a5a Michal Hocko          2018-04-05  41  #define VM_WARN_ON_ONCE(cond) (void)WARN_ON_ONCE(cond)
91241681c62a5a Michal Hocko          2018-04-05  42  #define VM_WARN_ONCE(cond, format...) (void)WARN_ONCE(cond, format)
91241681c62a5a Michal Hocko          2018-04-05  43  #define VM_WARN(cond, format...) (void)WARN(cond, format)
59ea746337c69f Jiri Slaby            2008-06-12  44  #else
02602a18c32d76 Konstantin Khlebnikov 2012-05-29  45  #define VM_BUG_ON(cond) BUILD_BUG_ON_INVALID(cond)
309381feaee564 Sasha Levin           2014-01-23  46  #define VM_BUG_ON_PAGE(cond, page) VM_BUG_ON(cond)
fa3759ccd5651c Sasha Levin           2014-10-09  47  #define VM_BUG_ON_VMA(cond, vma) VM_BUG_ON(cond)
31c9afa6db122a Sasha Levin           2014-10-09  48  #define VM_BUG_ON_MM(cond, mm) VM_BUG_ON(cond)
02a8efeda894d3 Andrew Morton         2014-06-04  49  #define VM_WARN_ON(cond) BUILD_BUG_ON_INVALID(cond)
02a8efeda894d3 Andrew Morton         2014-06-04  50  #define VM_WARN_ON_ONCE(cond) BUILD_BUG_ON_INVALID(cond)
ef6b571fb8920d Andrew Morton         2014-08-06  51  #define VM_WARN_ONCE(cond, format...) BUILD_BUG_ON_INVALID(cond)
a54f9aebaa9f0e Aneesh Kumar K.V      2016-07-26  52  #define VM_WARN(cond, format...) BUILD_BUG_ON_INVALID(cond)
59ea746337c69f Jiri Slaby            2008-06-12  53  #endif
59ea746337c69f Jiri Slaby            2008-06-12  54  

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org

[-- Attachment #2: config.gz --]
[-- Type: application/gzip, Size: 30523 bytes --]

^ permalink raw reply	[flat|nested] 282+ messages in thread

* [PATCH] crypto: lib/chacha20poly1305 - Set SG_MITER_ATOMIC unconditionally
  2020-09-14 22:37         ` Linus Torvalds
                             ` (2 preceding siblings ...)
  (?)
@ 2020-09-15  3:21           ` Herbert Xu
  -1 siblings, 0 replies; 282+ messages in thread
From: Herbert Xu @ 2020-09-15  3:21 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Thomas Gleixner, Ard Biesheuvel, LKML, linux-arch,
	Sebastian Andrzej Siewior, Valentin Schneider, Richard Henderson,
	Ivan Kokshaysky, Matt Turner, alpha, Jeff Dike,
	Richard Weinberger, Anton Ivanov, linux-um, Brian Cain,
	linux-hexagon, Geert Uytterhoeven, linux-m68k, Ingo Molnar,
	Peter Zijlstra, Juri Lelli, Vincent Guittot, Dietmar Eggemann,
	Steven Rostedt, Ben Segall, Mel Gorman,
	Daniel Bristot de Oliveira, Will Deacon, Andrew Morton, Linux-MM,
	Ingo Molnar, Russell King, Linux ARM, Chris Zankel, Max Filippov,
	linux-xtensa, Jani Nikula, Joonas Lahtinen, Rodrigo Vivi,
	David Airlie, Daniel Vetter, intel-gfx, dri-devel,
	Paul E. McKenney, Josh Triplett, Mathieu Desnoyers,
	Lai Jiangshan, Shuah Khan, rcu,
	open list:KERNEL SELFTEST FRAMEWORK <linux-kselftest

On Mon, Sep 14, 2020 at 03:37:49PM -0700, Linus Torvalds wrote:
>
> So it _looks_ like this code started using kmap() - probably back when
> kmap_atomic() was so cumbersome to use - and was then converted
> (conditionally) to kmap_atomic() rather than just changed whole-sale.
> Is there actually something that wants to use those sg_miter functions
> and sleep?

I dug up the old zinc patch submissions and this wasn't present at
all in the original.  The original zinc code used blkcipher_walk
which unconditinoally does kmap_atomic.

So it's only the SG miter conversion that introduced this change,
which appears to be a simple oversight (I think Ard was working on
latency issues at that time, perhaps he was worried about keeping
preemption off unnecessarily).

---8<---
There is no reason for the chacha20poly1305 SG miter code to use
kmap instead of kmap_atomic as the critical section doesn't sleep
anyway.  So we can simply get rid of the preemptible check and
set SG_MITER_ATOMIC unconditionally.

Even if we need to reenable preemption to lower latency we should
be doing that by interrupting the SG miter walk rather than using
kmap.

Reported-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

diff --git a/lib/crypto/chacha20poly1305.c b/lib/crypto/chacha20poly1305.c
index 431e04280332..5850f3b87359 100644
--- a/lib/crypto/chacha20poly1305.c
+++ b/lib/crypto/chacha20poly1305.c
@@ -251,9 +251,7 @@ bool chacha20poly1305_crypt_sg_inplace(struct scatterlist *src,
 			poly1305_update(&poly1305_state, pad0, 0x10 - (ad_len & 0xf));
 	}
 
-	flags = SG_MITER_TO_SG;
-	if (!preemptible())
-		flags |= SG_MITER_ATOMIC;
+	flags = SG_MITER_TO_SG | SG_MITER_ATOMIC;
 
 	sg_miter_start(&miter, src, sg_nents(src), flags);
 
-- 
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt


^ permalink raw reply related	[flat|nested] 282+ messages in thread

* [PATCH] crypto: lib/chacha20poly1305 - Set SG_MITER_ATOMIC unconditionally
@ 2020-09-15  3:21           ` Herbert Xu
  0 siblings, 0 replies; 282+ messages in thread
From: Herbert Xu @ 2020-09-15  3:21 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, Linux-MM, linux-hexagon,
	Will Deacon, Ingo Molnar, Anton Ivanov, linux-arch,
	Vincent Guittot, Brian Cain, Richard Weinberger, Russell King,
	Ard Biesheuvel, David Airlie, Ingo Molnar, Geert Uytterhoeven,
	Mel Gorman, intel-gfx, Matt Turner, Valentin Schneider,
	linux-xtensa, Shuah Khan, Paul E. McKenney,
	open list:KERNEL SELFTEST FRAMEWORK, Jeff Dike, linux-um,
	Josh Triplett, Steven Rostedt, rcu, linux-m68k, Ivan Kokshaysky,
	Rodrigo Vivi, Thomas Gleixner, Dietmar Eggemann, Linux ARM,
	Richard Henderson, Chris Zankel, Max Filippov, LKML, alpha,
	Mathieu Desnoyers, Andrew Morton, Daniel Bristot de Oliveira

On Mon, Sep 14, 2020 at 03:37:49PM -0700, Linus Torvalds wrote:
>
> So it _looks_ like this code started using kmap() - probably back when
> kmap_atomic() was so cumbersome to use - and was then converted
> (conditionally) to kmap_atomic() rather than just changed whole-sale.
> Is there actually something that wants to use those sg_miter functions
> and sleep?

I dug up the old zinc patch submissions and this wasn't present at
all in the original.  The original zinc code used blkcipher_walk
which unconditinoally does kmap_atomic.

So it's only the SG miter conversion that introduced this change,
which appears to be a simple oversight (I think Ard was working on
latency issues at that time, perhaps he was worried about keeping
preemption off unnecessarily).

---8<---
There is no reason for the chacha20poly1305 SG miter code to use
kmap instead of kmap_atomic as the critical section doesn't sleep
anyway.  So we can simply get rid of the preemptible check and
set SG_MITER_ATOMIC unconditionally.

Even if we need to reenable preemption to lower latency we should
be doing that by interrupting the SG miter walk rather than using
kmap.

Reported-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

diff --git a/lib/crypto/chacha20poly1305.c b/lib/crypto/chacha20poly1305.c
index 431e04280332..5850f3b87359 100644
--- a/lib/crypto/chacha20poly1305.c
+++ b/lib/crypto/chacha20poly1305.c
@@ -251,9 +251,7 @@ bool chacha20poly1305_crypt_sg_inplace(struct scatterlist *src,
 			poly1305_update(&poly1305_state, pad0, 0x10 - (ad_len & 0xf));
 	}
 
-	flags = SG_MITER_TO_SG;
-	if (!preemptible())
-		flags |= SG_MITER_ATOMIC;
+	flags = SG_MITER_TO_SG | SG_MITER_ATOMIC;
 
 	sg_miter_start(&miter, src, sg_nents(src), flags);
 
-- 
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply related	[flat|nested] 282+ messages in thread

* [Intel-gfx] [PATCH] crypto: lib/chacha20poly1305 - Set SG_MITER_ATOMIC unconditionally
@ 2020-09-15  3:21           ` Herbert Xu
  0 siblings, 0 replies; 282+ messages in thread
From: Herbert Xu @ 2020-09-15  3:21 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, Linux-MM, linux-hexagon,
	Will Deacon, Ingo Molnar, Anton Ivanov, linux-arch, Brian Cain,
	Richard Weinberger, Russell King, Ard Biesheuvel, David Airlie,
	Ingo Molnar, Geert Uytterhoeven, Mel Gorman, intel-gfx,
	Matt Turner, Valentin Schneider, linux-xtensa, Shuah Khan,
	Paul E. McKenney, open list:KERNEL SELFTEST FRAMEWORK, Jeff Dike,
	linux-um, Josh Triplett, Steven Rostedt, rcu, linux-m68k,
	Ivan Kokshaysky, Thomas Gleixner, Dietmar Eggemann, Linux ARM,
	Richard Henderson, Chris Zankel, Max Filippov, LKML, alpha,
	Mathieu Desnoyers, Andrew Morton, Daniel Bristot de Oliveira

On Mon, Sep 14, 2020 at 03:37:49PM -0700, Linus Torvalds wrote:
>
> So it _looks_ like this code started using kmap() - probably back when
> kmap_atomic() was so cumbersome to use - and was then converted
> (conditionally) to kmap_atomic() rather than just changed whole-sale.
> Is there actually something that wants to use those sg_miter functions
> and sleep?

I dug up the old zinc patch submissions and this wasn't present at
all in the original.  The original zinc code used blkcipher_walk
which unconditinoally does kmap_atomic.

So it's only the SG miter conversion that introduced this change,
which appears to be a simple oversight (I think Ard was working on
latency issues at that time, perhaps he was worried about keeping
preemption off unnecessarily).

---8<---
There is no reason for the chacha20poly1305 SG miter code to use
kmap instead of kmap_atomic as the critical section doesn't sleep
anyway.  So we can simply get rid of the preemptible check and
set SG_MITER_ATOMIC unconditionally.

Even if we need to reenable preemption to lower latency we should
be doing that by interrupting the SG miter walk rather than using
kmap.

Reported-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

diff --git a/lib/crypto/chacha20poly1305.c b/lib/crypto/chacha20poly1305.c
index 431e04280332..5850f3b87359 100644
--- a/lib/crypto/chacha20poly1305.c
+++ b/lib/crypto/chacha20poly1305.c
@@ -251,9 +251,7 @@ bool chacha20poly1305_crypt_sg_inplace(struct scatterlist *src,
 			poly1305_update(&poly1305_state, pad0, 0x10 - (ad_len & 0xf));
 	}
 
-	flags = SG_MITER_TO_SG;
-	if (!preemptible())
-		flags |= SG_MITER_ATOMIC;
+	flags = SG_MITER_TO_SG | SG_MITER_ATOMIC;
 
 	sg_miter_start(&miter, src, sg_nents(src), flags);
 
-- 
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

^ permalink raw reply related	[flat|nested] 282+ messages in thread

* [PATCH] crypto: lib/chacha20poly1305 - Set SG_MITER_ATOMIC unconditionally
@ 2020-09-15  3:21           ` Herbert Xu
  0 siblings, 0 replies; 282+ messages in thread
From: Herbert Xu @ 2020-09-15  3:21 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Thomas Gleixner, Ard Biesheuvel, LKML, linux-arch,
	Sebastian Andrzej Siewior, Valentin Schneider, Richard Henderson,
	Ivan Kokshaysky, Matt Turner, alpha, Jeff Dike,
	Richard Weinberger, Anton Ivanov, linux-um, Brian Cain,
	linux-hexagon, Geert Uytterhoeven, linux-m68k, Ingo Molnar,
	Peter Zijlstra, Juri Lelli, Vincent Guittot, Dietmar Eggemann,
	Steven Rostedt, Ben Segall, Mel Gorman,
	Daniel Bristot de Oliveira, Will Deacon, Andrew Morton, Linux-MM,
	Ingo Molnar, Russell King, Linux ARM, Chris Zankel, Max Filippov,
	linux-xtensa, Jani Nikula, Joonas Lahtinen, Rodrigo Vivi,
	David Airlie, Daniel Vetter, intel-gfx, dri-devel,
	Paul E. McKenney, Josh Triplett, Mathieu Desnoyers,
	Lai Jiangshan, Shuah Khan, rcu,
	open list:KERNEL SELFTEST FRAMEWORK

On Mon, Sep 14, 2020 at 03:37:49PM -0700, Linus Torvalds wrote:
>
> So it _looks_ like this code started using kmap() - probably back when
> kmap_atomic() was so cumbersome to use - and was then converted
> (conditionally) to kmap_atomic() rather than just changed whole-sale.
> Is there actually something that wants to use those sg_miter functions
> and sleep?

I dug up the old zinc patch submissions and this wasn't present at
all in the original.  The original zinc code used blkcipher_walk
which unconditinoally does kmap_atomic.

So it's only the SG miter conversion that introduced this change,
which appears to be a simple oversight (I think Ard was working on
latency issues at that time, perhaps he was worried about keeping
preemption off unnecessarily).

---8<---
There is no reason for the chacha20poly1305 SG miter code to use
kmap instead of kmap_atomic as the critical section doesn't sleep
anyway.  So we can simply get rid of the preemptible check and
set SG_MITER_ATOMIC unconditionally.

Even if we need to reenable preemption to lower latency we should
be doing that by interrupting the SG miter walk rather than using
kmap.

Reported-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

diff --git a/lib/crypto/chacha20poly1305.c b/lib/crypto/chacha20poly1305.c
index 431e04280332..5850f3b87359 100644
--- a/lib/crypto/chacha20poly1305.c
+++ b/lib/crypto/chacha20poly1305.c
@@ -251,9 +251,7 @@ bool chacha20poly1305_crypt_sg_inplace(struct scatterlist *src,
 			poly1305_update(&poly1305_state, pad0, 0x10 - (ad_len & 0xf));
 	}
 
-	flags = SG_MITER_TO_SG;
-	if (!preemptible())
-		flags |= SG_MITER_ATOMIC;
+	flags = SG_MITER_TO_SG | SG_MITER_ATOMIC;
 
 	sg_miter_start(&miter, src, sg_nents(src), flags);
 
-- 
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt


^ permalink raw reply related	[flat|nested] 282+ messages in thread

* [PATCH] crypto: lib/chacha20poly1305 - Set SG_MITER_ATOMIC unconditionally
@ 2020-09-15  3:21           ` Herbert Xu
  0 siblings, 0 replies; 282+ messages in thread
From: Herbert Xu @ 2020-09-15  3:21 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, Linux-MM, linux-hexagon,
	Will Deacon, Ingo Molnar, Anton Ivanov, linux-arch, Brian Cain,
	Richard Weinberger, Russell King, Ard Biesheuvel, David Airlie,
	Ingo Molnar, Geert Uytterhoeven, Mel Gorman, intel-gfx,
	Matt Turner, Valentin Schneider, linux-xtensa

On Mon, Sep 14, 2020 at 03:37:49PM -0700, Linus Torvalds wrote:
>
> So it _looks_ like this code started using kmap() - probably back when
> kmap_atomic() was so cumbersome to use - and was then converted
> (conditionally) to kmap_atomic() rather than just changed whole-sale.
> Is there actually something that wants to use those sg_miter functions
> and sleep?

I dug up the old zinc patch submissions and this wasn't present at
all in the original.  The original zinc code used blkcipher_walk
which unconditinoally does kmap_atomic.

So it's only the SG miter conversion that introduced this change,
which appears to be a simple oversight (I think Ard was working on
latency issues at that time, perhaps he was worried about keeping
preemption off unnecessarily).

---8<---
There is no reason for the chacha20poly1305 SG miter code to use
kmap instead of kmap_atomic as the critical section doesn't sleep
anyway.  So we can simply get rid of the preemptible check and
set SG_MITER_ATOMIC unconditionally.

Even if we need to reenable preemption to lower latency we should
be doing that by interrupting the SG miter walk rather than using
kmap.

Reported-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

diff --git a/lib/crypto/chacha20poly1305.c b/lib/crypto/chacha20poly1305.c
index 431e04280332..5850f3b87359 100644
--- a/lib/crypto/chacha20poly1305.c
+++ b/lib/crypto/chacha20poly1305.c
@@ -251,9 +251,7 @@ bool chacha20poly1305_crypt_sg_inplace(struct scatterlist *src,
 			poly1305_update(&poly1305_state, pad0, 0x10 - (ad_len & 0xf));
 	}
 
-	flags = SG_MITER_TO_SG;
-	if (!preemptible())
-		flags |= SG_MITER_ATOMIC;
+	flags = SG_MITER_TO_SG | SG_MITER_ATOMIC;
 
 	sg_miter_start(&miter, src, sg_nents(src), flags);
 
-- 
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

^ permalink raw reply related	[flat|nested] 282+ messages in thread

* [PATCH] crypto: lib/chacha20poly1305 - Set SG_MITER_ATOMIC unconditionally
  2020-09-14 22:37         ` Linus Torvalds
                           ` (3 preceding siblings ...)
  (?)
@ 2020-09-15  3:30         ` Herbert Xu
  2020-09-15  6:03           ` Ard Biesheuvel
  2020-09-15  6:45           ` Linus Torvalds
  -1 siblings, 2 replies; 282+ messages in thread
From: Herbert Xu @ 2020-09-15  3:30 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Thomas Gleixner, Ard Biesheuvel, LKML, Linux Crypto Mailing List

I trimmed the cc as the mailing lists appear to be blocking this
email because of it.

On Mon, Sep 14, 2020 at 03:37:49PM -0700, Linus Torvalds wrote:
>
> So it _looks_ like this code started using kmap() - probably back when
> kmap_atomic() was so cumbersome to use - and was then converted
> (conditionally) to kmap_atomic() rather than just changed whole-sale.
> Is there actually something that wants to use those sg_miter functions
> and sleep?

I dug up the old zinc patch submissions and this wasn't present at
all in the original.  The original zinc code used blkcipher_walk
which unconditinoally does kmap_atomic.

So it's only the SG miter conversion that introduced this change,
which appears to be a simple oversight (I think Ard was working on
latency issues at that time, perhaps he was worried about keeping
preemption off unnecessarily).

---8<---
There is no reason for the chacha20poly1305 SG miter code to use
kmap instead of kmap_atomic as the critical section doesn't sleep
anyway.  So we can simply get rid of the preemptible check and
set SG_MITER_ATOMIC unconditionally.

Even if we need to reenable preemption to lower latency we should
be doing that by interrupting the SG miter walk rather than using
kmap.

Reported-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

diff --git a/lib/crypto/chacha20poly1305.c b/lib/crypto/chacha20poly1305.c
index 431e04280332..5850f3b87359 100644
--- a/lib/crypto/chacha20poly1305.c
+++ b/lib/crypto/chacha20poly1305.c
@@ -251,9 +251,7 @@ bool chacha20poly1305_crypt_sg_inplace(struct scatterlist *src,
 			poly1305_update(&poly1305_state, pad0, 0x10 - (ad_len & 0xf));
 	}
 
-	flags = SG_MITER_TO_SG;
-	if (!preemptible())
-		flags |= SG_MITER_ATOMIC;
+	flags = SG_MITER_TO_SG | SG_MITER_ATOMIC;
 
 	sg_miter_start(&miter, src, sg_nents(src), flags);
 
-- 
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

^ permalink raw reply related	[flat|nested] 282+ messages in thread

* Re: [patch 05/13] mm/pagemap: Clenaup PREEMPT_COUNT leftovers
  2020-09-14 20:42   ` Thomas Gleixner
                     ` (5 preceding siblings ...)
  (?)
@ 2020-09-15  5:28   ` kernel test robot
  -1 siblings, 0 replies; 282+ messages in thread
From: kernel test robot @ 2020-09-15  5:28 UTC (permalink / raw)
  To: kbuild-all

[-- Attachment #1: Type: text/plain, Size: 20406 bytes --]

Hi Thomas,

I love your patch! Yet something to improve:

[auto build test ERROR on drm-intel/for-linux-next]
[also build test ERROR on linus/master v5.9-rc5 next-20200914]
[cannot apply to rcu/dev arm/for-next tip/sched/core linux/master]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]

url:    https://github.com/0day-ci/linux/commits/Thomas-Gleixner/preempt-Make-preempt-count-unconditional/20200915-044640
base:   git://anongit.freedesktop.org/drm-intel for-linux-next
config: microblaze-randconfig-r033-20200913 (attached as .config)
compiler: microblaze-linux-gcc (GCC) 9.3.0
reproduce (this is a W=1 build):
        wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
        chmod +x ~/bin/make.cross
        # save the attached .config to linux build tree
        COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-9.3.0 make.cross ARCH=microblaze 

If you fix the issue, kindly add following tag as appropriate
Reported-by: kernel test robot <lkp@intel.com>

All errors (new ones prefixed by >>):

   In file included from include/linux/bits.h:22,
                    from include/linux/bitops.h:5,
                    from include/linux/kernel.h:12,
                    from include/linux/interrupt.h:6,
                    from drivers/net/can/m_can/m_can.c:12:
   include/linux/pagemap.h: In function '__page_cache_add_speculative':
>> include/linux/build_bug.h:30:34: error: called object is not a function or function pointer
      30 | #define BUILD_BUG_ON_INVALID(e) ((void)(sizeof((__force long)(e))))
         |                                 ~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
   include/linux/mmdebug.h:45:25: note: in expansion of macro 'BUILD_BUG_ON_INVALID'
      45 | #define VM_BUG_ON(cond) BUILD_BUG_ON_INVALID(cond)
         |                         ^~~~~~~~~~~~~~~~~~~~
   include/linux/pagemap.h:171:2: note: in expansion of macro 'VM_BUG_ON'
     171 |  VM_BUG_ON(preemptible())
         |  ^~~~~~~~~
--
   In file included from include/linux/bits.h:22,
                    from include/linux/bitops.h:5,
                    from include/linux/kernel.h:12,
                    from include/linux/list.h:9,
                    from include/linux/timer.h:5,
                    from drivers/target/target_core_pscsi.c:15:
   include/linux/pagemap.h: In function '__page_cache_add_speculative':
>> include/linux/build_bug.h:30:34: error: called object is not a function or function pointer
      30 | #define BUILD_BUG_ON_INVALID(e) ((void)(sizeof((__force long)(e))))
         |                                 ~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
   include/linux/mmdebug.h:45:25: note: in expansion of macro 'BUILD_BUG_ON_INVALID'
      45 | #define VM_BUG_ON(cond) BUILD_BUG_ON_INVALID(cond)
         |                         ^~~~~~~~~~~~~~~~~~~~
   include/linux/pagemap.h:171:2: note: in expansion of macro 'VM_BUG_ON'
     171 |  VM_BUG_ON(preemptible())
         |  ^~~~~~~~~
   drivers/target/target_core_pscsi.c: In function 'pscsi_complete_cmd':
   drivers/target/target_core_pscsi.c:624:5: warning: suggest braces around empty body in an 'if' statement [-Wempty-body]
     624 |     ; /* XXX: TCM_LOGICAL_UNIT_COMMUNICATION_FAILURE */
         |     ^
--
   In file included from include/linux/bits.h:22,
                    from include/linux/bitops.h:5,
                    from include/linux/kernel.h:12,
                    from include/linux/list.h:9,
                    from include/linux/module.h:12,
                    from drivers/nvme/target/core.c:7:
   include/linux/pagemap.h: In function '__page_cache_add_speculative':
>> include/linux/build_bug.h:30:34: error: called object is not a function or function pointer
      30 | #define BUILD_BUG_ON_INVALID(e) ((void)(sizeof((__force long)(e))))
         |                                 ~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
   include/linux/mmdebug.h:45:25: note: in expansion of macro 'BUILD_BUG_ON_INVALID'
      45 | #define VM_BUG_ON(cond) BUILD_BUG_ON_INVALID(cond)
         |                         ^~~~~~~~~~~~~~~~~~~~
   include/linux/pagemap.h:171:2: note: in expansion of macro 'VM_BUG_ON'
     171 |  VM_BUG_ON(preemptible())
         |  ^~~~~~~~~
   In file included from drivers/nvme/target/trace.h:169,
                    from drivers/nvme/target/core.c:14:
   include/trace/define_trace.h: At top level:
   include/trace/define_trace.h:95:42: fatal error: ./trace.h: No such file or directory
      95 | #include TRACE_INCLUDE(TRACE_INCLUDE_FILE)
         |                                          ^
   compilation terminated.
--
   In file included from include/linux/bits.h:22,
                    from include/linux/bitops.h:5,
                    from include/linux/kernel.h:12,
                    from include/linux/uio.h:8,
                    from include/linux/socket.h:8,
                    from include/rdma/rdma_cm.h:10,
                    from drivers/infiniband/core/cma_trace.c:12:
   include/linux/pagemap.h: In function '__page_cache_add_speculative':
>> include/linux/build_bug.h:30:34: error: called object is not a function or function pointer
      30 | #define BUILD_BUG_ON_INVALID(e) ((void)(sizeof((__force long)(e))))
         |                                 ~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
   include/linux/mmdebug.h:45:25: note: in expansion of macro 'BUILD_BUG_ON_INVALID'
      45 | #define VM_BUG_ON(cond) BUILD_BUG_ON_INVALID(cond)
         |                         ^~~~~~~~~~~~~~~~~~~~
   include/linux/pagemap.h:171:2: note: in expansion of macro 'VM_BUG_ON'
     171 |  VM_BUG_ON(preemptible())
         |  ^~~~~~~~~
   In file included from drivers/infiniband/core/cma_trace.h:401,
                    from drivers/infiniband/core/cma_trace.c:16:
   include/trace/define_trace.h: At top level:
   include/trace/define_trace.h:95:42: fatal error: ./cma_trace.h: No such file or directory
      95 | #include TRACE_INCLUDE(TRACE_INCLUDE_FILE)
         |                                          ^
   compilation terminated.
--
   In file included from include/linux/bits.h:22,
                    from include/linux/bitops.h:5,
                    from include/linux/kernel.h:12,
                    from drivers/ata/ahci_platform.c:11:
   include/linux/pagemap.h: In function '__page_cache_add_speculative':
>> include/linux/build_bug.h:30:34: error: called object is not a function or function pointer
      30 | #define BUILD_BUG_ON_INVALID(e) ((void)(sizeof((__force long)(e))))
         |                                 ~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
   include/linux/mmdebug.h:45:25: note: in expansion of macro 'BUILD_BUG_ON_INVALID'
      45 | #define VM_BUG_ON(cond) BUILD_BUG_ON_INVALID(cond)
         |                         ^~~~~~~~~~~~~~~~~~~~
   include/linux/pagemap.h:171:2: note: in expansion of macro 'VM_BUG_ON'
     171 |  VM_BUG_ON(preemptible())
         |  ^~~~~~~~~
   In file included from drivers/ata/ahci_platform.c:21:
   drivers/ata/ahci_platform.c: At top level:
   drivers/ata/ahci.h:385:16: warning: initialized field overwritten [-Woverride-init]
     385 |  .can_queue  = AHCI_MAX_CMDS,   \
         |                ^~~~~~~~~~~~~
   drivers/ata/ahci_platform.c:40:2: note: in expansion of macro 'AHCI_SHT'
      40 |  AHCI_SHT(DRV_NAME),
         |  ^~~~~~~~
   drivers/ata/ahci.h:385:16: note: (near initialization for 'ahci_platform_sht.can_queue')
     385 |  .can_queue  = AHCI_MAX_CMDS,   \
         |                ^~~~~~~~~~~~~
   drivers/ata/ahci_platform.c:40:2: note: in expansion of macro 'AHCI_SHT'
      40 |  AHCI_SHT(DRV_NAME),
         |  ^~~~~~~~
   drivers/ata/ahci.h:389:17: warning: initialized field overwritten [-Woverride-init]
     389 |  .sdev_attrs  = ahci_sdev_attrs
         |                 ^~~~~~~~~~~~~~~
   drivers/ata/ahci_platform.c:40:2: note: in expansion of macro 'AHCI_SHT'
      40 |  AHCI_SHT(DRV_NAME),
         |  ^~~~~~~~
   drivers/ata/ahci.h:389:17: note: (near initialization for 'ahci_platform_sht.sdev_attrs')
     389 |  .sdev_attrs  = ahci_sdev_attrs
         |                 ^~~~~~~~~~~~~~~
   drivers/ata/ahci_platform.c:40:2: note: in expansion of macro 'AHCI_SHT'
      40 |  AHCI_SHT(DRV_NAME),
         |  ^~~~~~~~
--
   In file included from include/linux/bits.h:22,
                    from include/linux/bitops.h:5,
                    from include/linux/kernel.h:12,
                    from drivers/ata/ahci_ceva.c:10:
   include/linux/pagemap.h: In function '__page_cache_add_speculative':
>> include/linux/build_bug.h:30:34: error: called object is not a function or function pointer
      30 | #define BUILD_BUG_ON_INVALID(e) ((void)(sizeof((__force long)(e))))
         |                                 ~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
   include/linux/mmdebug.h:45:25: note: in expansion of macro 'BUILD_BUG_ON_INVALID'
      45 | #define VM_BUG_ON(cond) BUILD_BUG_ON_INVALID(cond)
         |                         ^~~~~~~~~~~~~~~~~~~~
   include/linux/pagemap.h:171:2: note: in expansion of macro 'VM_BUG_ON'
     171 |  VM_BUG_ON(preemptible())
         |  ^~~~~~~~~
   In file included from drivers/ata/ahci_ceva.c:15:
   drivers/ata/ahci_ceva.c: At top level:
   drivers/ata/ahci.h:385:16: warning: initialized field overwritten [-Woverride-init]
     385 |  .can_queue  = AHCI_MAX_CMDS,   \
         |                ^~~~~~~~~~~~~
   drivers/ata/ahci_ceva.c:187:2: note: in expansion of macro 'AHCI_SHT'
     187 |  AHCI_SHT(DRV_NAME),
         |  ^~~~~~~~
   drivers/ata/ahci.h:385:16: note: (near initialization for 'ahci_platform_sht.can_queue')
     385 |  .can_queue  = AHCI_MAX_CMDS,   \
         |                ^~~~~~~~~~~~~
   drivers/ata/ahci_ceva.c:187:2: note: in expansion of macro 'AHCI_SHT'
     187 |  AHCI_SHT(DRV_NAME),
         |  ^~~~~~~~
   drivers/ata/ahci.h:389:17: warning: initialized field overwritten [-Woverride-init]
     389 |  .sdev_attrs  = ahci_sdev_attrs
         |                 ^~~~~~~~~~~~~~~
   drivers/ata/ahci_ceva.c:187:2: note: in expansion of macro 'AHCI_SHT'
     187 |  AHCI_SHT(DRV_NAME),
         |  ^~~~~~~~
   drivers/ata/ahci.h:389:17: note: (near initialization for 'ahci_platform_sht.sdev_attrs')
     389 |  .sdev_attrs  = ahci_sdev_attrs
         |                 ^~~~~~~~~~~~~~~
   drivers/ata/ahci_ceva.c:187:2: note: in expansion of macro 'AHCI_SHT'
     187 |  AHCI_SHT(DRV_NAME),
         |  ^~~~~~~~
--
   In file included from include/linux/bits.h:22,
                    from include/linux/bitops.h:5,
                    from include/linux/kernel.h:12,
                    from include/linux/list.h:9,
                    from include/linux/module.h:12,
                    from drivers/net/ethernet/intel/e1000/e1000.h:10,
                    from drivers/net/ethernet/intel/e1000/e1000_hw.c:8:
   include/linux/pagemap.h: In function '__page_cache_add_speculative':
>> include/linux/build_bug.h:30:34: error: called object is not a function or function pointer
      30 | #define BUILD_BUG_ON_INVALID(e) ((void)(sizeof((__force long)(e))))
         |                                 ~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
   include/linux/mmdebug.h:45:25: note: in expansion of macro 'BUILD_BUG_ON_INVALID'
      45 | #define VM_BUG_ON(cond) BUILD_BUG_ON_INVALID(cond)
         |                         ^~~~~~~~~~~~~~~~~~~~
   include/linux/pagemap.h:171:2: note: in expansion of macro 'VM_BUG_ON'
     171 |  VM_BUG_ON(preemptible())
         |  ^~~~~~~~~
   drivers/net/ethernet/intel/e1000/e1000_hw.c: In function 'e1000_phy_init_script':
   drivers/net/ethernet/intel/e1000/e1000_hw.c:132:6: warning: variable 'ret_val' set but not used [-Wunused-but-set-variable]
     132 |  u32 ret_val;
         |      ^~~~~~~
   drivers/net/ethernet/intel/e1000/e1000_hw.c: In function 'e1000_reset_hw':
   drivers/net/ethernet/intel/e1000/e1000_hw.c:380:6: warning: variable 'icr' set but not used [-Wunused-but-set-variable]
     380 |  u32 icr;
         |      ^~~
   drivers/net/ethernet/intel/e1000/e1000_hw.c: In function 'e1000_check_for_link':
   drivers/net/ethernet/intel/e1000/e1000_hw.c:2378:6: warning: variable 'signal' set but not used [-Wunused-but-set-variable]
    2378 |  u32 signal = 0;
         |      ^~~~~~
   drivers/net/ethernet/intel/e1000/e1000_hw.c:2374:6: warning: variable 'ctrl' set but not used [-Wunused-but-set-variable]
    2374 |  u32 ctrl;
         |      ^~~~
   drivers/net/ethernet/intel/e1000/e1000_hw.c:2373:6: warning: variable 'rxcw' set but not used [-Wunused-but-set-variable]
    2373 |  u32 rxcw = 0;
         |      ^~~~
   drivers/net/ethernet/intel/e1000/e1000_hw.c: In function 'e1000_clear_hw_cntrs':
   drivers/net/ethernet/intel/e1000/e1000_hw.c:4678:15: warning: variable 'temp' set but not used [-Wunused-but-set-variable]
    4678 |  volatile u32 temp;
         |               ^~~~
--
   In file included from include/linux/bits.h:22,
                    from include/linux/bitops.h:5,
                    from include/linux/kernel.h:12,
                    from include/linux/list.h:9,
                    from include/linux/module.h:12,
                    from drivers/video/fbdev/uvesafb.c:13:
   include/linux/pagemap.h: In function '__page_cache_add_speculative':
>> include/linux/build_bug.h:30:34: error: called object is not a function or function pointer
      30 | #define BUILD_BUG_ON_INVALID(e) ((void)(sizeof((__force long)(e))))
         |                                 ~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
   include/linux/mmdebug.h:45:25: note: in expansion of macro 'BUILD_BUG_ON_INVALID'
      45 | #define VM_BUG_ON(cond) BUILD_BUG_ON_INVALID(cond)
         |                         ^~~~~~~~~~~~~~~~~~~~
   include/linux/pagemap.h:171:2: note: in expansion of macro 'VM_BUG_ON'
     171 |  VM_BUG_ON(preemptible())
         |  ^~~~~~~~~
   drivers/video/fbdev/uvesafb.c: In function 'uvesafb_vbe_getinfo':
   drivers/video/fbdev/uvesafb.c:426:2: warning: 'strncpy' output truncated before terminating nul copying 4 bytes from a string of the same length [-Wstringop-truncation]
     426 |  strncpy(par->vbe_ib.vbe_signature, "VBE2", 4);
         |  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
--
   In file included from include/linux/bits.h:22,
                    from include/linux/bitops.h:5,
                    from include/linux/kernel.h:12,
                    from include/linux/list.h:9,
                    from include/linux/rculist.h:10,
                    from include/linux/pid.h:5,
                    from include/linux/sched.h:14,
                    from include/linux/utsname.h:6,
                    from drivers/char/random.c:312:
   include/linux/pagemap.h: In function '__page_cache_add_speculative':
>> include/linux/build_bug.h:30:34: error: called object is not a function or function pointer
      30 | #define BUILD_BUG_ON_INVALID(e) ((void)(sizeof((__force long)(e))))
         |                                 ~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
   include/linux/mmdebug.h:45:25: note: in expansion of macro 'BUILD_BUG_ON_INVALID'
      45 | #define VM_BUG_ON(cond) BUILD_BUG_ON_INVALID(cond)
         |                         ^~~~~~~~~~~~~~~~~~~~
   include/linux/pagemap.h:171:2: note: in expansion of macro 'VM_BUG_ON'
     171 |  VM_BUG_ON(preemptible())
         |  ^~~~~~~~~
   drivers/char/random.c: At top level:
   drivers/char/random.c:2297:6: warning: no previous prototype for 'add_hwgenerator_randomness' [-Wmissing-prototypes]
    2297 | void add_hwgenerator_randomness(const char *buffer, size_t count,
         |      ^~~~~~~~~~~~~~~~~~~~~~~~~~
--
   In file included from include/linux/bits.h:22,
                    from include/linux/bitops.h:5,
                    from include/linux/kernel.h:12,
                    from include/asm-generic/bug.h:20,
                    from ./arch/microblaze/include/generated/asm/bug.h:1,
                    from include/linux/bug.h:5,
                    from include/linux/mmdebug.h:5,
                    from include/linux/mm.h:9,
                    from include/linux/pagemap.h:8,
                    from drivers/dax/super.c:5:
   include/linux/pagemap.h: In function '__page_cache_add_speculative':
>> include/linux/build_bug.h:30:34: error: called object is not a function or function pointer
      30 | #define BUILD_BUG_ON_INVALID(e) ((void)(sizeof((__force long)(e))))
         |                                 ~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
   include/linux/mmdebug.h:45:25: note: in expansion of macro 'BUILD_BUG_ON_INVALID'
      45 | #define VM_BUG_ON(cond) BUILD_BUG_ON_INVALID(cond)
         |                         ^~~~~~~~~~~~~~~~~~~~
   include/linux/pagemap.h:171:2: note: in expansion of macro 'VM_BUG_ON'
     171 |  VM_BUG_ON(preemptible())
         |  ^~~~~~~~~
   drivers/dax/super.c: At top level:
   drivers/dax/super.c:447:6: warning: no previous prototype for 'run_dax' [-Wmissing-prototypes]
     447 | void run_dax(struct dax_device *dax_dev)
         |      ^~~~~~~
--
   In file included from include/linux/bits.h:22,
                    from include/linux/bitops.h:5,
                    from include/linux/kernel.h:12,
                    from include/linux/list.h:9,
                    from include/linux/module.h:12,
                    from drivers/block/drbd/drbd_main.c:19:
   include/linux/pagemap.h: In function '__page_cache_add_speculative':
>> include/linux/build_bug.h:30:34: error: called object is not a function or function pointer
      30 | #define BUILD_BUG_ON_INVALID(e) ((void)(sizeof((__force long)(e))))
         |                                 ~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
   include/linux/mmdebug.h:45:25: note: in expansion of macro 'BUILD_BUG_ON_INVALID'
      45 | #define VM_BUG_ON(cond) BUILD_BUG_ON_INVALID(cond)
         |                         ^~~~~~~~~~~~~~~~~~~~
   include/linux/pagemap.h:171:2: note: in expansion of macro 'VM_BUG_ON'
     171 |  VM_BUG_ON(preemptible())
         |  ^~~~~~~~~
   drivers/block/drbd/drbd_main.c: In function 'cmdname':
   drivers/block/drbd/drbd_main.c:3683:22: warning: initialized field overwritten [-Woverride-init]
    3683 |   [P_RETRY_WRITE]  = "retry_write",
         |                      ^~~~~~~~~~~~~
   drivers/block/drbd/drbd_main.c:3683:22: note: (near initialization for 'cmdnames[44]')
..

# https://github.com/0day-ci/linux/commit/a4a0f54fdd08d95dfe20d684b405db8a47fb61d8
git remote add linux-review https://github.com/0day-ci/linux
git fetch --no-tags linux-review Thomas-Gleixner/preempt-Make-preempt-count-unconditional/20200915-044640
git checkout a4a0f54fdd08d95dfe20d684b405db8a47fb61d8
vim +30 include/linux/build_bug.h

527edbc18a70e7 Masahiro Yamada 2019-01-03  18  
527edbc18a70e7 Masahiro Yamada 2019-01-03  19  /* Force a compilation error if a constant expression is not a power of 2 */
527edbc18a70e7 Masahiro Yamada 2019-01-03  20  #define __BUILD_BUG_ON_NOT_POWER_OF_2(n)	\
527edbc18a70e7 Masahiro Yamada 2019-01-03  21  	BUILD_BUG_ON(((n) & ((n) - 1)) != 0)
527edbc18a70e7 Masahiro Yamada 2019-01-03  22  #define BUILD_BUG_ON_NOT_POWER_OF_2(n)			\
527edbc18a70e7 Masahiro Yamada 2019-01-03  23  	BUILD_BUG_ON((n) == 0 || (((n) & ((n) - 1)) != 0))
bc6245e5efd70c Ian Abbott      2017-07-10  24  
bc6245e5efd70c Ian Abbott      2017-07-10  25  /*
bc6245e5efd70c Ian Abbott      2017-07-10  26   * BUILD_BUG_ON_INVALID() permits the compiler to check the validity of the
bc6245e5efd70c Ian Abbott      2017-07-10  27   * expression but avoids the generation of any code, even if that expression
bc6245e5efd70c Ian Abbott      2017-07-10  28   * has side-effects.
bc6245e5efd70c Ian Abbott      2017-07-10  29   */
bc6245e5efd70c Ian Abbott      2017-07-10 @30  #define BUILD_BUG_ON_INVALID(e) ((void)(sizeof((__force long)(e))))
bc6245e5efd70c Ian Abbott      2017-07-10  31  

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org

[-- Attachment #2: config.gz --]
[-- Type: application/gzip, Size: 29593 bytes --]

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [PATCH] crypto: lib/chacha20poly1305 - Set SG_MITER_ATOMIC unconditionally
  2020-09-15  3:30         ` Herbert Xu
@ 2020-09-15  6:03           ` Ard Biesheuvel
  2020-09-15  6:40             ` Herbert Xu
  2020-09-15  6:45           ` Linus Torvalds
  1 sibling, 1 reply; 282+ messages in thread
From: Ard Biesheuvel @ 2020-09-15  6:03 UTC (permalink / raw)
  To: Herbert Xu, Jason A. Donenfeld
  Cc: Linus Torvalds, Thomas Gleixner, LKML, Linux Crypto Mailing List

(+ Jason)

On Tue, 15 Sep 2020 at 06:30, Herbert Xu <herbert@gondor.apana.org.au> wrote:
>
> I trimmed the cc as the mailing lists appear to be blocking this
> email because of it.
>
> On Mon, Sep 14, 2020 at 03:37:49PM -0700, Linus Torvalds wrote:
> >
> > So it _looks_ like this code started using kmap() - probably back when
> > kmap_atomic() was so cumbersome to use - and was then converted
> > (conditionally) to kmap_atomic() rather than just changed whole-sale.
> > Is there actually something that wants to use those sg_miter functions
> > and sleep?
>
> I dug up the old zinc patch submissions and this wasn't present at
> all in the original.  The original zinc code used blkcipher_walk
> which unconditinoally does kmap_atomic.
>

Remember that the Zinc patchset was very vocal about not relying on
the Linux crypto API, yet it [ab]used the crypto blkcipher_walk API
(which was already deprecated at that point) in a rather horrid way,
by going around the blkcipher API itself, and creating some mock
objects that the blkcipher scatterlist walker would expect to exist.

So instead, I opted to rewrite this code using the SG miter API so that:
- src == dst, and so we only need to traverse (and kmap) a single
scatterlist instead of two in parallel (as Wireguard has no need for
the latter)
- no elaborate handling of the scatterlist elements when they are not
a multiple of the cipher chunk size (which is not needed for a stream
cipher liker ChaCha)
- no need to use scatterwalk_map_and_copy() (and do another kmap()) to
access the tag if it was covered by the last scatterlist element.

> So it's only the SG miter conversion that introduced this change,
> which appears to be a simple oversight (I think Ard was working on
> latency issues at that time, perhaps he was worried about keeping
> preemption off unnecessarily).
>

No, the problem with using kmap_atomic() is that it disables
preemption even on !HIGHMEM architectures. So using it unconditionally
here means that all chacha/poly processing will execute with
preemption disabled on 64-bit architectures as well.

This means that, even if you avoid the SIMD accelerated ciphers for
latency reasons (as they disable preemption as well), you are still
running the bulk of the WireGuard processing with preemption disabled.

> ---8<---
> There is no reason for the chacha20poly1305 SG miter code to use
> kmap instead of kmap_atomic as the critical section doesn't sleep
> anyway.  So we can simply get rid of the preemptible check and
> set SG_MITER_ATOMIC unconditionally.
>
> Even if we need to reenable preemption to lower latency we should
> be doing that by interrupting the SG miter walk rather than using
> kmap.
>

AIUI, the common case is that the entire packet is covered by a single
scatterlist element, so there is no room for latency reduction here.

The problem is really that kmap_atomic() is not simply a no-op on
!HIGHMEM architectures. If we can fix that, I have no objections to
this patch.

> Reported-by: Linus Torvalds <torvalds@linux-foundation.org>
> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
>
> diff --git a/lib/crypto/chacha20poly1305.c b/lib/crypto/chacha20poly1305.c
> index 431e04280332..5850f3b87359 100644
> --- a/lib/crypto/chacha20poly1305.c
> +++ b/lib/crypto/chacha20poly1305.c
> @@ -251,9 +251,7 @@ bool chacha20poly1305_crypt_sg_inplace(struct scatterlist *src,
>                         poly1305_update(&poly1305_state, pad0, 0x10 - (ad_len & 0xf));
>         }
>
> -       flags = SG_MITER_TO_SG;
> -       if (!preemptible())
> -               flags |= SG_MITER_ATOMIC;
> +       flags = SG_MITER_TO_SG | SG_MITER_ATOMIC;
>
>         sg_miter_start(&miter, src, sg_nents(src), flags);
>
> --
> Email: Herbert Xu <herbert@gondor.apana.org.au>
> Home Page: http://gondor.apana.org.au/~herbert/
> PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
  2020-09-14 22:37         ` Linus Torvalds
                             ` (2 preceding siblings ...)
  (?)
@ 2020-09-15  6:20           ` Ard Biesheuvel
  -1 siblings, 0 replies; 282+ messages in thread
From: Ard Biesheuvel @ 2020-09-15  6:20 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Thomas Gleixner, Herbert Xu, LKML, linux-arch,
	Sebastian Andrzej Siewior, Valentin Schneider, Richard Henderson,
	Ivan Kokshaysky, Matt Turner, alpha, Jeff Dike,
	Richard Weinberger, Anton Ivanov, linux-um, Brian Cain,
	linux-hexagon, Geert Uytterhoeven, linux-m68k, Ingo Molnar,
	Peter Zijlstra, Juri Lelli, Vincent Guittot, Dietmar Eggemann,
	Steven Rostedt, Ben Segall, Mel Gorman,
	Daniel Bristot de Oliveira, Will Deacon, Andrew Morton, Linux-MM,
	Ingo Molnar, Russell King, Linux ARM, Chris Zankel, Max Filippov,
	linux-xtensa, Jani Nikula, Joonas Lahtinen, Rodrigo Vivi,
	David Airlie, Daniel Vetter, intel-gfx, dri-devel,
	Paul E. McKenney, Josh Triplett, Mathieu Desnoyers,
	Lai Jiangshan, Shuah Khan, rcu,
	open list:KERNEL SELFTEST FRAMEWORK

On Tue, 15 Sep 2020 at 01:43, Linus Torvalds
<torvalds@linux-foundation.org> wrote:
>
> On Mon, Sep 14, 2020 at 3:24 PM Linus Torvalds
> <torvalds@linux-foundation.org> wrote:
> >
> > Ard and Herbert added to participants: see
> > chacha20poly1305_crypt_sg_inplace(), which does
> >
> >         flags = SG_MITER_TO_SG;
> >         if (!preemptible())
> >                 flags |= SG_MITER_ATOMIC;
> >
> > introduced in commit d95312a3ccc0 ("crypto: lib/chacha20poly1305 -
> > reimplement crypt_from_sg() routine").
>
> As far as I can tell, the only reason for this all is to try to use
> "kmap()" rather than "kmap_atomic()".
>
> And kmap() actually has the much more complex "might_sleep()" tests,
> and apparently the "preemptible()" check wasn't even the proper full
> debug check, it was just a complete hack to catch the one that
> triggered.
>

This was not driven by a failing check.

The documentation of kmap_atomic() states the following:

 * The use of kmap_atomic/kunmap_atomic is discouraged - kmap/kunmap
 * gives a more generic (and caching) interface. But kmap_atomic can
 * be used in IRQ contexts, so in some (very limited) cases we need
 * it.

so if this is no longer accurate, perhaps we should fix it?

But another reason I tried to avoid kmap_atomic() is that it disables
preemption unconditionally, even on 64-bit architectures where HIGHMEM
is irrelevant. So using kmap_atomic() here means that the bulk of
WireGuard packet encryption runs with preemption disabled, essentially
for legacy reasons.


> From a quick look, that code should probably just get rid of
> SG_MITER_ATOMIC entirely, and alwayse use kmap_atomic().
>
> kmap_atomic() is actually the faster and proper interface to use
> anyway (never mind that any of this matters on any sane hardware). The
> old kmap() and kunmap() interfaces should generally be avoided like
> the plague - yes, they allow sleeping in the middle and that is
> sometimes required, but if you don't need that, you should never ever
> use them.
>
> We used to have a very nasty kmap_atomic() that required people to be
> very careful and know exactly which atomic entry to use, and that was
> admitedly quite nasty.
>
> So it _looks_ like this code started using kmap() - probably back when
> kmap_atomic() was so cumbersome to use - and was then converted
> (conditionally) to kmap_atomic() rather than just changed whole-sale.
> Is there actually something that wants to use those sg_miter functions
> and sleep?
>
> Because if there is, that choice should come from the outside, not
> from inside lib/scatterlist.c trying to make some bad guess based on
> the wrong thing entirely.
>
>                  Linus

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-15  6:20           ` Ard Biesheuvel
  0 siblings, 0 replies; 282+ messages in thread
From: Ard Biesheuvel @ 2020-09-15  6:20 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Thomas Gleixner, Herbert Xu, LKML, linux-arch,
	Sebastian Andrzej Siewior, Valentin Schneider, Richard Henderson,
	Ivan Kokshaysky, Matt Turner, alpha, Jeff Dike,
	Richard Weinberger, Anton Ivanov, linux-um, Brian Cain,
	linux-hexagon, Geert Uytterhoeven, linux-m68k, Ingo Molnar,
	Peter Zijlstra, Juri Lelli, Vincent Guittot, Dietmar Eggemann,
	Steven Rostedt, Ben Segall, Mel Gorman,
	Daniel Bristot de Oliveira, Will Deacon, Andrew Morton, Linux-MM,
	Ingo Molnar, Russell King, Linux ARM, Chris Zankel, Max Filippov,
	linux-xtensa, Jani Nikula, Joonas Lahtinen, Rodrigo Vivi,
	David Airlie, Daniel Vetter, intel-gfx, dri-devel,
	Paul E. McKenney, Josh Triplett, Mathieu Desnoyers,
	Lai Jiangshan, Shuah Khan, rcu,
	open list:KERNEL SELFTEST FRAMEWORK

On Tue, 15 Sep 2020 at 01:43, Linus Torvalds
<torvalds@linux-foundation.org> wrote:
>
> On Mon, Sep 14, 2020 at 3:24 PM Linus Torvalds
> <torvalds@linux-foundation.org> wrote:
> >
> > Ard and Herbert added to participants: see
> > chacha20poly1305_crypt_sg_inplace(), which does
> >
> >         flags = SG_MITER_TO_SG;
> >         if (!preemptible())
> >                 flags |= SG_MITER_ATOMIC;
> >
> > introduced in commit d95312a3ccc0 ("crypto: lib/chacha20poly1305 -
> > reimplement crypt_from_sg() routine").
>
> As far as I can tell, the only reason for this all is to try to use
> "kmap()" rather than "kmap_atomic()".
>
> And kmap() actually has the much more complex "might_sleep()" tests,
> and apparently the "preemptible()" check wasn't even the proper full
> debug check, it was just a complete hack to catch the one that
> triggered.
>

This was not driven by a failing check.

The documentation of kmap_atomic() states the following:

 * The use of kmap_atomic/kunmap_atomic is discouraged - kmap/kunmap
 * gives a more generic (and caching) interface. But kmap_atomic can
 * be used in IRQ contexts, so in some (very limited) cases we need
 * it.

so if this is no longer accurate, perhaps we should fix it?

But another reason I tried to avoid kmap_atomic() is that it disables
preemption unconditionally, even on 64-bit architectures where HIGHMEM
is irrelevant. So using kmap_atomic() here means that the bulk of
WireGuard packet encryption runs with preemption disabled, essentially
for legacy reasons.


> From a quick look, that code should probably just get rid of
> SG_MITER_ATOMIC entirely, and alwayse use kmap_atomic().
>
> kmap_atomic() is actually the faster and proper interface to use
> anyway (never mind that any of this matters on any sane hardware). The
> old kmap() and kunmap() interfaces should generally be avoided like
> the plague - yes, they allow sleeping in the middle and that is
> sometimes required, but if you don't need that, you should never ever
> use them.
>
> We used to have a very nasty kmap_atomic() that required people to be
> very careful and know exactly which atomic entry to use, and that was
> admitedly quite nasty.
>
> So it _looks_ like this code started using kmap() - probably back when
> kmap_atomic() was so cumbersome to use - and was then converted
> (conditionally) to kmap_atomic() rather than just changed whole-sale.
> Is there actually something that wants to use those sg_miter functions
> and sleep?
>
> Because if there is, that choice should come from the outside, not
> from inside lib/scatterlist.c trying to make some bad guess based on
> the wrong thing entirely.
>
>                  Linus


^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-15  6:20           ` Ard Biesheuvel
  0 siblings, 0 replies; 282+ messages in thread
From: Ard Biesheuvel @ 2020-09-15  6:20 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, Linux-MM,
	open list:KERNEL SELFTEST FRAMEWORK, linux-hexagon, Will Deacon,
	Ingo Molnar, Anton Ivanov, linux-arch, Vincent Guittot,
	Herbert Xu, Brian Cain, Richard Weinberger, Russell King,
	David Airlie, Ingo Molnar, Geert Uytterhoeven, Mel Gorman,
	intel-gfx, Matt Turner, Valentin Schneider, linux-xtensa,
	Shuah Khan, Paul E. McKenney, Jeff Dike, linux-um, Josh Triplett,
	Steven Rostedt, rcu, linux-m68k, Ivan Kokshaysky, Rodrigo Vivi,
	Thomas Gleixner, Dietmar Eggemann, Linux ARM, Richard Henderson,
	Chris Zankel, Max Filippov, LKML, alpha, Mathieu Desnoyers,
	Andrew Morton, Daniel Bristot de Oliveira

On Tue, 15 Sep 2020 at 01:43, Linus Torvalds
<torvalds@linux-foundation.org> wrote:
>
> On Mon, Sep 14, 2020 at 3:24 PM Linus Torvalds
> <torvalds@linux-foundation.org> wrote:
> >
> > Ard and Herbert added to participants: see
> > chacha20poly1305_crypt_sg_inplace(), which does
> >
> >         flags = SG_MITER_TO_SG;
> >         if (!preemptible())
> >                 flags |= SG_MITER_ATOMIC;
> >
> > introduced in commit d95312a3ccc0 ("crypto: lib/chacha20poly1305 -
> > reimplement crypt_from_sg() routine").
>
> As far as I can tell, the only reason for this all is to try to use
> "kmap()" rather than "kmap_atomic()".
>
> And kmap() actually has the much more complex "might_sleep()" tests,
> and apparently the "preemptible()" check wasn't even the proper full
> debug check, it was just a complete hack to catch the one that
> triggered.
>

This was not driven by a failing check.

The documentation of kmap_atomic() states the following:

 * The use of kmap_atomic/kunmap_atomic is discouraged - kmap/kunmap
 * gives a more generic (and caching) interface. But kmap_atomic can
 * be used in IRQ contexts, so in some (very limited) cases we need
 * it.

so if this is no longer accurate, perhaps we should fix it?

But another reason I tried to avoid kmap_atomic() is that it disables
preemption unconditionally, even on 64-bit architectures where HIGHMEM
is irrelevant. So using kmap_atomic() here means that the bulk of
WireGuard packet encryption runs with preemption disabled, essentially
for legacy reasons.


> From a quick look, that code should probably just get rid of
> SG_MITER_ATOMIC entirely, and alwayse use kmap_atomic().
>
> kmap_atomic() is actually the faster and proper interface to use
> anyway (never mind that any of this matters on any sane hardware). The
> old kmap() and kunmap() interfaces should generally be avoided like
> the plague - yes, they allow sleeping in the middle and that is
> sometimes required, but if you don't need that, you should never ever
> use them.
>
> We used to have a very nasty kmap_atomic() that required people to be
> very careful and know exactly which atomic entry to use, and that was
> admitedly quite nasty.
>
> So it _looks_ like this code started using kmap() - probably back when
> kmap_atomic() was so cumbersome to use - and was then converted
> (conditionally) to kmap_atomic() rather than just changed whole-sale.
> Is there actually something that wants to use those sg_miter functions
> and sleep?
>
> Because if there is, that choice should come from the outside, not
> from inside lib/scatterlist.c trying to make some bad guess based on
> the wrong thing entirely.
>
>                  Linus
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [Intel-gfx] [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-15  6:20           ` Ard Biesheuvel
  0 siblings, 0 replies; 282+ messages in thread
From: Ard Biesheuvel @ 2020-09-15  6:20 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, Linux-MM,
	open list:KERNEL SELFTEST FRAMEWORK, linux-hexagon, Will Deacon,
	Ingo Molnar, Anton Ivanov, linux-arch, Herbert Xu, Brian Cain,
	Richard Weinberger, Russell King, David Airlie, Ingo Molnar,
	Geert Uytterhoeven, Mel Gorman, intel-gfx, Matt Turner,
	Valentin Schneider, linux-xtensa, Shuah Khan, Paul E. McKenney,
	Jeff Dike, linux-um, Josh Triplett, Steven Rostedt, rcu,
	linux-m68k, Ivan Kokshaysky, Thomas Gleixner, Dietmar Eggemann,
	Linux ARM, Richard Henderson, Chris Zankel, Max Filippov, LKML,
	alpha, Mathieu Desnoyers, Andrew Morton,
	Daniel Bristot de Oliveira

On Tue, 15 Sep 2020 at 01:43, Linus Torvalds
<torvalds@linux-foundation.org> wrote:
>
> On Mon, Sep 14, 2020 at 3:24 PM Linus Torvalds
> <torvalds@linux-foundation.org> wrote:
> >
> > Ard and Herbert added to participants: see
> > chacha20poly1305_crypt_sg_inplace(), which does
> >
> >         flags = SG_MITER_TO_SG;
> >         if (!preemptible())
> >                 flags |= SG_MITER_ATOMIC;
> >
> > introduced in commit d95312a3ccc0 ("crypto: lib/chacha20poly1305 -
> > reimplement crypt_from_sg() routine").
>
> As far as I can tell, the only reason for this all is to try to use
> "kmap()" rather than "kmap_atomic()".
>
> And kmap() actually has the much more complex "might_sleep()" tests,
> and apparently the "preemptible()" check wasn't even the proper full
> debug check, it was just a complete hack to catch the one that
> triggered.
>

This was not driven by a failing check.

The documentation of kmap_atomic() states the following:

 * The use of kmap_atomic/kunmap_atomic is discouraged - kmap/kunmap
 * gives a more generic (and caching) interface. But kmap_atomic can
 * be used in IRQ contexts, so in some (very limited) cases we need
 * it.

so if this is no longer accurate, perhaps we should fix it?

But another reason I tried to avoid kmap_atomic() is that it disables
preemption unconditionally, even on 64-bit architectures where HIGHMEM
is irrelevant. So using kmap_atomic() here means that the bulk of
WireGuard packet encryption runs with preemption disabled, essentially
for legacy reasons.


> From a quick look, that code should probably just get rid of
> SG_MITER_ATOMIC entirely, and alwayse use kmap_atomic().
>
> kmap_atomic() is actually the faster and proper interface to use
> anyway (never mind that any of this matters on any sane hardware). The
> old kmap() and kunmap() interfaces should generally be avoided like
> the plague - yes, they allow sleeping in the middle and that is
> sometimes required, but if you don't need that, you should never ever
> use them.
>
> We used to have a very nasty kmap_atomic() that required people to be
> very careful and know exactly which atomic entry to use, and that was
> admitedly quite nasty.
>
> So it _looks_ like this code started using kmap() - probably back when
> kmap_atomic() was so cumbersome to use - and was then converted
> (conditionally) to kmap_atomic() rather than just changed whole-sale.
> Is there actually something that wants to use those sg_miter functions
> and sleep?
>
> Because if there is, that choice should come from the outside, not
> from inside lib/scatterlist.c trying to make some bad guess based on
> the wrong thing entirely.
>
>                  Linus
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-15  6:20           ` Ard Biesheuvel
  0 siblings, 0 replies; 282+ messages in thread
From: Ard Biesheuvel @ 2020-09-15  6:20 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Thomas Gleixner, Herbert Xu, LKML, linux-arch,
	Sebastian Andrzej Siewior, Valentin Schneider, Richard Henderson,
	Ivan Kokshaysky, Matt Turner, alpha, Jeff Dike,
	Richard Weinberger, Anton Ivanov, linux-um, Brian Cain,
	linux-hexagon, Geert Uytterhoeven, linux-m68k, Ingo Molnar

On Tue, 15 Sep 2020 at 01:43, Linus Torvalds
<torvalds@linux-foundation.org> wrote:
>
> On Mon, Sep 14, 2020 at 3:24 PM Linus Torvalds
> <torvalds@linux-foundation.org> wrote:
> >
> > Ard and Herbert added to participants: see
> > chacha20poly1305_crypt_sg_inplace(), which does
> >
> >         flags = SG_MITER_TO_SG;
> >         if (!preemptible())
> >                 flags |= SG_MITER_ATOMIC;
> >
> > introduced in commit d95312a3ccc0 ("crypto: lib/chacha20poly1305 -
> > reimplement crypt_from_sg() routine").
>
> As far as I can tell, the only reason for this all is to try to use
> "kmap()" rather than "kmap_atomic()".
>
> And kmap() actually has the much more complex "might_sleep()" tests,
> and apparently the "preemptible()" check wasn't even the proper full
> debug check, it was just a complete hack to catch the one that
> triggered.
>

This was not driven by a failing check.

The documentation of kmap_atomic() states the following:

 * The use of kmap_atomic/kunmap_atomic is discouraged - kmap/kunmap
 * gives a more generic (and caching) interface. But kmap_atomic can
 * be used in IRQ contexts, so in some (very limited) cases we need
 * it.

so if this is no longer accurate, perhaps we should fix it?

But another reason I tried to avoid kmap_atomic() is that it disables
preemption unconditionally, even on 64-bit architectures where HIGHMEM
is irrelevant. So using kmap_atomic() here means that the bulk of
WireGuard packet encryption runs with preemption disabled, essentially
for legacy reasons.


> From a quick look, that code should probably just get rid of
> SG_MITER_ATOMIC entirely, and alwayse use kmap_atomic().
>
> kmap_atomic() is actually the faster and proper interface to use
> anyway (never mind that any of this matters on any sane hardware). The
> old kmap() and kunmap() interfaces should generally be avoided like
> the plague - yes, they allow sleeping in the middle and that is
> sometimes required, but if you don't need that, you should never ever
> use them.
>
> We used to have a very nasty kmap_atomic() that required people to be
> very careful and know exactly which atomic entry to use, and that was
> admitedly quite nasty.
>
> So it _looks_ like this code started using kmap() - probably back when
> kmap_atomic() was so cumbersome to use - and was then converted
> (conditionally) to kmap_atomic() rather than just changed whole-sale.
> Is there actually something that wants to use those sg_miter functions
> and sleep?
>
> Because if there is, that choice should come from the outside, not
> from inside lib/scatterlist.c trying to make some bad guess based on
> the wrong thing entirely.
>
>                  Linus

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
  2020-09-15  6:20           ` Ard Biesheuvel
                               ` (2 preceding siblings ...)
  (?)
@ 2020-09-15  6:22             ` Herbert Xu
  -1 siblings, 0 replies; 282+ messages in thread
From: Herbert Xu @ 2020-09-15  6:22 UTC (permalink / raw)
  To: Ard Biesheuvel
  Cc: Linus Torvalds, Thomas Gleixner, LKML, linux-arch,
	Sebastian Andrzej Siewior, Valentin Schneider, Richard Henderson,
	Ivan Kokshaysky, Matt Turner, alpha, Jeff Dike,
	Richard Weinberger, Anton Ivanov, linux-um, Brian Cain,
	linux-hexagon, Geert Uytterhoeven, linux-m68k, Ingo Molnar,
	Peter Zijlstra, Juri Lelli, Vincent Guittot, Dietmar Eggemann,
	Steven Rostedt, Ben Segall, Mel Gorman,
	Daniel Bristot de Oliveira, Will Deacon, Andrew Morton, Linux-MM,
	Ingo Molnar, Russell King, Linux ARM, Chris Zankel, Max Filippov,
	linux-xtensa, Jani Nikula, Joonas Lahtinen, Rodrigo Vivi,
	David Airlie, Daniel Vetter, intel-gfx, dri-devel,
	Paul E. McKenney, Josh Triplett, Mathieu Desnoyers,
	Lai Jiangshan, Shuah Khan, rcu,
	open list:KERNEL SELFTEST FRAMEWORK <linux-

On Tue, Sep 15, 2020 at 09:20:59AM +0300, Ard Biesheuvel wrote:
>
> The documentation of kmap_atomic() states the following:
> 
>  * The use of kmap_atomic/kunmap_atomic is discouraged - kmap/kunmap
>  * gives a more generic (and caching) interface. But kmap_atomic can
>  * be used in IRQ contexts, so in some (very limited) cases we need
>  * it.
> 
> so if this is no longer accurate, perhaps we should fix it?

This hasn't been accurate for at least ten years :)

> But another reason I tried to avoid kmap_atomic() is that it disables
> preemption unconditionally, even on 64-bit architectures where HIGHMEM
> is irrelevant. So using kmap_atomic() here means that the bulk of
> WireGuard packet encryption runs with preemption disabled, essentially
> for legacy reasons.

Agreed.  We should definitely fix that.

Thanks,
-- 
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt


^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-15  6:22             ` Herbert Xu
  0 siblings, 0 replies; 282+ messages in thread
From: Herbert Xu @ 2020-09-15  6:22 UTC (permalink / raw)
  To: Ard Biesheuvel
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, Linux-MM, linux-hexagon,
	Will Deacon, Ingo Molnar, Anton Ivanov, linux-arch,
	Vincent Guittot, Brian Cain, Richard Weinberger, Russell King,
	open list:KERNEL SELFTEST FRAMEWORK, David Airlie, Ingo Molnar,
	Geert Uytterhoeven, Mel Gorman, intel-gfx, Matt Turner,
	Valentin Schneider, linux-xtensa, Shuah Khan, Paul E. McKenney,
	Jeff Dike, linux-um, Josh Triplett, Steven Rostedt, rcu,
	linux-m68k, Ivan Kokshaysky, Rodrigo Vivi, Thomas Gleixner,
	Dietmar Eggemann, Linux ARM, Richard Henderson, Chris Zankel,
	Max Filippov, Daniel Bristot de Oliveira, LKML, alpha,
	Mathieu Desnoyers, Andrew Morton, Linus Torvalds

On Tue, Sep 15, 2020 at 09:20:59AM +0300, Ard Biesheuvel wrote:
>
> The documentation of kmap_atomic() states the following:
> 
>  * The use of kmap_atomic/kunmap_atomic is discouraged - kmap/kunmap
>  * gives a more generic (and caching) interface. But kmap_atomic can
>  * be used in IRQ contexts, so in some (very limited) cases we need
>  * it.
> 
> so if this is no longer accurate, perhaps we should fix it?

This hasn't been accurate for at least ten years :)

> But another reason I tried to avoid kmap_atomic() is that it disables
> preemption unconditionally, even on 64-bit architectures where HIGHMEM
> is irrelevant. So using kmap_atomic() here means that the bulk of
> WireGuard packet encryption runs with preemption disabled, essentially
> for legacy reasons.

Agreed.  We should definitely fix that.

Thanks,
-- 
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [Intel-gfx] [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-15  6:22             ` Herbert Xu
  0 siblings, 0 replies; 282+ messages in thread
From: Herbert Xu @ 2020-09-15  6:22 UTC (permalink / raw)
  To: Ard Biesheuvel
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, Linux-MM, linux-hexagon,
	Will Deacon, Ingo Molnar, Anton Ivanov, linux-arch, Brian Cain,
	Richard Weinberger, Russell King,
	open list:KERNEL SELFTEST FRAMEWORK, David Airlie, Ingo Molnar,
	Geert Uytterhoeven, Mel Gorman, intel-gfx, Matt Turner,
	Valentin Schneider, linux-xtensa, Shuah Khan, Paul E. McKenney,
	Jeff Dike, linux-um, Josh Triplett, Steven Rostedt, rcu,
	linux-m68k, Ivan Kokshaysky, Thomas Gleixner, Dietmar Eggemann,
	Linux ARM, Richard Henderson, Chris Zankel, Max Filippov,
	Daniel Bristot de Oliveira, LKML, alpha, Mathieu Desnoyers,
	Andrew Morton, Linus Torvalds

On Tue, Sep 15, 2020 at 09:20:59AM +0300, Ard Biesheuvel wrote:
>
> The documentation of kmap_atomic() states the following:
> 
>  * The use of kmap_atomic/kunmap_atomic is discouraged - kmap/kunmap
>  * gives a more generic (and caching) interface. But kmap_atomic can
>  * be used in IRQ contexts, so in some (very limited) cases we need
>  * it.
> 
> so if this is no longer accurate, perhaps we should fix it?

This hasn't been accurate for at least ten years :)

> But another reason I tried to avoid kmap_atomic() is that it disables
> preemption unconditionally, even on 64-bit architectures where HIGHMEM
> is irrelevant. So using kmap_atomic() here means that the bulk of
> WireGuard packet encryption runs with preemption disabled, essentially
> for legacy reasons.

Agreed.  We should definitely fix that.

Thanks,
-- 
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-15  6:22             ` Herbert Xu
  0 siblings, 0 replies; 282+ messages in thread
From: Herbert Xu @ 2020-09-15  6:22 UTC (permalink / raw)
  To: Ard Biesheuvel
  Cc: Linus Torvalds, Thomas Gleixner, LKML, linux-arch,
	Sebastian Andrzej Siewior, Valentin Schneider, Richard Henderson,
	Ivan Kokshaysky, Matt Turner, alpha, Jeff Dike,
	Richard Weinberger, Anton Ivanov, linux-um, Brian Cain,
	linux-hexagon, Geert Uytterhoeven, linux-m68k, Ingo Molnar,
	Peter Zijlstra, Juri Lelli, Vincent Guittot, Dietmar Eggemann,
	Steven Rostedt, Ben Segall, Mel Gorman,
	Daniel Bristot de Oliveira, Will Deacon, Andrew Morton, Linux-MM,
	Ingo Molnar, Russell King, Linux ARM, Chris Zankel, Max Filippov,
	linux-xtensa, Jani Nikula, Joonas Lahtinen, Rodrigo Vivi,
	David Airlie, Daniel Vetter, intel-gfx, dri-devel,
	Paul E. McKenney, Josh Triplett, Mathieu Desnoyers,
	Lai Jiangshan, Shuah Khan, rcu,
	open list:KERNEL SELFTEST FRAMEWORK

On Tue, Sep 15, 2020 at 09:20:59AM +0300, Ard Biesheuvel wrote:
>
> The documentation of kmap_atomic() states the following:
> 
>  * The use of kmap_atomic/kunmap_atomic is discouraged - kmap/kunmap
>  * gives a more generic (and caching) interface. But kmap_atomic can
>  * be used in IRQ contexts, so in some (very limited) cases we need
>  * it.
> 
> so if this is no longer accurate, perhaps we should fix it?

This hasn't been accurate for at least ten years :)

> But another reason I tried to avoid kmap_atomic() is that it disables
> preemption unconditionally, even on 64-bit architectures where HIGHMEM
> is irrelevant. So using kmap_atomic() here means that the bulk of
> WireGuard packet encryption runs with preemption disabled, essentially
> for legacy reasons.

Agreed.  We should definitely fix that.

Thanks,
-- 
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt


^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-15  6:22             ` Herbert Xu
  0 siblings, 0 replies; 282+ messages in thread
From: Herbert Xu @ 2020-09-15  6:22 UTC (permalink / raw)
  To: Ard Biesheuvel
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, Linux-MM, linux-hexagon,
	Will Deacon, Ingo Molnar, Anton Ivanov, linux-arch, Brian Cain,
	Richard Weinberger, Russell King,
	open list:KERNEL SELFTEST FRAMEWORK, David Airlie, Ingo Molnar,
	Geert Uytterhoeven, Mel Gorman, intel-gfx, Matt Turner,
	Valentin Schneider

On Tue, Sep 15, 2020 at 09:20:59AM +0300, Ard Biesheuvel wrote:
>
> The documentation of kmap_atomic() states the following:
> 
>  * The use of kmap_atomic/kunmap_atomic is discouraged - kmap/kunmap
>  * gives a more generic (and caching) interface. But kmap_atomic can
>  * be used in IRQ contexts, so in some (very limited) cases we need
>  * it.
> 
> so if this is no longer accurate, perhaps we should fix it?

This hasn't been accurate for at least ten years :)

> But another reason I tried to avoid kmap_atomic() is that it disables
> preemption unconditionally, even on 64-bit architectures where HIGHMEM
> is irrelevant. So using kmap_atomic() here means that the bulk of
> WireGuard packet encryption runs with preemption disabled, essentially
> for legacy reasons.

Agreed.  We should definitely fix that.

Thanks,
-- 
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
  2020-09-15  6:22             ` Herbert Xu
                                 ` (2 preceding siblings ...)
  (?)
@ 2020-09-15  6:39               ` Linus Torvalds
  -1 siblings, 0 replies; 282+ messages in thread
From: Linus Torvalds @ 2020-09-15  6:39 UTC (permalink / raw)
  To: Herbert Xu
  Cc: Ard Biesheuvel, Thomas Gleixner, LKML, linux-arch,
	Sebastian Andrzej Siewior, Valentin Schneider, Richard Henderson,
	Ivan Kokshaysky, Matt Turner, alpha, Jeff Dike,
	Richard Weinberger, Anton Ivanov, linux-um, Brian Cain,
	linux-hexagon, Geert Uytterhoeven, linux-m68k, Ingo Molnar,
	Peter Zijlstra, Juri Lelli, Vincent Guittot, Dietmar Eggemann,
	Steven Rostedt, Ben Segall, Mel Gorman,
	Daniel Bristot de Oliveira, Will Deacon, Andrew Morton, Linux-MM,
	Ingo Molnar, Russell King, Linux ARM, Chris Zankel, Max Filippov,
	linux-xtensa, Jani Nikula, Joonas Lahtinen, Rodrigo Vivi,
	David Airlie, Daniel Vetter, intel-gfx, dri-devel,
	Paul E. McKenney, Josh Triplett, Mathieu Desnoyers,
	Lai Jiangshan, Shuah Khan, rcu

On Mon, Sep 14, 2020 at 11:24 PM Herbert Xu <herbert@gondor.apana.org.au> wrote:
>
> On Tue, Sep 15, 2020 at 09:20:59AM +0300, Ard Biesheuvel wrote:
> >
> > The documentation of kmap_atomic() states the following:
> >
> >  * The use of kmap_atomic/kunmap_atomic is discouraged - kmap/kunmap
> >  * gives a more generic (and caching) interface. But kmap_atomic can
> >  * be used in IRQ contexts, so in some (very limited) cases we need
> >  * it.
> >
> > so if this is no longer accurate, perhaps we should fix it?
>
> This hasn't been accurate for at least ten years :)

Yeah, that used to be true a long long time ago, but the comment is very stale.

> > But another reason I tried to avoid kmap_atomic() is that it disables
> > preemption unconditionally, even on 64-bit architectures where HIGHMEM
> > is irrelevant. So using kmap_atomic() here means that the bulk of
> > WireGuard packet encryption runs with preemption disabled, essentially
> > for legacy reasons.
>
> Agreed.  We should definitely fix that.

Well, honestly, one big reason for that is debugging.

The *semantics* of the kmap_atomic() is in the name - you can't sleep
in between it and the kunmap_atomic().

On any sane architecture, kmap_atomic() ends up being a no-op from an
implementation standpoint, and sleeping would work just fine.

But we very much want to make sure that people don't then write code
that doesn't work on the bad old 32-bit machines where it really needs
that sequence to be safe from preemption.

So it's mostly a debug thing.

I say "mostly", because there might be small other details too, like
shared code, and perhaps even a couple of users out in the wild that
depend on the pagefault_disable() inherent in the current
kmap_atomic(), who knows..

So no, the preemption disabling isn't inherent in the operation
itself. But it does have some argument for it.

                   Linus

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-15  6:39               ` Linus Torvalds
  0 siblings, 0 replies; 282+ messages in thread
From: Linus Torvalds @ 2020-09-15  6:39 UTC (permalink / raw)
  To: Herbert Xu
  Cc: Ard Biesheuvel, Thomas Gleixner, LKML, linux-arch,
	Sebastian Andrzej Siewior, Valentin Schneider, Richard Henderson,
	Ivan Kokshaysky, Matt Turner, alpha, Jeff Dike,
	Richard Weinberger, Anton Ivanov, linux-um, Brian Cain,
	linux-hexagon, Geert Uytterhoeven, linux-m68k, Ingo Molnar,
	Peter Zijlstra, Juri Lelli, Vincent Guittot, Dietmar Eggemann,
	Steven Rostedt, Ben Segall, Mel Gorman,
	Daniel Bristot de Oliveira, Will Deacon, Andrew Morton, Linux-MM,
	Ingo Molnar, Russell King, Linux ARM, Chris Zankel, Max Filippov,
	linux-xtensa, Jani Nikula, Joonas Lahtinen, Rodrigo Vivi,
	David Airlie, Daniel Vetter, intel-gfx, dri-devel,
	Paul E. McKenney, Josh Triplett, Mathieu Desnoyers,
	Lai Jiangshan, Shuah Khan, rcu

On Mon, Sep 14, 2020 at 11:24 PM Herbert Xu <herbert@gondor.apana.org.au> wrote:
>
> On Tue, Sep 15, 2020 at 09:20:59AM +0300, Ard Biesheuvel wrote:
> >
> > The documentation of kmap_atomic() states the following:
> >
> >  * The use of kmap_atomic/kunmap_atomic is discouraged - kmap/kunmap
> >  * gives a more generic (and caching) interface. But kmap_atomic can
> >  * be used in IRQ contexts, so in some (very limited) cases we need
> >  * it.
> >
> > so if this is no longer accurate, perhaps we should fix it?
>
> This hasn't been accurate for at least ten years :)

Yeah, that used to be true a long long time ago, but the comment is very stale.

> > But another reason I tried to avoid kmap_atomic() is that it disables
> > preemption unconditionally, even on 64-bit architectures where HIGHMEM
> > is irrelevant. So using kmap_atomic() here means that the bulk of
> > WireGuard packet encryption runs with preemption disabled, essentially
> > for legacy reasons.
>
> Agreed.  We should definitely fix that.

Well, honestly, one big reason for that is debugging.

The *semantics* of the kmap_atomic() is in the name - you can't sleep
in between it and the kunmap_atomic().

On any sane architecture, kmap_atomic() ends up being a no-op from an
implementation standpoint, and sleeping would work just fine.

But we very much want to make sure that people don't then write code
that doesn't work on the bad old 32-bit machines where it really needs
that sequence to be safe from preemption.

So it's mostly a debug thing.

I say "mostly", because there might be small other details too, like
shared code, and perhaps even a couple of users out in the wild that
depend on the pagefault_disable() inherent in the current
kmap_atomic(), who knows..

So no, the preemption disabling isn't inherent in the operation
itself. But it does have some argument for it.

                   Linus


^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-15  6:39               ` Linus Torvalds
  0 siblings, 0 replies; 282+ messages in thread
From: Linus Torvalds @ 2020-09-15  6:39 UTC (permalink / raw)
  To: Herbert Xu
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, Linux-MM, linux-hexagon,
	Will Deacon, Ingo Molnar, Anton Ivanov, linux-arch,
	Vincent Guittot, Brian Cain, Richard Weinberger, Russell King,
	Ard Biesheuvel, David Airlie, Ingo Molnar, Geert Uytterhoeven,
	Mel Gorman, intel-gfx, Matt Turner, Valentin Schneider,
	linux-xtensa, Shuah Khan, Paul E. McKenney, Jeff Dike, linux-um,
	Josh Triplett, Steven Rostedt, rcu, linux-m68k, Ivan Kokshaysky,
	Rodrigo Vivi, Thomas Gleixner, Dietmar Eggemann, Linux ARM,
	Richard Henderson, Chris Zankel, Max Filippov, LKML, alpha,
	Mathieu Desnoyers, Andrew Morton, Daniel Bristot de Oliveira

On Mon, Sep 14, 2020 at 11:24 PM Herbert Xu <herbert@gondor.apana.org.au> wrote:
>
> On Tue, Sep 15, 2020 at 09:20:59AM +0300, Ard Biesheuvel wrote:
> >
> > The documentation of kmap_atomic() states the following:
> >
> >  * The use of kmap_atomic/kunmap_atomic is discouraged - kmap/kunmap
> >  * gives a more generic (and caching) interface. But kmap_atomic can
> >  * be used in IRQ contexts, so in some (very limited) cases we need
> >  * it.
> >
> > so if this is no longer accurate, perhaps we should fix it?
>
> This hasn't been accurate for at least ten years :)

Yeah, that used to be true a long long time ago, but the comment is very stale.

> > But another reason I tried to avoid kmap_atomic() is that it disables
> > preemption unconditionally, even on 64-bit architectures where HIGHMEM
> > is irrelevant. So using kmap_atomic() here means that the bulk of
> > WireGuard packet encryption runs with preemption disabled, essentially
> > for legacy reasons.
>
> Agreed.  We should definitely fix that.

Well, honestly, one big reason for that is debugging.

The *semantics* of the kmap_atomic() is in the name - you can't sleep
in between it and the kunmap_atomic().

On any sane architecture, kmap_atomic() ends up being a no-op from an
implementation standpoint, and sleeping would work just fine.

But we very much want to make sure that people don't then write code
that doesn't work on the bad old 32-bit machines where it really needs
that sequence to be safe from preemption.

So it's mostly a debug thing.

I say "mostly", because there might be small other details too, like
shared code, and perhaps even a couple of users out in the wild that
depend on the pagefault_disable() inherent in the current
kmap_atomic(), who knows..

So no, the preemption disabling isn't inherent in the operation
itself. But it does have some argument for it.

                   Linus
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [Intel-gfx] [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-15  6:39               ` Linus Torvalds
  0 siblings, 0 replies; 282+ messages in thread
From: Linus Torvalds @ 2020-09-15  6:39 UTC (permalink / raw)
  To: Herbert Xu
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, Linux-MM, linux-hexagon,
	Will Deacon, Ingo Molnar, Anton Ivanov, linux-arch, Brian Cain,
	Richard Weinberger, Russell King, Ard Biesheuvel, David Airlie,
	Ingo Molnar, Geert Uytterhoeven, Mel Gorman, intel-gfx,
	Matt Turner, Valentin Schneider, linux-xtensa, Shuah Khan,
	Paul E. McKenney, Jeff Dike, linux-um, Josh Triplett,
	Steven Rostedt, rcu, linux-m68k, Ivan Kokshaysky,
	Thomas Gleixner, Dietmar Eggemann, Linux ARM, Richard Henderson,
	Chris Zankel, Max Filippov, LKML, alpha, Mathieu Desnoyers,
	Andrew Morton, Daniel Bristot de Oliveira

On Mon, Sep 14, 2020 at 11:24 PM Herbert Xu <herbert@gondor.apana.org.au> wrote:
>
> On Tue, Sep 15, 2020 at 09:20:59AM +0300, Ard Biesheuvel wrote:
> >
> > The documentation of kmap_atomic() states the following:
> >
> >  * The use of kmap_atomic/kunmap_atomic is discouraged - kmap/kunmap
> >  * gives a more generic (and caching) interface. But kmap_atomic can
> >  * be used in IRQ contexts, so in some (very limited) cases we need
> >  * it.
> >
> > so if this is no longer accurate, perhaps we should fix it?
>
> This hasn't been accurate for at least ten years :)

Yeah, that used to be true a long long time ago, but the comment is very stale.

> > But another reason I tried to avoid kmap_atomic() is that it disables
> > preemption unconditionally, even on 64-bit architectures where HIGHMEM
> > is irrelevant. So using kmap_atomic() here means that the bulk of
> > WireGuard packet encryption runs with preemption disabled, essentially
> > for legacy reasons.
>
> Agreed.  We should definitely fix that.

Well, honestly, one big reason for that is debugging.

The *semantics* of the kmap_atomic() is in the name - you can't sleep
in between it and the kunmap_atomic().

On any sane architecture, kmap_atomic() ends up being a no-op from an
implementation standpoint, and sleeping would work just fine.

But we very much want to make sure that people don't then write code
that doesn't work on the bad old 32-bit machines where it really needs
that sequence to be safe from preemption.

So it's mostly a debug thing.

I say "mostly", because there might be small other details too, like
shared code, and perhaps even a couple of users out in the wild that
depend on the pagefault_disable() inherent in the current
kmap_atomic(), who knows..

So no, the preemption disabling isn't inherent in the operation
itself. But it does have some argument for it.

                   Linus
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-15  6:39               ` Linus Torvalds
  0 siblings, 0 replies; 282+ messages in thread
From: Linus Torvalds @ 2020-09-15  6:39 UTC (permalink / raw)
  To: Herbert Xu
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, Linux-MM, linux-hexagon,
	Will Deacon, Ingo Molnar, Anton Ivanov, linux-arch, Brian Cain,
	Richard Weinberger, Russell King, Ard Biesheuvel, David Airlie,
	Ingo Molnar, Geert Uytterhoeven, Mel Gorman, intel-gfx,
	Matt Turner, Valentin Schneider, linux-xtensa

On Mon, Sep 14, 2020 at 11:24 PM Herbert Xu <herbert@gondor.apana.org.au> wrote:
>
> On Tue, Sep 15, 2020 at 09:20:59AM +0300, Ard Biesheuvel wrote:
> >
> > The documentation of kmap_atomic() states the following:
> >
> >  * The use of kmap_atomic/kunmap_atomic is discouraged - kmap/kunmap
> >  * gives a more generic (and caching) interface. But kmap_atomic can
> >  * be used in IRQ contexts, so in some (very limited) cases we need
> >  * it.
> >
> > so if this is no longer accurate, perhaps we should fix it?
>
> This hasn't been accurate for at least ten years :)

Yeah, that used to be true a long long time ago, but the comment is very stale.

> > But another reason I tried to avoid kmap_atomic() is that it disables
> > preemption unconditionally, even on 64-bit architectures where HIGHMEM
> > is irrelevant. So using kmap_atomic() here means that the bulk of
> > WireGuard packet encryption runs with preemption disabled, essentially
> > for legacy reasons.
>
> Agreed.  We should definitely fix that.

Well, honestly, one big reason for that is debugging.

The *semantics* of the kmap_atomic() is in the name - you can't sleep
in between it and the kunmap_atomic().

On any sane architecture, kmap_atomic() ends up being a no-op from an
implementation standpoint, and sleeping would work just fine.

But we very much want to make sure that people don't then write code
that doesn't work on the bad old 32-bit machines where it really needs
that sequence to be safe from preemption.

So it's mostly a debug thing.

I say "mostly", because there might be small other details too, like
shared code, and perhaps even a couple of users out in the wild that
depend on the pagefault_disable() inherent in the current
kmap_atomic(), who knows..

So no, the preemption disabling isn't inherent in the operation
itself. But it does have some argument for it.

                   Linus

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [PATCH] crypto: lib/chacha20poly1305 - Set SG_MITER_ATOMIC unconditionally
  2020-09-15  6:03           ` Ard Biesheuvel
@ 2020-09-15  6:40             ` Herbert Xu
  0 siblings, 0 replies; 282+ messages in thread
From: Herbert Xu @ 2020-09-15  6:40 UTC (permalink / raw)
  To: Ard Biesheuvel
  Cc: Jason A. Donenfeld, Linus Torvalds, Thomas Gleixner, LKML,
	Linux Crypto Mailing List

On Tue, Sep 15, 2020 at 09:03:46AM +0300, Ard Biesheuvel wrote:
>
> The problem is really that kmap_atomic() is not simply a no-op on
> !HIGHMEM architectures. If we can fix that, I have no objections to
> this patch.

Yes we should definitely fix that.  However, doing so will involve
manually checking every instance of kmap_atomic.

Cheers,
-- 
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [PATCH] crypto: lib/chacha20poly1305 - Set SG_MITER_ATOMIC unconditionally
  2020-09-15  3:30         ` Herbert Xu
  2020-09-15  6:03           ` Ard Biesheuvel
@ 2020-09-15  6:45           ` Linus Torvalds
  2020-09-15  6:55             ` Linus Torvalds
  1 sibling, 1 reply; 282+ messages in thread
From: Linus Torvalds @ 2020-09-15  6:45 UTC (permalink / raw)
  To: Herbert Xu
  Cc: Thomas Gleixner, Ard Biesheuvel, LKML, Linux Crypto Mailing List

On Mon, Sep 14, 2020 at 8:30 PM Herbert Xu <herbert@gondor.apana.org.au> wrote:
>
> There is no reason for the chacha20poly1305 SG miter code to use
> kmap instead of kmap_atomic as the critical section doesn't sleep
> anyway.  So we can simply get rid of the preemptible check and
> set SG_MITER_ATOMIC unconditionally.

So I'd prefer to make SG_MITER_ATOMIC go away entirely, and just
remove the non-atomic case..

A quick grep seems to imply that just about all users set the ATOMIC
bit anyway. I didn't look at everything, but every case I _did_ look
at did seem to set the ATOMIC bit.

So it really did seem like there isn't a lot of reason to have the
non-atomic case, and this flag could go away - not by virtue of the
atomic case going away, but by virtue of the atomic case being the
only actual case.

I mean, I did find one case that didn't set it (cb710-mmc.c), but
pattern-matching to the other mmc cases, that one looks like it
_should_ have set the atomic flag like everybody else did.

Did I miss something?

             Linus

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [PATCH] crypto: lib/chacha20poly1305 - Set SG_MITER_ATOMIC unconditionally
  2020-09-15  6:45           ` Linus Torvalds
@ 2020-09-15  6:55             ` Linus Torvalds
  2020-09-15  7:05               ` Herbert Xu
  2020-09-15  7:08               ` Ard Biesheuvel
  0 siblings, 2 replies; 282+ messages in thread
From: Linus Torvalds @ 2020-09-15  6:55 UTC (permalink / raw)
  To: Herbert Xu
  Cc: Thomas Gleixner, Ard Biesheuvel, LKML, Linux Crypto Mailing List

On Mon, Sep 14, 2020 at 11:45 PM Linus Torvalds
<torvalds@linux-foundation.org> wrote:
>
> I mean, I did find one case that didn't set it (cb710-mmc.c), but
> pattern-matching to the other mmc cases, that one looks like it
> _should_ have set the atomic flag like everybody else did.

Oh, and immediately after sending that out I notice
nvmet_bdev_execute_rw(), which does seem to make allocations inside
that sg_miter loop.

So those non-atomic cases do clearly exist.

It does make the case for why kmap_atomic() wants to have the
debugging code. It will "just work" on 64-bit to do it wrong, because
the address doesn't become invalid just because you sleep or get
rescheduled. But then the code that every developer tests (since
developers tend to run on good hardware) might be completely broken on
bad old hardware.

Maybe we could hide it behind a debug option, at least.

Or, alterantively, introduce a new "debug_preempt_count" that doesn't
actually disable preemption, but warns about actual sleeping
operations..

             Linus

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [PATCH] crypto: lib/chacha20poly1305 - Set SG_MITER_ATOMIC unconditionally
  2020-09-15  6:55             ` Linus Torvalds
@ 2020-09-15  7:05               ` Herbert Xu
  2020-09-15  7:10                 ` Ard Biesheuvel
  2020-09-15  9:34                 ` Thomas Gleixner
  2020-09-15  7:08               ` Ard Biesheuvel
  1 sibling, 2 replies; 282+ messages in thread
From: Herbert Xu @ 2020-09-15  7:05 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Thomas Gleixner, Ard Biesheuvel, LKML, Linux Crypto Mailing List

On Mon, Sep 14, 2020 at 11:55:53PM -0700, Linus Torvalds wrote:
>
> Maybe we could hide it behind a debug option, at least.
> 
> Or, alterantively, introduce a new "debug_preempt_count" that doesn't
> actually disable preemption, but warns about actual sleeping
> operations..

I'm more worried about existing users of kmap_atomic relying on
the preemption disabling semantics.  Short of someone checking
on every single instance (and that would include derived cases
such as all users of sg miter), I think the safer option is to
create something brand new and then migrate the existing users
to it.  Something like

static inline void *kmap_atomic_ifhigh(struct page *page)
{
	if (PageHighMem(page))
		return kmap_atomic(page);
	return page_address(page);
}

static inline void kunmap_atomic_ifhigh(struct page *page, void *addr)
{
	if (PageHighMem(page))
		kunmap_atomic(addr);
}

Cheers,
-- 
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [PATCH] crypto: lib/chacha20poly1305 - Set SG_MITER_ATOMIC unconditionally
  2020-09-15  6:55             ` Linus Torvalds
  2020-09-15  7:05               ` Herbert Xu
@ 2020-09-15  7:08               ` Ard Biesheuvel
  1 sibling, 0 replies; 282+ messages in thread
From: Ard Biesheuvel @ 2020-09-15  7:08 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Herbert Xu, Thomas Gleixner, LKML, Linux Crypto Mailing List

On Tue, 15 Sep 2020 at 09:56, Linus Torvalds
<torvalds@linux-foundation.org> wrote:
>
> On Mon, Sep 14, 2020 at 11:45 PM Linus Torvalds
> <torvalds@linux-foundation.org> wrote:
> >
> > I mean, I did find one case that didn't set it (cb710-mmc.c), but
> > pattern-matching to the other mmc cases, that one looks like it
> > _should_ have set the atomic flag like everybody else did.
>
> Oh, and immediately after sending that out I notice
> nvmet_bdev_execute_rw(), which does seem to make allocations inside
> that sg_miter loop.
>
> So those non-atomic cases do clearly exist.
>
> It does make the case for why kmap_atomic() wants to have the
> debugging code. It will "just work" on 64-bit to do it wrong, because
> the address doesn't become invalid just because you sleep or get
> rescheduled. But then the code that every developer tests (since
> developers tend to run on good hardware) might be completely broken on
> bad old hardware.
>

If we want code that is optimal on recent hardware, and yet still
correct on older 32-bit hardware, kmap() is definitely a better choice
here than kmap_atomic(), since it is a no-op on !HIGHMEM, and
tolerates sleeping on 32-bit. /That/ is why I wrote the code this way.

The problem is of course that kmap() itself might sleep.

So I would argue that the semantics in the name of kmap_atomic() are
not about the fact that it starts a non-preemptible section, but that
it can be *called from* a non-preemptible section. And starting a
non-preemptible section is unnecessary on !HIGHMEM, and should be
avoided if possible.

> Maybe we could hide it behind a debug option, at least.
>
> Or, alterantively, introduce a new "debug_preempt_count" that doesn't
> actually disable preemption, but warns about actual sleeping
> operations..
>
>              Linus

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [PATCH] crypto: lib/chacha20poly1305 - Set SG_MITER_ATOMIC unconditionally
  2020-09-15  7:05               ` Herbert Xu
@ 2020-09-15  7:10                 ` Ard Biesheuvel
  2020-09-15  9:34                 ` Thomas Gleixner
  1 sibling, 0 replies; 282+ messages in thread
From: Ard Biesheuvel @ 2020-09-15  7:10 UTC (permalink / raw)
  To: Herbert Xu
  Cc: Linus Torvalds, Thomas Gleixner, LKML, Linux Crypto Mailing List

On Tue, 15 Sep 2020 at 10:05, Herbert Xu <herbert@gondor.apana.org.au> wrote:
>
> On Mon, Sep 14, 2020 at 11:55:53PM -0700, Linus Torvalds wrote:
> >
> > Maybe we could hide it behind a debug option, at least.
> >
> > Or, alterantively, introduce a new "debug_preempt_count" that doesn't
> > actually disable preemption, but warns about actual sleeping
> > operations..
>
> I'm more worried about existing users of kmap_atomic relying on
> the preemption disabling semantics.  Short of someone checking
> on every single instance (and that would include derived cases
> such as all users of sg miter), I think the safer option is to
> create something brand new and then migrate the existing users
> to it.  Something like
>
> static inline void *kmap_atomic_ifhigh(struct page *page)
> {
>         if (PageHighMem(page))
>                 return kmap_atomic(page);
>         return page_address(page);
> }
>
> static inline void kunmap_atomic_ifhigh(struct page *page, void *addr)
> {
>         if (PageHighMem(page))
>                 kunmap_atomic(addr);
> }
>

But we would still need to check all users of SG miter before we could
move it to this interface.

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
  2020-09-15  6:39               ` Linus Torvalds
  (?)
@ 2020-09-15  7:24                 ` Thomas Gleixner
  -1 siblings, 0 replies; 282+ messages in thread
From: Thomas Gleixner @ 2020-09-15  7:24 UTC (permalink / raw)
  To: Linus Torvalds, Herbert Xu
  Cc: Ard Biesheuvel, LKML, linux-arch, Sebastian Andrzej Siewior,
	Valentin Schneider, Richard Henderson, Ivan Kokshaysky,
	Matt Turner, alpha, Jeff Dike, Richard Weinberger, Anton Ivanov,
	linux-um, Brian Cain, linux-hexagon, Geert Uytterhoeven,
	linux-m68k, Ingo Molnar, Peter Zijlstra, Juri Lelli,
	Vincent Guittot, Dietmar Eggemann, Steven Rostedt, Ben Segall,
	Mel Gorman, Daniel Bristot de Oliveira, Will Deacon,
	Andrew Morton, Linux-MM, Ingo Molnar, Russell King, Linux ARM,
	Chris Zankel, Max Filippov, linux-xtensa, Jani Nikula,
	Joonas Lahtinen, Rodrigo Vivi, David Airlie, Daniel Vetter,
	intel-gfx, dri-devel, Paul E. McKenney, Josh Triplett,
	Mathieu Desnoyers, Lai Jiangshan, Shuah Khan, rcu

On Mon, Sep 14 2020 at 23:39, Linus Torvalds wrote:
> On Mon, Sep 14, 2020 at 11:24 PM Herbert Xu <herbert@gondor.apana.org.au> wrote:
>> > But another reason I tried to avoid kmap_atomic() is that it disables
>> > preemption unconditionally, even on 64-bit architectures where HIGHMEM
>> > is irrelevant. So using kmap_atomic() here means that the bulk of
>> > WireGuard packet encryption runs with preemption disabled, essentially
>> > for legacy reasons.
>>
>> Agreed.  We should definitely fix that.
>
> Well, honestly, one big reason for that is debugging.
>
> The *semantics* of the kmap_atomic() is in the name - you can't sleep
> in between it and the kunmap_atomic().
>
> On any sane architecture, kmap_atomic() ends up being a no-op from an
> implementation standpoint, and sleeping would work just fine.
>
> But we very much want to make sure that people don't then write code
> that doesn't work on the bad old 32-bit machines where it really needs
> that sequence to be safe from preemption.

Alternatively we just make highmem a bit more expensive by making these
maps preemptible. RT is doing this for a long time and it's not that
horrible.

The approach is to keep track about the number of active maps in a task
and on an eventual context switch save them away in the task struct and
restore them when the task is scheduled back in.

Thanks,

        tglx



^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [Intel-gfx] [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-15  7:24                 ` Thomas Gleixner
  0 siblings, 0 replies; 282+ messages in thread
From: Thomas Gleixner @ 2020-09-15  7:24 UTC (permalink / raw)
  To: Linus Torvalds, Herbert Xu
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, Linux-MM, linux-hexagon,
	Will Deacon, Ingo Molnar, Anton Ivanov, linux-arch, Brian Cain,
	Richard Weinberger, Russell King, Ard Biesheuvel, David Airlie,
	Ingo Molnar, Geert Uytterhoeven, Mel Gorman, intel-gfx,
	Matt Turner, Valentin Schneider, linux-xtensa, Shuah Khan,
	Paul E. McKenney, Jeff Dike, linux-um, Josh Triplett,
	Steven Rostedt, rcu, linux-m68k, Ivan Kokshaysky,
	Dietmar Eggemann, Linux ARM, Richard Henderson, Chris Zankel,
	Max Filippov, LKML, alpha, Mathieu Desnoyers, Andrew Morton,
	Daniel Bristot de Oliveira

On Mon, Sep 14 2020 at 23:39, Linus Torvalds wrote:
> On Mon, Sep 14, 2020 at 11:24 PM Herbert Xu <herbert@gondor.apana.org.au> wrote:
>> > But another reason I tried to avoid kmap_atomic() is that it disables
>> > preemption unconditionally, even on 64-bit architectures where HIGHMEM
>> > is irrelevant. So using kmap_atomic() here means that the bulk of
>> > WireGuard packet encryption runs with preemption disabled, essentially
>> > for legacy reasons.
>>
>> Agreed.  We should definitely fix that.
>
> Well, honestly, one big reason for that is debugging.
>
> The *semantics* of the kmap_atomic() is in the name - you can't sleep
> in between it and the kunmap_atomic().
>
> On any sane architecture, kmap_atomic() ends up being a no-op from an
> implementation standpoint, and sleeping would work just fine.
>
> But we very much want to make sure that people don't then write code
> that doesn't work on the bad old 32-bit machines where it really needs
> that sequence to be safe from preemption.

Alternatively we just make highmem a bit more expensive by making these
maps preemptible. RT is doing this for a long time and it's not that
horrible.

The approach is to keep track about the number of active maps in a task
and on an eventual context switch save them away in the task struct and
restore them when the task is scheduled back in.

Thanks,

        tglx


_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-15  7:24                 ` Thomas Gleixner
  0 siblings, 0 replies; 282+ messages in thread
From: Thomas Gleixner @ 2020-09-15  7:24 UTC (permalink / raw)
  To: Linus Torvalds, Herbert Xu
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, Linux-MM, linux-hexagon,
	Will Deacon, Ingo Molnar, Anton Ivanov, linux-arch, Brian Cain,
	Richard Weinberger, Russell King, Ard Biesheuvel, David Airlie,
	Ingo Molnar, Geert Uytterhoeven, Mel Gorman, intel-gfx,
	Matt Turner, Valentin Schneider, linux-xtensa

On Mon, Sep 14 2020 at 23:39, Linus Torvalds wrote:
> On Mon, Sep 14, 2020 at 11:24 PM Herbert Xu <herbert@gondor.apana.org.au> wrote:
>> > But another reason I tried to avoid kmap_atomic() is that it disables
>> > preemption unconditionally, even on 64-bit architectures where HIGHMEM
>> > is irrelevant. So using kmap_atomic() here means that the bulk of
>> > WireGuard packet encryption runs with preemption disabled, essentially
>> > for legacy reasons.
>>
>> Agreed.  We should definitely fix that.
>
> Well, honestly, one big reason for that is debugging.
>
> The *semantics* of the kmap_atomic() is in the name - you can't sleep
> in between it and the kunmap_atomic().
>
> On any sane architecture, kmap_atomic() ends up being a no-op from an
> implementation standpoint, and sleeping would work just fine.
>
> But we very much want to make sure that people don't then write code
> that doesn't work on the bad old 32-bit machines where it really needs
> that sequence to be safe from preemption.

Alternatively we just make highmem a bit more expensive by making these
maps preemptible. RT is doing this for a long time and it's not that
horrible.

The approach is to keep track about the number of active maps in a task
and on an eventual context switch save them away in the task struct and
restore them when the task is scheduled back in.

Thanks,

        tglx

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
  2020-09-14 22:24       ` Linus Torvalds
                           ` (2 preceding siblings ...)
  (?)
@ 2020-09-15  8:39         ` Thomas Gleixner
  -1 siblings, 0 replies; 282+ messages in thread
From: Thomas Gleixner @ 2020-09-15  8:39 UTC (permalink / raw)
  To: Linus Torvalds, Ard Biesheuvel, Herbert Xu
  Cc: LKML, linux-arch, Sebastian Andrzej Siewior, Valentin Schneider,
	Richard Henderson, Ivan Kokshaysky, Matt Turner, alpha,
	Jeff Dike, Richard Weinberger, Anton Ivanov, linux-um,
	Brian Cain, linux-hexagon, Geert Uytterhoeven, linux-m68k,
	Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
	Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman,
	Daniel Bristot de Oliveira, Will Deacon, Andrew Morton, Linux-MM,
	Ingo Molnar, Russell King, Linux ARM, Chris Zankel, Max Filippov,
	linux-xtensa, Jani Nikula, Joonas Lahtinen, Rodrigo Vivi,
	David Airlie, Daniel Vetter, intel-gfx, dri-devel,
	Paul E. McKenney, Josh Triplett, Mathieu Desnoyers,
	Lai Jiangshan, Shuah Khan, rcu,
	open list:KERNEL SELFTEST FRAMEWORK

On Mon, Sep 14 2020 at 15:24, Linus Torvalds wrote:
> On Mon, Sep 14, 2020 at 2:55 PM Thomas Gleixner <tglx@linutronix.de> wrote:
>>
>> Yes it does generate better code, but I tried hard to spot a difference
>> in various metrics exposed by perf. It's all in the noise and I only
>> can spot a difference when the actual preemption check after the
>> decrement
>
> I'm somewhat more worried about the small-device case.

I just checked on one of my old UP ARM toys which I run at home. The .text
increase is about 2% (75k) and none of the tests I ran showed any
significant difference. Couldn't verify with perf though as the PMU on
that piece of art is unusable.

> That said, the diffstat certainly has its very clear charm, and I do
> agree that it makes things simpler.
>
> I'm just not convinced people should ever EVER do things like that "if
> (preemptible())" garbage. It sounds like somebody is doing seriously
> bad things.

OTOH, having a working 'preemptible()' or maybe better named
'can_schedule()' check makes tons of sense to make decisions about
allocation modes or other things.

We're currently looking through all of in_atomic(), in_interrupt()
etc. usage sites and quite some of them are historic and have the clear
intent of checking whether the code is called from task context or
hard/softirq context. Lots of them are completely broken or just work by
chance.

But there is clearly historic precendence that context checks are
useful, but they only can be useful if we have a consistent mechanism
which works everywhere.

Of course we could mandate that every interface which might be called
from one or the other context has a context argument or provides two
variants of the same thing. But I'm not really convinced whether that's
a win over having a consistent and reliable set of checks.

Thanks,

        tglx





^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-15  8:39         ` Thomas Gleixner
  0 siblings, 0 replies; 282+ messages in thread
From: Thomas Gleixner @ 2020-09-15  8:39 UTC (permalink / raw)
  To: Linus Torvalds, Ard Biesheuvel, Herbert Xu
  Cc: LKML, linux-arch, Sebastian Andrzej Siewior, Valentin Schneider,
	Richard Henderson, Ivan Kokshaysky, Matt Turner, alpha,
	Jeff Dike, Richard Weinberger, Anton Ivanov, linux-um,
	Brian Cain, linux-hexagon, Geert Uytterhoeven, linux-m68k,
	Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
	Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman,
	Daniel Bristot de Oliveira, Will Deacon, Andrew Morton, Linux-MM,
	Ingo Molnar, Russell King, Linux ARM, Chris Zankel, Max Filippov,
	linux-xtensa, Jani Nikula, Joonas Lahtinen, Rodrigo Vivi,
	David Airlie, Daniel Vetter, intel-gfx, dri-devel,
	Paul E. McKenney, Josh Triplett, Mathieu Desnoyers,
	Lai Jiangshan, Shuah Khan, rcu,
	open list:KERNEL SELFTEST FRAMEWORK

On Mon, Sep 14 2020 at 15:24, Linus Torvalds wrote:
> On Mon, Sep 14, 2020 at 2:55 PM Thomas Gleixner <tglx@linutronix.de> wrote:
>>
>> Yes it does generate better code, but I tried hard to spot a difference
>> in various metrics exposed by perf. It's all in the noise and I only
>> can spot a difference when the actual preemption check after the
>> decrement
>
> I'm somewhat more worried about the small-device case.

I just checked on one of my old UP ARM toys which I run at home. The .text
increase is about 2% (75k) and none of the tests I ran showed any
significant difference. Couldn't verify with perf though as the PMU on
that piece of art is unusable.

> That said, the diffstat certainly has its very clear charm, and I do
> agree that it makes things simpler.
>
> I'm just not convinced people should ever EVER do things like that "if
> (preemptible())" garbage. It sounds like somebody is doing seriously
> bad things.

OTOH, having a working 'preemptible()' or maybe better named
'can_schedule()' check makes tons of sense to make decisions about
allocation modes or other things.

We're currently looking through all of in_atomic(), in_interrupt()
etc. usage sites and quite some of them are historic and have the clear
intent of checking whether the code is called from task context or
hard/softirq context. Lots of them are completely broken or just work by
chance.

But there is clearly historic precendence that context checks are
useful, but they only can be useful if we have a consistent mechanism
which works everywhere.

Of course we could mandate that every interface which might be called
from one or the other context has a context argument or provides two
variants of the same thing. But I'm not really convinced whether that's
a win over having a consistent and reliable set of checks.

Thanks,

        tglx






^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [Intel-gfx] [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-15  8:39         ` Thomas Gleixner
  0 siblings, 0 replies; 282+ messages in thread
From: Thomas Gleixner @ 2020-09-15  8:39 UTC (permalink / raw)
  To: Linus Torvalds, Ard Biesheuvel, Herbert Xu
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, Linux-MM,
	open list:KERNEL SELFTEST FRAMEWORK, linux-hexagon, Will Deacon,
	Ingo Molnar, Anton Ivanov, linux-arch, Brian Cain,
	Richard Weinberger, Russell King, David Airlie, Ingo Molnar,
	Geert Uytterhoeven, Mel Gorman, intel-gfx, Matt Turner,
	Valentin Schneider, linux-xtensa, Shuah Khan, Paul E. McKenney,
	Jeff Dike, linux-um, Josh Triplett, Steven Rostedt, rcu,
	linux-m68k, Ivan Kokshaysky, Dietmar Eggemann, Linux ARM,
	Richard Henderson, Chris Zankel, Max Filippov, LKML, alpha,
	Mathieu Desnoyers, Andrew Morton, Daniel Bristot de Oliveira

On Mon, Sep 14 2020 at 15:24, Linus Torvalds wrote:
> On Mon, Sep 14, 2020 at 2:55 PM Thomas Gleixner <tglx@linutronix.de> wrote:
>>
>> Yes it does generate better code, but I tried hard to spot a difference
>> in various metrics exposed by perf. It's all in the noise and I only
>> can spot a difference when the actual preemption check after the
>> decrement
>
> I'm somewhat more worried about the small-device case.

I just checked on one of my old UP ARM toys which I run at home. The .text
increase is about 2% (75k) and none of the tests I ran showed any
significant difference. Couldn't verify with perf though as the PMU on
that piece of art is unusable.

> That said, the diffstat certainly has its very clear charm, and I do
> agree that it makes things simpler.
>
> I'm just not convinced people should ever EVER do things like that "if
> (preemptible())" garbage. It sounds like somebody is doing seriously
> bad things.

OTOH, having a working 'preemptible()' or maybe better named
'can_schedule()' check makes tons of sense to make decisions about
allocation modes or other things.

We're currently looking through all of in_atomic(), in_interrupt()
etc. usage sites and quite some of them are historic and have the clear
intent of checking whether the code is called from task context or
hard/softirq context. Lots of them are completely broken or just work by
chance.

But there is clearly historic precendence that context checks are
useful, but they only can be useful if we have a consistent mechanism
which works everywhere.

Of course we could mandate that every interface which might be called
from one or the other context has a context argument or provides two
variants of the same thing. But I'm not really convinced whether that's
a win over having a consistent and reliable set of checks.

Thanks,

        tglx




_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-15  8:39         ` Thomas Gleixner
  0 siblings, 0 replies; 282+ messages in thread
From: Thomas Gleixner @ 2020-09-15  8:39 UTC (permalink / raw)
  To: Linus Torvalds, Ard Biesheuvel, Herbert Xu
  Cc: LKML, linux-arch, Sebastian Andrzej Siewior, Valentin Schneider,
	Richard Henderson, Ivan Kokshaysky, Matt Turner, alpha,
	Jeff Dike, Richard Weinberger, Anton Ivanov, linux-um,
	Brian Cain, linux-hexagon, Geert Uytterhoeven, linux-m68k,
	Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
	Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman,
	Daniel Bristot de Oliveira, Will Deacon, Andrew Morton, Linux-MM,
	Ingo Molnar, Russell King, Linux ARM, Chris Zankel, Max Filippov,
	linux-xtensa, Jani Nikula, Joonas Lahtinen, Rodrigo Vivi,
	David Airlie, Daniel Vetter, intel-gfx, dri-devel,
	Paul E. McKenney, Josh Triplett, Mathieu Desnoyers,
	Lai Jiangshan, Shuah Khan, rcu,
	open list:KERNEL SELFTEST FRAMEWORK

On Mon, Sep 14 2020 at 15:24, Linus Torvalds wrote:
> On Mon, Sep 14, 2020 at 2:55 PM Thomas Gleixner <tglx@linutronix.de> wrote:
>>
>> Yes it does generate better code, but I tried hard to spot a difference
>> in various metrics exposed by perf. It's all in the noise and I only
>> can spot a difference when the actual preemption check after the
>> decrement
>
> I'm somewhat more worried about the small-device case.

I just checked on one of my old UP ARM toys which I run at home. The .text
increase is about 2% (75k) and none of the tests I ran showed any
significant difference. Couldn't verify with perf though as the PMU on
that piece of art is unusable.

> That said, the diffstat certainly has its very clear charm, and I do
> agree that it makes things simpler.
>
> I'm just not convinced people should ever EVER do things like that "if
> (preemptible())" garbage. It sounds like somebody is doing seriously
> bad things.

OTOH, having a working 'preemptible()' or maybe better named
'can_schedule()' check makes tons of sense to make decisions about
allocation modes or other things.

We're currently looking through all of in_atomic(), in_interrupt()
etc. usage sites and quite some of them are historic and have the clear
intent of checking whether the code is called from task context or
hard/softirq context. Lots of them are completely broken or just work by
chance.

But there is clearly historic precendence that context checks are
useful, but they only can be useful if we have a consistent mechanism
which works everywhere.

Of course we could mandate that every interface which might be called
from one or the other context has a context argument or provides two
variants of the same thing. But I'm not really convinced whether that's
a win over having a consistent and reliable set of checks.

Thanks,

        tglx






^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-15  8:39         ` Thomas Gleixner
  0 siblings, 0 replies; 282+ messages in thread
From: Thomas Gleixner @ 2020-09-15  8:39 UTC (permalink / raw)
  To: Linus Torvalds, Ard Biesheuvel, Herbert Xu
  Cc: LKML, linux-arch, Sebastian Andrzej Siewior, Valentin Schneider,
	Richard Henderson, Ivan Kokshaysky, Matt Turner, alpha,
	Jeff Dike, Richard Weinberger, Anton Ivanov, linux-um,
	Brian Cain, linux-hexagon, Geert Uytterhoeven, linux-m68k,
	Ingo Molnar, Peter Zijlstra, Juri Lelli

On Mon, Sep 14 2020 at 15:24, Linus Torvalds wrote:
> On Mon, Sep 14, 2020 at 2:55 PM Thomas Gleixner <tglx@linutronix.de> wrote:
>>
>> Yes it does generate better code, but I tried hard to spot a difference
>> in various metrics exposed by perf. It's all in the noise and I only
>> can spot a difference when the actual preemption check after the
>> decrement
>
> I'm somewhat more worried about the small-device case.

I just checked on one of my old UP ARM toys which I run at home. The .text
increase is about 2% (75k) and none of the tests I ran showed any
significant difference. Couldn't verify with perf though as the PMU on
that piece of art is unusable.

> That said, the diffstat certainly has its very clear charm, and I do
> agree that it makes things simpler.
>
> I'm just not convinced people should ever EVER do things like that "if
> (preemptible())" garbage. It sounds like somebody is doing seriously
> bad things.

OTOH, having a working 'preemptible()' or maybe better named
'can_schedule()' check makes tons of sense to make decisions about
allocation modes or other things.

We're currently looking through all of in_atomic(), in_interrupt()
etc. usage sites and quite some of them are historic and have the clear
intent of checking whether the code is called from task context or
hard/softirq context. Lots of them are completely broken or just work by
chance.

But there is clearly historic precendence that context checks are
useful, but they only can be useful if we have a consistent mechanism
which works everywhere.

Of course we could mandate that every interface which might be called
from one or the other context has a context argument or provides two
variants of the same thing. But I'm not really convinced whether that's
a win over having a consistent and reliable set of checks.

Thanks,

        tglx





^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [PATCH] crypto: lib/chacha20poly1305 - Set SG_MITER_ATOMIC unconditionally
  2020-09-15  7:05               ` Herbert Xu
  2020-09-15  7:10                 ` Ard Biesheuvel
@ 2020-09-15  9:34                 ` Thomas Gleixner
  2020-09-15 10:02                   ` Ard Biesheuvel
  1 sibling, 1 reply; 282+ messages in thread
From: Thomas Gleixner @ 2020-09-15  9:34 UTC (permalink / raw)
  To: Herbert Xu, Linus Torvalds
  Cc: Ard Biesheuvel, LKML, Linux Crypto Mailing List

On Tue, Sep 15 2020 at 17:05, Herbert Xu wrote:
> On Mon, Sep 14, 2020 at 11:55:53PM -0700, Linus Torvalds wrote:
>>
>> Maybe we could hide it behind a debug option, at least.
>> 
>> Or, alterantively, introduce a new "debug_preempt_count" that doesn't
>> actually disable preemption, but warns about actual sleeping
>> operations..
>
> I'm more worried about existing users of kmap_atomic relying on
> the preemption disabling semantics.  Short of someone checking
> on every single instance (and that would include derived cases
> such as all users of sg miter), I think the safer option is to
> create something brand new and then migrate the existing users
> to it.  Something like
>
> static inline void *kmap_atomic_ifhigh(struct page *page)
> {
> 	if (PageHighMem(page))
> 		return kmap_atomic(page);
> 	return page_address(page);
> }
>
> static inline void kunmap_atomic_ifhigh(struct page *page, void *addr)
> {
> 	if (PageHighMem(page))
> 		kunmap_atomic(addr);
> }

Hmm, that still has the issue that the code between map and unmap must
not sleep and the conversion must carefully check whether anything in
this region relies on preemption being disabled by kmap_atomic()
regardless of highmem or not.

kmap_atomic() is at least consistent vs. preemption, the above not so
much.

I'd rather go for a preemptible/sleepable version of highmem mapping
which is in itself consistent for both highmen and not highmem.

Thanks,

        tglx



^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [PATCH] crypto: lib/chacha20poly1305 - Set SG_MITER_ATOMIC unconditionally
  2020-09-15  9:34                 ` Thomas Gleixner
@ 2020-09-15 10:02                   ` Ard Biesheuvel
  2020-09-15 10:05                     ` Herbert Xu
  0 siblings, 1 reply; 282+ messages in thread
From: Ard Biesheuvel @ 2020-09-15 10:02 UTC (permalink / raw)
  To: Thomas Gleixner
  Cc: Herbert Xu, Linus Torvalds, LKML, Linux Crypto Mailing List

On Tue, 15 Sep 2020 at 12:34, Thomas Gleixner <tglx@linutronix.de> wrote:
>
> On Tue, Sep 15 2020 at 17:05, Herbert Xu wrote:
> > On Mon, Sep 14, 2020 at 11:55:53PM -0700, Linus Torvalds wrote:
> >>
> >> Maybe we could hide it behind a debug option, at least.
> >>
> >> Or, alterantively, introduce a new "debug_preempt_count" that doesn't
> >> actually disable preemption, but warns about actual sleeping
> >> operations..
> >
> > I'm more worried about existing users of kmap_atomic relying on
> > the preemption disabling semantics.  Short of someone checking
> > on every single instance (and that would include derived cases
> > such as all users of sg miter), I think the safer option is to
> > create something brand new and then migrate the existing users
> > to it.  Something like
> >
> > static inline void *kmap_atomic_ifhigh(struct page *page)
> > {
> >       if (PageHighMem(page))
> >               return kmap_atomic(page);
> >       return page_address(page);
> > }
> >
> > static inline void kunmap_atomic_ifhigh(struct page *page, void *addr)
> > {
> >       if (PageHighMem(page))
> >               kunmap_atomic(addr);
> > }
>
> Hmm, that still has the issue that the code between map and unmap must
> not sleep and the conversion must carefully check whether anything in
> this region relies on preemption being disabled by kmap_atomic()
> regardless of highmem or not.
>
> kmap_atomic() is at least consistent vs. preemption, the above not so
> much.
>

But that is really the point. I don't *want* to be forced to disable
preemption in brand new code simply because some legacy highmem API
conflates being callable from atomic context with instantiating an
atomic context by disabling preemption for no good reason. IIUC, in
the past, you would really only call kmap_atomic() if you absolutely
had to, and so you would never rely on the preemption disabling
semantics accidentally. By making kmap_atomic() the preferred API even
for calls from non-atomic contexts, this line has blurred and we no
longer know why individual kmap_atomic() occurrences exist in the
first place.

> I'd rather go for a preemptible/sleepable version of highmem mapping
> which is in itself consistent for both highmen and not highmem.
>

I don't think we need to obsess about highmem, although we should
obviously take care not to regress its performance unnecessarily. What
I want to avoid is to burden a brand new subsystem with legacy highmem
baggage simply because we could not agree on how to avoid that.

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [PATCH] crypto: lib/chacha20poly1305 - Set SG_MITER_ATOMIC unconditionally
  2020-09-15 10:02                   ` Ard Biesheuvel
@ 2020-09-15 10:05                     ` Herbert Xu
  2020-09-15 10:08                       ` Ard Biesheuvel
  0 siblings, 1 reply; 282+ messages in thread
From: Herbert Xu @ 2020-09-15 10:05 UTC (permalink / raw)
  To: Ard Biesheuvel
  Cc: Thomas Gleixner, Linus Torvalds, LKML, Linux Crypto Mailing List

On Tue, Sep 15, 2020 at 01:02:10PM +0300, Ard Biesheuvel wrote:
>
> > I'd rather go for a preemptible/sleepable version of highmem mapping
> > which is in itself consistent for both highmen and not highmem.
> 
> I don't think we need to obsess about highmem, although we should
> obviously take care not to regress its performance unnecessarily. What
> I want to avoid is to burden a brand new subsystem with legacy highmem
> baggage simply because we could not agree on how to avoid that.

I think what Thomas is proposing should address your concerns Ard.
As long as nobody objects to the slight performance degradation on
legacy highmem platforms it should make kmap_atomic just go away on
modern platforms.

Cheers,
-- 
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [PATCH] crypto: lib/chacha20poly1305 - Set SG_MITER_ATOMIC unconditionally
  2020-09-15 10:05                     ` Herbert Xu
@ 2020-09-15 10:08                       ` Ard Biesheuvel
  2020-09-15 10:10                         ` Herbert Xu
  0 siblings, 1 reply; 282+ messages in thread
From: Ard Biesheuvel @ 2020-09-15 10:08 UTC (permalink / raw)
  To: Herbert Xu
  Cc: Thomas Gleixner, Linus Torvalds, LKML, Linux Crypto Mailing List

On Tue, 15 Sep 2020 at 13:05, Herbert Xu <herbert@gondor.apana.org.au> wrote:
>
> On Tue, Sep 15, 2020 at 01:02:10PM +0300, Ard Biesheuvel wrote:
> >
> > > I'd rather go for a preemptible/sleepable version of highmem mapping
> > > which is in itself consistent for both highmen and not highmem.
> >
> > I don't think we need to obsess about highmem, although we should
> > obviously take care not to regress its performance unnecessarily. What
> > I want to avoid is to burden a brand new subsystem with legacy highmem
> > baggage simply because we could not agree on how to avoid that.
>
> I think what Thomas is proposing should address your concerns Ard.
> As long as nobody objects to the slight performance degradation on
> legacy highmem platforms it should make kmap_atomic just go away on
> modern platforms.
>

But making atomic kmap preemptible/sleepable creates the exact same
problem, i.e., that we have no idea which existing callers are
currently relying on those preemption disabling semantics, so we can't
just take them away. Or am I missing something?

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [PATCH] crypto: lib/chacha20poly1305 - Set SG_MITER_ATOMIC unconditionally
  2020-09-15 10:08                       ` Ard Biesheuvel
@ 2020-09-15 10:10                         ` Herbert Xu
  2020-09-15 19:04                           ` Thomas Gleixner
  0 siblings, 1 reply; 282+ messages in thread
From: Herbert Xu @ 2020-09-15 10:10 UTC (permalink / raw)
  To: Ard Biesheuvel
  Cc: Thomas Gleixner, Linus Torvalds, LKML, Linux Crypto Mailing List

On Tue, Sep 15, 2020 at 01:08:31PM +0300, Ard Biesheuvel wrote:
>
> But making atomic kmap preemptible/sleepable creates the exact same
> problem, i.e., that we have no idea which existing callers are
> currently relying on those preemption disabling semantics, so we can't
> just take them away. Or am I missing something?

Good point.

Thomas mentioned that RT has been doing this for a while now so
perhaps someone has studied this problem already? Thomas?

Cheers,
-- 
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 06/13] locking/bitspinlock: Clenaup PREEMPT_COUNT leftovers
  2020-09-14 20:42   ` Thomas Gleixner
                       ` (2 preceding siblings ...)
  (?)
@ 2020-09-15 16:10     ` Will Deacon
  -1 siblings, 0 replies; 282+ messages in thread
From: Will Deacon @ 2020-09-15 16:10 UTC (permalink / raw)
  To: Thomas Gleixner
  Cc: LKML, linux-arch, Linus Torvalds, Sebastian Andrzej Siewior,
	Valentin Schneider, Richard Henderson, Ivan Kokshaysky,
	Matt Turner, linux-alpha, Jeff Dike, Richard Weinberger,
	Anton Ivanov, linux-um, Brian Cain, linux-hexagon,
	Geert Uytterhoeven, linux-m68k, Ingo Molnar, Peter Zijlstra,
	Juri Lelli, Vincent Guittot, Dietmar Eggemann, Steven Rostedt,
	Ben Segall, Mel Gorman, Daniel Bristot de Oliveira,
	Andrew Morton, linux-mm, Ingo Molnar, Russell King,
	linux-arm-kernel, Chris Zankel, Max Filippov, linux-xtensa,
	Jani Nikula, Joonas Lahtinen, Rodrigo Vivi, David Airlie,
	Daniel Vetter, intel-gfx, dri-devel, Paul E. McKenney,
	Josh Triplett, Mathieu Desnoyers, Lai Jiangshan, Shuah Khan, rcu,
	linux-kselftest

On Mon, Sep 14, 2020 at 10:42:15PM +0200, Thomas Gleixner wrote:
> CONFIG_PREEMPT_COUNT is now unconditionally enabled and will be
> removed. Cleanup the leftovers before doing so.
> 
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> ---
>  include/linux/bit_spinlock.h |    4 +---
>  1 file changed, 1 insertion(+), 3 deletions(-)
> 
> --- a/include/linux/bit_spinlock.h
> +++ b/include/linux/bit_spinlock.h
> @@ -90,10 +90,8 @@ static inline int bit_spin_is_locked(int
>  {
>  #if defined(CONFIG_SMP) || defined(CONFIG_DEBUG_SPINLOCK)
>  	return test_bit(bitnum, addr);
> -#elif defined CONFIG_PREEMPT_COUNT
> -	return preempt_count();
>  #else
> -	return 1;
> +	return preempt_count();
>  #endif

Acked-by: Will Deacon <will@kernel.org>

Will

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 06/13] locking/bitspinlock: Clenaup PREEMPT_COUNT leftovers
@ 2020-09-15 16:10     ` Will Deacon
  0 siblings, 0 replies; 282+ messages in thread
From: Will Deacon @ 2020-09-15 16:10 UTC (permalink / raw)
  To: Thomas Gleixner
  Cc: Juri Lelli, Peter Zijlstra, Linus Torvalds,
	Sebastian Andrzej Siewior, Joonas Lahtinen, Lai Jiangshan,
	dri-devel, Ben Segall, linux-mm, linux-kselftest, linux-hexagon,
	Shuah Khan, Ingo Molnar, Anton Ivanov, linux-arch,
	Vincent Guittot, Brian Cain, Richard Weinberger, Russell King,
	David Airlie, Ingo Molnar, Geert Uytterhoeven, Mel Gorman,
	intel-gfx, Matt Turner, Valentin Schneider, linux-xtensa,
	Paul E. McKenney, Jeff Dike, linux-um, Josh Triplett,
	Steven Rostedt, rcu, linux-m68k, Ivan Kokshaysky, Jani Nikula,
	Rodrigo Vivi, Dietmar Eggemann, linux-arm-kernel,
	Richard Henderson, Chris Zankel, Max Filippov, LKML,
	Daniel Vetter, linux-alpha, Mathieu Desnoyers, Andrew Morton,
	Daniel Bristot de Oliveira

On Mon, Sep 14, 2020 at 10:42:15PM +0200, Thomas Gleixner wrote:
> CONFIG_PREEMPT_COUNT is now unconditionally enabled and will be
> removed. Cleanup the leftovers before doing so.
> 
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> ---
>  include/linux/bit_spinlock.h |    4 +---
>  1 file changed, 1 insertion(+), 3 deletions(-)
> 
> --- a/include/linux/bit_spinlock.h
> +++ b/include/linux/bit_spinlock.h
> @@ -90,10 +90,8 @@ static inline int bit_spin_is_locked(int
>  {
>  #if defined(CONFIG_SMP) || defined(CONFIG_DEBUG_SPINLOCK)
>  	return test_bit(bitnum, addr);
> -#elif defined CONFIG_PREEMPT_COUNT
> -	return preempt_count();
>  #else
> -	return 1;
> +	return preempt_count();
>  #endif

Acked-by: Will Deacon <will@kernel.org>

Will

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 06/13] locking/bitspinlock: Clenaup PREEMPT_COUNT leftovers
@ 2020-09-15 16:10     ` Will Deacon
  0 siblings, 0 replies; 282+ messages in thread
From: Will Deacon @ 2020-09-15 16:10 UTC (permalink / raw)
  To: Thomas Gleixner
  Cc: Juri Lelli, Peter Zijlstra, Linus Torvalds,
	Sebastian Andrzej Siewior, Lai Jiangshan, dri-devel, Ben Segall,
	linux-mm, linux-kselftest, linux-hexagon, Shuah Khan,
	Ingo Molnar, Anton Ivanov, linux-arch, Vincent Guittot,
	Brian Cain, Richard Weinberger, Russell King, David Airlie,
	Ingo Molnar, Geert Uytterhoeven, Mel Gorman, intel-gfx,
	Matt Turner, Valentin Schneider, linux-xtensa, Paul E. McKenney,
	Jeff Dike, linux-um, Josh Triplett, Steven Rostedt, rcu,
	linux-m68k, Ivan Kokshaysky, Rodrigo Vivi, Dietmar Eggemann,
	linux-arm-kernel, Richard Henderson, Chris Zankel, Max Filippov,
	LKML, linux-alpha, Mathieu Desnoyers, Andrew Morton,
	Daniel Bristot de Oliveira

On Mon, Sep 14, 2020 at 10:42:15PM +0200, Thomas Gleixner wrote:
> CONFIG_PREEMPT_COUNT is now unconditionally enabled and will be
> removed. Cleanup the leftovers before doing so.
> 
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> ---
>  include/linux/bit_spinlock.h |    4 +---
>  1 file changed, 1 insertion(+), 3 deletions(-)
> 
> --- a/include/linux/bit_spinlock.h
> +++ b/include/linux/bit_spinlock.h
> @@ -90,10 +90,8 @@ static inline int bit_spin_is_locked(int
>  {
>  #if defined(CONFIG_SMP) || defined(CONFIG_DEBUG_SPINLOCK)
>  	return test_bit(bitnum, addr);
> -#elif defined CONFIG_PREEMPT_COUNT
> -	return preempt_count();
>  #else
> -	return 1;
> +	return preempt_count();
>  #endif

Acked-by: Will Deacon <will@kernel.org>

Will
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [Intel-gfx] [patch 06/13] locking/bitspinlock: Clenaup PREEMPT_COUNT leftovers
@ 2020-09-15 16:10     ` Will Deacon
  0 siblings, 0 replies; 282+ messages in thread
From: Will Deacon @ 2020-09-15 16:10 UTC (permalink / raw)
  To: Thomas Gleixner
  Cc: Juri Lelli, Peter Zijlstra, Linus Torvalds,
	Sebastian Andrzej Siewior, Lai Jiangshan, dri-devel, Ben Segall,
	linux-mm, linux-kselftest, linux-hexagon, Shuah Khan,
	Ingo Molnar, Anton Ivanov, linux-arch, Brian Cain,
	Richard Weinberger, Russell King, David Airlie, Ingo Molnar,
	Geert Uytterhoeven, Mel Gorman, intel-gfx, Matt Turner,
	Valentin Schneider, linux-xtensa, Paul E. McKenney, Jeff Dike,
	linux-um, Josh Triplett, Steven Rostedt, rcu, linux-m68k,
	Ivan Kokshaysky, Dietmar Eggemann, linux-arm-kernel,
	Richard Henderson, Chris Zankel, Max Filippov, LKML, linux-alpha,
	Mathieu Desnoyers, Andrew Morton, Daniel Bristot de Oliveira

On Mon, Sep 14, 2020 at 10:42:15PM +0200, Thomas Gleixner wrote:
> CONFIG_PREEMPT_COUNT is now unconditionally enabled and will be
> removed. Cleanup the leftovers before doing so.
> 
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> ---
>  include/linux/bit_spinlock.h |    4 +---
>  1 file changed, 1 insertion(+), 3 deletions(-)
> 
> --- a/include/linux/bit_spinlock.h
> +++ b/include/linux/bit_spinlock.h
> @@ -90,10 +90,8 @@ static inline int bit_spin_is_locked(int
>  {
>  #if defined(CONFIG_SMP) || defined(CONFIG_DEBUG_SPINLOCK)
>  	return test_bit(bitnum, addr);
> -#elif defined CONFIG_PREEMPT_COUNT
> -	return preempt_count();
>  #else
> -	return 1;
> +	return preempt_count();
>  #endif

Acked-by: Will Deacon <will@kernel.org>

Will
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 06/13] locking/bitspinlock: Clenaup PREEMPT_COUNT leftovers
@ 2020-09-15 16:10     ` Will Deacon
  0 siblings, 0 replies; 282+ messages in thread
From: Will Deacon @ 2020-09-15 16:10 UTC (permalink / raw)
  To: Thomas Gleixner
  Cc: Juri Lelli, Peter Zijlstra, Linus Torvalds,
	Sebastian Andrzej Siewior, Joonas Lahtinen, Lai Jiangshan,
	dri-devel, Ben Segall, linux-mm, linux-kselftest, linux-hexagon,
	Shuah Khan, Ingo Molnar, Anton Ivanov, linux-arch,
	Vincent Guittot, Brian Cain, Richard Weinberger, Russell King,
	David Airlie, Ingo Molnar, Geert Uytterhoeven, Mel Gorman,
	intel-gfx, Ma

On Mon, Sep 14, 2020 at 10:42:15PM +0200, Thomas Gleixner wrote:
> CONFIG_PREEMPT_COUNT is now unconditionally enabled and will be
> removed. Cleanup the leftovers before doing so.
> 
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> ---
>  include/linux/bit_spinlock.h |    4 +---
>  1 file changed, 1 insertion(+), 3 deletions(-)
> 
> --- a/include/linux/bit_spinlock.h
> +++ b/include/linux/bit_spinlock.h
> @@ -90,10 +90,8 @@ static inline int bit_spin_is_locked(int
>  {
>  #if defined(CONFIG_SMP) || defined(CONFIG_DEBUG_SPINLOCK)
>  	return test_bit(bitnum, addr);
> -#elif defined CONFIG_PREEMPT_COUNT
> -	return preempt_count();
>  #else
> -	return 1;
> +	return preempt_count();
>  #endif

Acked-by: Will Deacon <will@kernel.org>

Will

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 04/13] lockdep: Clenaup PREEMPT_COUNT leftovers
  2020-09-14 20:42   ` Thomas Gleixner
                       ` (2 preceding siblings ...)
  (?)
@ 2020-09-15 16:11     ` Will Deacon
  -1 siblings, 0 replies; 282+ messages in thread
From: Will Deacon @ 2020-09-15 16:11 UTC (permalink / raw)
  To: Thomas Gleixner
  Cc: LKML, linux-arch, Linus Torvalds, Sebastian Andrzej Siewior,
	Valentin Schneider, Peter Zijlstra, Ingo Molnar,
	Richard Henderson, Ivan Kokshaysky, Matt Turner, linux-alpha,
	Jeff Dike, Richard Weinberger, Anton Ivanov, linux-um,
	Brian Cain, linux-hexagon, Geert Uytterhoeven, linux-m68k,
	Juri Lelli, Vincent Guittot, Dietmar Eggemann, Steven Rostedt,
	Ben Segall, Mel Gorman, Daniel Bristot de Oliveira,
	Andrew Morton, linux-mm, Ingo Molnar, Russell King,
	linux-arm-kernel, Chris Zankel, Max Filippov, linux-xtensa,
	Jani Nikula, Joonas Lahtinen, Rodrigo Vivi, David Airlie,
	Daniel Vetter, intel-gfx, dri-devel, Paul E. McKenney,
	Josh Triplett, Mathieu Desnoyers, Lai Jiangshan, Shuah Khan, rcu,
	linux-kselftest

On Mon, Sep 14, 2020 at 10:42:13PM +0200, Thomas Gleixner wrote:
> CONFIG_PREEMPT_COUNT is now unconditionally enabled and will be
> removed. Cleanup the leftovers before doing so.
> 
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> Cc: Peter Zijlstra <peterz@infradead.org>
> Cc: Ingo Molnar <mingo@kernel.org>
> Cc: Will Deacon <will@kernel.org>
> ---
>  include/linux/lockdep.h |    6 ++----
>  lib/Kconfig.debug       |    1 -
>  2 files changed, 2 insertions(+), 5 deletions(-)
> 
> --- a/include/linux/lockdep.h
> +++ b/include/linux/lockdep.h
> @@ -585,16 +585,14 @@ do {									\
>  
>  #define lockdep_assert_preemption_enabled()				\
>  do {									\
> -	WARN_ON_ONCE(IS_ENABLED(CONFIG_PREEMPT_COUNT)	&&		\
> -		     debug_locks			&&		\
> +	WARN_ON_ONCE(debug_locks			&&		\
>  		     (preempt_count() != 0		||		\
>  		      !raw_cpu_read(hardirqs_enabled)));		\
>  } while (0)
>  
>  #define lockdep_assert_preemption_disabled()				\
>  do {									\
> -	WARN_ON_ONCE(IS_ENABLED(CONFIG_PREEMPT_COUNT)	&&		\
> -		     debug_locks			&&		\
> +	WARN_ON_ONCE(debug_locks			&&		\
>  		     (preempt_count() == 0		&&		\
>  		      raw_cpu_read(hardirqs_enabled)));			\
>  } while (0)
> --- a/lib/Kconfig.debug
> +++ b/lib/Kconfig.debug
> @@ -1161,7 +1161,6 @@ config PROVE_LOCKING
>  	select DEBUG_RWSEMS
>  	select DEBUG_WW_MUTEX_SLOWPATH
>  	select DEBUG_LOCK_ALLOC
> -	select PREEMPT_COUNT
>  	select TRACE_IRQFLAGS
>  	default n
>  	help

Acked-by: Will Deacon <will@kernel.org>

Will

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 04/13] lockdep: Clenaup PREEMPT_COUNT leftovers
@ 2020-09-15 16:11     ` Will Deacon
  0 siblings, 0 replies; 282+ messages in thread
From: Will Deacon @ 2020-09-15 16:11 UTC (permalink / raw)
  To: Thomas Gleixner
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Joonas Lahtinen, Lai Jiangshan, dri-devel, Ben Segall, linux-mm,
	linux-kselftest, linux-hexagon, Shuah Khan, Ingo Molnar,
	Anton Ivanov, linux-arch, Linus Torvalds, Brian Cain,
	Richard Weinberger, Russell King, David Airlie, Ingo Molnar,
	Geert Uytterhoeven, Mel Gorman, intel-gfx, Matt Turner,
	Valentin Schneider, linux-xtensa, Paul E. McKenney, Jeff Dike,
	linux-um, Josh Triplett, Steven Rostedt, rcu, linux-m68k,
	Ivan Kokshaysky, Jani Nikula, Rodrigo Vivi, Vincent Guittot,
	Dietmar Eggemann, linux-arm-kernel, Richard Henderson,
	Chris Zankel, Max Filippov, LKML, Daniel Vetter, linux-alpha,
	Mathieu Desnoyers, Andrew Morton, Daniel Bristot de Oliveira

On Mon, Sep 14, 2020 at 10:42:13PM +0200, Thomas Gleixner wrote:
> CONFIG_PREEMPT_COUNT is now unconditionally enabled and will be
> removed. Cleanup the leftovers before doing so.
> 
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> Cc: Peter Zijlstra <peterz@infradead.org>
> Cc: Ingo Molnar <mingo@kernel.org>
> Cc: Will Deacon <will@kernel.org>
> ---
>  include/linux/lockdep.h |    6 ++----
>  lib/Kconfig.debug       |    1 -
>  2 files changed, 2 insertions(+), 5 deletions(-)
> 
> --- a/include/linux/lockdep.h
> +++ b/include/linux/lockdep.h
> @@ -585,16 +585,14 @@ do {									\
>  
>  #define lockdep_assert_preemption_enabled()				\
>  do {									\
> -	WARN_ON_ONCE(IS_ENABLED(CONFIG_PREEMPT_COUNT)	&&		\
> -		     debug_locks			&&		\
> +	WARN_ON_ONCE(debug_locks			&&		\
>  		     (preempt_count() != 0		||		\
>  		      !raw_cpu_read(hardirqs_enabled)));		\
>  } while (0)
>  
>  #define lockdep_assert_preemption_disabled()				\
>  do {									\
> -	WARN_ON_ONCE(IS_ENABLED(CONFIG_PREEMPT_COUNT)	&&		\
> -		     debug_locks			&&		\
> +	WARN_ON_ONCE(debug_locks			&&		\
>  		     (preempt_count() == 0		&&		\
>  		      raw_cpu_read(hardirqs_enabled)));			\
>  } while (0)
> --- a/lib/Kconfig.debug
> +++ b/lib/Kconfig.debug
> @@ -1161,7 +1161,6 @@ config PROVE_LOCKING
>  	select DEBUG_RWSEMS
>  	select DEBUG_WW_MUTEX_SLOWPATH
>  	select DEBUG_LOCK_ALLOC
> -	select PREEMPT_COUNT
>  	select TRACE_IRQFLAGS
>  	default n
>  	help

Acked-by: Will Deacon <will@kernel.org>

Will

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 04/13] lockdep: Clenaup PREEMPT_COUNT leftovers
@ 2020-09-15 16:11     ` Will Deacon
  0 siblings, 0 replies; 282+ messages in thread
From: Will Deacon @ 2020-09-15 16:11 UTC (permalink / raw)
  To: Thomas Gleixner
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, linux-mm, linux-kselftest,
	linux-hexagon, Shuah Khan, Ingo Molnar, Anton Ivanov, linux-arch,
	Linus Torvalds, Brian Cain, Richard Weinberger, Russell King,
	David Airlie, Ingo Molnar, Geert Uytterhoeven, Mel Gorman,
	intel-gfx, Matt Turner, Valentin Schneider, linux-xtensa,
	Paul E. McKenney, Jeff Dike, linux-um, Josh Triplett,
	Steven Rostedt, rcu, linux-m68k, Ivan Kokshaysky, Rodrigo Vivi,
	Vincent Guittot, Dietmar Eggemann, linux-arm-kernel,
	Richard Henderson, Chris Zankel, Max Filippov, LKML, linux-alpha,
	Mathieu Desnoyers, Andrew Morton, Daniel Bristot de Oliveira

On Mon, Sep 14, 2020 at 10:42:13PM +0200, Thomas Gleixner wrote:
> CONFIG_PREEMPT_COUNT is now unconditionally enabled and will be
> removed. Cleanup the leftovers before doing so.
> 
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> Cc: Peter Zijlstra <peterz@infradead.org>
> Cc: Ingo Molnar <mingo@kernel.org>
> Cc: Will Deacon <will@kernel.org>
> ---
>  include/linux/lockdep.h |    6 ++----
>  lib/Kconfig.debug       |    1 -
>  2 files changed, 2 insertions(+), 5 deletions(-)
> 
> --- a/include/linux/lockdep.h
> +++ b/include/linux/lockdep.h
> @@ -585,16 +585,14 @@ do {									\
>  
>  #define lockdep_assert_preemption_enabled()				\
>  do {									\
> -	WARN_ON_ONCE(IS_ENABLED(CONFIG_PREEMPT_COUNT)	&&		\
> -		     debug_locks			&&		\
> +	WARN_ON_ONCE(debug_locks			&&		\
>  		     (preempt_count() != 0		||		\
>  		      !raw_cpu_read(hardirqs_enabled)));		\
>  } while (0)
>  
>  #define lockdep_assert_preemption_disabled()				\
>  do {									\
> -	WARN_ON_ONCE(IS_ENABLED(CONFIG_PREEMPT_COUNT)	&&		\
> -		     debug_locks			&&		\
> +	WARN_ON_ONCE(debug_locks			&&		\
>  		     (preempt_count() == 0		&&		\
>  		      raw_cpu_read(hardirqs_enabled)));			\
>  } while (0)
> --- a/lib/Kconfig.debug
> +++ b/lib/Kconfig.debug
> @@ -1161,7 +1161,6 @@ config PROVE_LOCKING
>  	select DEBUG_RWSEMS
>  	select DEBUG_WW_MUTEX_SLOWPATH
>  	select DEBUG_LOCK_ALLOC
> -	select PREEMPT_COUNT
>  	select TRACE_IRQFLAGS
>  	default n
>  	help

Acked-by: Will Deacon <will@kernel.org>

Will
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [Intel-gfx] [patch 04/13] lockdep: Clenaup PREEMPT_COUNT leftovers
@ 2020-09-15 16:11     ` Will Deacon
  0 siblings, 0 replies; 282+ messages in thread
From: Will Deacon @ 2020-09-15 16:11 UTC (permalink / raw)
  To: Thomas Gleixner
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, linux-mm, linux-kselftest,
	linux-hexagon, Shuah Khan, Ingo Molnar, Anton Ivanov, linux-arch,
	Linus Torvalds, Brian Cain, Richard Weinberger, Russell King,
	David Airlie, Ingo Molnar, Geert Uytterhoeven, Mel Gorman,
	intel-gfx, Matt Turner, Valentin Schneider, linux-xtensa,
	Paul E. McKenney, Jeff Dike, linux-um, Josh Triplett,
	Steven Rostedt, rcu, linux-m68k, Ivan Kokshaysky,
	Dietmar Eggemann, linux-arm-kernel, Richard Henderson,
	Chris Zankel, Max Filippov, LKML, linux-alpha, Mathieu Desnoyers,
	Andrew Morton, Daniel Bristot de Oliveira

On Mon, Sep 14, 2020 at 10:42:13PM +0200, Thomas Gleixner wrote:
> CONFIG_PREEMPT_COUNT is now unconditionally enabled and will be
> removed. Cleanup the leftovers before doing so.
> 
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> Cc: Peter Zijlstra <peterz@infradead.org>
> Cc: Ingo Molnar <mingo@kernel.org>
> Cc: Will Deacon <will@kernel.org>
> ---
>  include/linux/lockdep.h |    6 ++----
>  lib/Kconfig.debug       |    1 -
>  2 files changed, 2 insertions(+), 5 deletions(-)
> 
> --- a/include/linux/lockdep.h
> +++ b/include/linux/lockdep.h
> @@ -585,16 +585,14 @@ do {									\
>  
>  #define lockdep_assert_preemption_enabled()				\
>  do {									\
> -	WARN_ON_ONCE(IS_ENABLED(CONFIG_PREEMPT_COUNT)	&&		\
> -		     debug_locks			&&		\
> +	WARN_ON_ONCE(debug_locks			&&		\
>  		     (preempt_count() != 0		||		\
>  		      !raw_cpu_read(hardirqs_enabled)));		\
>  } while (0)
>  
>  #define lockdep_assert_preemption_disabled()				\
>  do {									\
> -	WARN_ON_ONCE(IS_ENABLED(CONFIG_PREEMPT_COUNT)	&&		\
> -		     debug_locks			&&		\
> +	WARN_ON_ONCE(debug_locks			&&		\
>  		     (preempt_count() == 0		&&		\
>  		      raw_cpu_read(hardirqs_enabled)));			\
>  } while (0)
> --- a/lib/Kconfig.debug
> +++ b/lib/Kconfig.debug
> @@ -1161,7 +1161,6 @@ config PROVE_LOCKING
>  	select DEBUG_RWSEMS
>  	select DEBUG_WW_MUTEX_SLOWPATH
>  	select DEBUG_LOCK_ALLOC
> -	select PREEMPT_COUNT
>  	select TRACE_IRQFLAGS
>  	default n
>  	help

Acked-by: Will Deacon <will@kernel.org>

Will
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 04/13] lockdep: Clenaup PREEMPT_COUNT leftovers
@ 2020-09-15 16:11     ` Will Deacon
  0 siblings, 0 replies; 282+ messages in thread
From: Will Deacon @ 2020-09-15 16:11 UTC (permalink / raw)
  To: Thomas Gleixner
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Joonas Lahtinen, Lai Jiangshan, dri-devel, Ben Segall, linux-mm,
	linux-kselftest, linux-hexagon, Shuah Khan, Ingo Molnar,
	Anton Ivanov, linux-arch, Linus Torvalds, Brian Cain,
	Richard Weinberger, Russell King, David Airlie, Ingo Molnar,
	Geert Uytterhoeven, Mel Gorman, intel-gfx, Matt Turner, Valentin

On Mon, Sep 14, 2020 at 10:42:13PM +0200, Thomas Gleixner wrote:
> CONFIG_PREEMPT_COUNT is now unconditionally enabled and will be
> removed. Cleanup the leftovers before doing so.
> 
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> Cc: Peter Zijlstra <peterz@infradead.org>
> Cc: Ingo Molnar <mingo@kernel.org>
> Cc: Will Deacon <will@kernel.org>
> ---
>  include/linux/lockdep.h |    6 ++----
>  lib/Kconfig.debug       |    1 -
>  2 files changed, 2 insertions(+), 5 deletions(-)
> 
> --- a/include/linux/lockdep.h
> +++ b/include/linux/lockdep.h
> @@ -585,16 +585,14 @@ do {									\
>  
>  #define lockdep_assert_preemption_enabled()				\
>  do {									\
> -	WARN_ON_ONCE(IS_ENABLED(CONFIG_PREEMPT_COUNT)	&&		\
> -		     debug_locks			&&		\
> +	WARN_ON_ONCE(debug_locks			&&		\
>  		     (preempt_count() != 0		||		\
>  		      !raw_cpu_read(hardirqs_enabled)));		\
>  } while (0)
>  
>  #define lockdep_assert_preemption_disabled()				\
>  do {									\
> -	WARN_ON_ONCE(IS_ENABLED(CONFIG_PREEMPT_COUNT)	&&		\
> -		     debug_locks			&&		\
> +	WARN_ON_ONCE(debug_locks			&&		\
>  		     (preempt_count() == 0		&&		\
>  		      raw_cpu_read(hardirqs_enabled)));			\
>  } while (0)
> --- a/lib/Kconfig.debug
> +++ b/lib/Kconfig.debug
> @@ -1161,7 +1161,6 @@ config PROVE_LOCKING
>  	select DEBUG_RWSEMS
>  	select DEBUG_WW_MUTEX_SLOWPATH
>  	select DEBUG_LOCK_ALLOC
> -	select PREEMPT_COUNT
>  	select TRACE_IRQFLAGS
>  	default n
>  	help

Acked-by: Will Deacon <will@kernel.org>

Will

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
  2020-09-14 20:59   ` Linus Torvalds
                       ` (3 preceding siblings ...)
  (?)
@ 2020-09-15 17:25     ` Paul E. McKenney
  -1 siblings, 0 replies; 282+ messages in thread
From: Paul E. McKenney @ 2020-09-15 17:25 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Thomas Gleixner, LKML, linux-arch, Sebastian Andrzej Siewior,
	Valentin Schneider, Richard Henderson, Ivan Kokshaysky,
	Matt Turner, alpha, Jeff Dike, Richard Weinberger, Anton Ivanov,
	linux-um, Brian Cain, linux-hexagon, Geert Uytterhoeven,
	linux-m68k, Ingo Molnar, Peter Zijlstra, Juri Lelli,
	Vincent Guittot, Dietmar Eggemann, Steven Rostedt, Ben Segall,
	Mel Gorman, Daniel Bristot de Oliveira, Will Deacon,
	Andrew Morton, Linux-MM, Ingo Molnar, Russell King, Linux ARM,
	Chris Zankel, Max Filippov, linux-xtensa, Jani Nikula,
	Joonas Lahtinen, Rodrigo Vivi, David Airlie, Daniel Vetter,
	intel-gfx, dri-devel, Josh Triplett, Mathieu Desnoyers,
	Lai Jiangshan, Shuah Khan, rcu,
	open list:KERNEL SELFTEST FRAMEWORK

On Mon, Sep 14, 2020 at 01:59:15PM -0700, Linus Torvalds wrote:
> On Mon, Sep 14, 2020 at 1:45 PM Thomas Gleixner <tglx@linutronix.de> wrote:
> >
> > Recently merged code does:
> >
> >          gfp = preemptible() ? GFP_KERNEL : GFP_ATOMIC;
> >
> > Looks obviously correct, except for the fact that preemptible() is
> > unconditionally false for CONFIF_PREEMPT_COUNT=n, i.e. all allocations in
> > that code use GFP_ATOMIC on such kernels.
> 
> I don't think this is a good reason to entirely get rid of the no-preempt thing.
> 
> The above is just garbage. It's bogus. You can't do it.
> 
> Blaming the no-preempt code for this bug is extremely unfair, imho.
> 
> And the no-preempt code does help make for much better code generation
> for simple spinlocks.
> 
> Where is that horribly buggy recent code? It's not in that exact
> format, certainly, since 'grep' doesn't find it.

It would be convenient for that "gfp =" code to work, as this would
allow better cache locality while invoking RCU callbacks, and would
further provide better robustness to callback floods.  The full story
is quite long, but here are alternatives have not yet been proven to be
abject failures:

1.	Use workqueues to do the allocations in a clean context.
	While waiting for the allocations, the callbacks are queued
	in the old cache-busting manner.  This functions correctly,
	but in the meantime (which on busy systems can be some time)
	the cache locality and robustness are lost.

2.	Provide the ability to allocate memory in raw atomic context.
	This is extremely effective, especially when used in combination
	with #1 above, but as you might suspect, the MM guys don't like
	it much.

In contrast, with Thomas's patch series, call_rcu() and kvfree_rcu()
could just look at preemptible() to see whether or not it was safe to
allocate memory, even in !PREEMPT kernels -- and in the common case,
it almost always would be safe.  It is quite possible that this approach
would work in isolation, or failing that, that adding #1 above would do
the trick.

I understand that this is all very hand-wavy, and I do apologize for that.
If you really want the full sad story with performance numbers and the
works, let me know!

							Thanx, Paul

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-15 17:25     ` Paul E. McKenney
  0 siblings, 0 replies; 282+ messages in thread
From: Paul E. McKenney @ 2020-09-15 17:25 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Thomas Gleixner, LKML, linux-arch, Sebastian Andrzej Siewior,
	Valentin Schneider, Richard Henderson, Ivan Kokshaysky,
	Matt Turner, alpha, Jeff Dike, Richard Weinberger, Anton Ivanov,
	linux-um, Brian Cain, linux-hexagon, Geert Uytterhoeven,
	linux-m68k, Ingo Molnar, Peter Zijlstra, Juri Lelli,
	Vincent Guittot, Dietmar Eggemann, Steven Rostedt, Ben Segall,
	Mel Gorman, Daniel Bristot de Oliveira, Will Deacon,
	Andrew Morton, Linux-MM, Ingo Molnar, Russell King, Linux ARM,
	Chris Zankel, Max Filippov, linux-xtensa, Jani Nikula,
	Joonas Lahtinen, Rodrigo Vivi, David Airlie, Daniel Vetter,
	intel-gfx, dri-devel, Josh Triplett, Mathieu Desnoyers,
	Lai Jiangshan, Shuah Khan, rcu,
	open list:KERNEL SELFTEST FRAMEWORK

On Mon, Sep 14, 2020 at 01:59:15PM -0700, Linus Torvalds wrote:
> On Mon, Sep 14, 2020 at 1:45 PM Thomas Gleixner <tglx@linutronix.de> wrote:
> >
> > Recently merged code does:
> >
> >          gfp = preemptible() ? GFP_KERNEL : GFP_ATOMIC;
> >
> > Looks obviously correct, except for the fact that preemptible() is
> > unconditionally false for CONFIF_PREEMPT_COUNT=n, i.e. all allocations in
> > that code use GFP_ATOMIC on such kernels.
> 
> I don't think this is a good reason to entirely get rid of the no-preempt thing.
> 
> The above is just garbage. It's bogus. You can't do it.
> 
> Blaming the no-preempt code for this bug is extremely unfair, imho.
> 
> And the no-preempt code does help make for much better code generation
> for simple spinlocks.
> 
> Where is that horribly buggy recent code? It's not in that exact
> format, certainly, since 'grep' doesn't find it.

It would be convenient for that "gfp =" code to work, as this would
allow better cache locality while invoking RCU callbacks, and would
further provide better robustness to callback floods.  The full story
is quite long, but here are alternatives have not yet been proven to be
abject failures:

1.	Use workqueues to do the allocations in a clean context.
	While waiting for the allocations, the callbacks are queued
	in the old cache-busting manner.  This functions correctly,
	but in the meantime (which on busy systems can be some time)
	the cache locality and robustness are lost.

2.	Provide the ability to allocate memory in raw atomic context.
	This is extremely effective, especially when used in combination
	with #1 above, but as you might suspect, the MM guys don't like
	it much.

In contrast, with Thomas's patch series, call_rcu() and kvfree_rcu()
could just look at preemptible() to see whether or not it was safe to
allocate memory, even in !PREEMPT kernels -- and in the common case,
it almost always would be safe.  It is quite possible that this approach
would work in isolation, or failing that, that adding #1 above would do
the trick.

I understand that this is all very hand-wavy, and I do apologize for that.
If you really want the full sad story with performance numbers and the
works, let me know!

							Thanx, Paul


^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-15 17:25     ` Paul E. McKenney
  0 siblings, 0 replies; 282+ messages in thread
From: Paul E. McKenney @ 2020-09-15 17:25 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Joonas Lahtinen, Lai Jiangshan, dri-devel, Ben Segall, Linux-MM,
	open list:KERNEL SELFTEST FRAMEWORK, linux-hexagon, Will Deacon,
	Ingo Molnar, Anton Ivanov, linux-arch, Vincent Guittot,
	Brian Cain, Richard Weinberger, Russell King, David Airlie,
	Ingo Molnar, Geert Uytterhoeven, Mel Gorman, intel-gfx,
	Matt Turner, Valentin Schneider, linux-xtensa, Shuah Khan,
	Jeff Dike, linux-um, Josh Triplett, Steven Rostedt, rcu,
	linux-m68k, Ivan Kokshaysky, Jani Nikula, Rodrigo Vivi,
	Thomas Gleixner, Dietmar Eggemann, Linux ARM, Richard Henderson,
	Chris Zankel, Max Filippov, LKML, Daniel Vetter, alpha,
	Mathieu Desnoyers, Andrew Morton, Daniel Bristot de Oliveira

On Mon, Sep 14, 2020 at 01:59:15PM -0700, Linus Torvalds wrote:
> On Mon, Sep 14, 2020 at 1:45 PM Thomas Gleixner <tglx@linutronix.de> wrote:
> >
> > Recently merged code does:
> >
> >          gfp = preemptible() ? GFP_KERNEL : GFP_ATOMIC;
> >
> > Looks obviously correct, except for the fact that preemptible() is
> > unconditionally false for CONFIF_PREEMPT_COUNT=n, i.e. all allocations in
> > that code use GFP_ATOMIC on such kernels.
> 
> I don't think this is a good reason to entirely get rid of the no-preempt thing.
> 
> The above is just garbage. It's bogus. You can't do it.
> 
> Blaming the no-preempt code for this bug is extremely unfair, imho.
> 
> And the no-preempt code does help make for much better code generation
> for simple spinlocks.
> 
> Where is that horribly buggy recent code? It's not in that exact
> format, certainly, since 'grep' doesn't find it.

It would be convenient for that "gfp =" code to work, as this would
allow better cache locality while invoking RCU callbacks, and would
further provide better robustness to callback floods.  The full story
is quite long, but here are alternatives have not yet been proven to be
abject failures:

1.	Use workqueues to do the allocations in a clean context.
	While waiting for the allocations, the callbacks are queued
	in the old cache-busting manner.  This functions correctly,
	but in the meantime (which on busy systems can be some time)
	the cache locality and robustness are lost.

2.	Provide the ability to allocate memory in raw atomic context.
	This is extremely effective, especially when used in combination
	with #1 above, but as you might suspect, the MM guys don't like
	it much.

In contrast, with Thomas's patch series, call_rcu() and kvfree_rcu()
could just look at preemptible() to see whether or not it was safe to
allocate memory, even in !PREEMPT kernels -- and in the common case,
it almost always would be safe.  It is quite possible that this approach
would work in isolation, or failing that, that adding #1 above would do
the trick.

I understand that this is all very hand-wavy, and I do apologize for that.
If you really want the full sad story with performance numbers and the
works, let me know!

							Thanx, Paul

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-15 17:25     ` Paul E. McKenney
  0 siblings, 0 replies; 282+ messages in thread
From: Paul E. McKenney @ 2020-09-15 17:25 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, Linux-MM,
	open list:KERNEL SELFTEST FRAMEWORK, linux-hexagon, Will Deacon,
	Ingo Molnar, Anton Ivanov, linux-arch, Vincent Guittot,
	Brian Cain, Richard Weinberger, Russell King, David Airlie,
	Ingo Molnar, Geert Uytterhoeven, Mel Gorman, intel-gfx,
	Matt Turner, Valentin Schneider, linux-xtensa, Shuah Khan,
	Jeff Dike, linux-um, Josh Triplett, Steven Rostedt, rcu,
	linux-m68k, Ivan Kokshaysky, Rodrigo Vivi, Thomas Gleixner,
	Dietmar Eggemann, Linux ARM, Richard Henderson, Chris Zankel,
	Max Filippov, LKML, alpha, Mathieu Desnoyers, Andrew Morton,
	Daniel Bristot de Oliveira

On Mon, Sep 14, 2020 at 01:59:15PM -0700, Linus Torvalds wrote:
> On Mon, Sep 14, 2020 at 1:45 PM Thomas Gleixner <tglx@linutronix.de> wrote:
> >
> > Recently merged code does:
> >
> >          gfp = preemptible() ? GFP_KERNEL : GFP_ATOMIC;
> >
> > Looks obviously correct, except for the fact that preemptible() is
> > unconditionally false for CONFIF_PREEMPT_COUNT=n, i.e. all allocations in
> > that code use GFP_ATOMIC on such kernels.
> 
> I don't think this is a good reason to entirely get rid of the no-preempt thing.
> 
> The above is just garbage. It's bogus. You can't do it.
> 
> Blaming the no-preempt code for this bug is extremely unfair, imho.
> 
> And the no-preempt code does help make for much better code generation
> for simple spinlocks.
> 
> Where is that horribly buggy recent code? It's not in that exact
> format, certainly, since 'grep' doesn't find it.

It would be convenient for that "gfp =" code to work, as this would
allow better cache locality while invoking RCU callbacks, and would
further provide better robustness to callback floods.  The full story
is quite long, but here are alternatives have not yet been proven to be
abject failures:

1.	Use workqueues to do the allocations in a clean context.
	While waiting for the allocations, the callbacks are queued
	in the old cache-busting manner.  This functions correctly,
	but in the meantime (which on busy systems can be some time)
	the cache locality and robustness are lost.

2.	Provide the ability to allocate memory in raw atomic context.
	This is extremely effective, especially when used in combination
	with #1 above, but as you might suspect, the MM guys don't like
	it much.

In contrast, with Thomas's patch series, call_rcu() and kvfree_rcu()
could just look at preemptible() to see whether or not it was safe to
allocate memory, even in !PREEMPT kernels -- and in the common case,
it almost always would be safe.  It is quite possible that this approach
would work in isolation, or failing that, that adding #1 above would do
the trick.

I understand that this is all very hand-wavy, and I do apologize for that.
If you really want the full sad story with performance numbers and the
works, let me know!

							Thanx, Paul
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [Intel-gfx] [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-15 17:25     ` Paul E. McKenney
  0 siblings, 0 replies; 282+ messages in thread
From: Paul E. McKenney @ 2020-09-15 17:25 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, Linux-MM,
	open list:KERNEL SELFTEST FRAMEWORK, linux-hexagon, Will Deacon,
	Ingo Molnar, Anton Ivanov, linux-arch, Brian Cain,
	Richard Weinberger, Russell King, David Airlie, Ingo Molnar,
	Geert Uytterhoeven, Mel Gorman, intel-gfx, Matt Turner,
	Valentin Schneider, linux-xtensa, Shuah Khan, Jeff Dike,
	linux-um, Josh Triplett, Steven Rostedt, rcu, linux-m68k,
	Ivan Kokshaysky, Thomas Gleixner, Dietmar Eggemann, Linux ARM,
	Richard Henderson, Chris Zankel, Max Filippov, LKML, alpha,
	Mathieu Desnoyers, Andrew Morton, Daniel Bristot de Oliveira

On Mon, Sep 14, 2020 at 01:59:15PM -0700, Linus Torvalds wrote:
> On Mon, Sep 14, 2020 at 1:45 PM Thomas Gleixner <tglx@linutronix.de> wrote:
> >
> > Recently merged code does:
> >
> >          gfp = preemptible() ? GFP_KERNEL : GFP_ATOMIC;
> >
> > Looks obviously correct, except for the fact that preemptible() is
> > unconditionally false for CONFIF_PREEMPT_COUNT=n, i.e. all allocations in
> > that code use GFP_ATOMIC on such kernels.
> 
> I don't think this is a good reason to entirely get rid of the no-preempt thing.
> 
> The above is just garbage. It's bogus. You can't do it.
> 
> Blaming the no-preempt code for this bug is extremely unfair, imho.
> 
> And the no-preempt code does help make for much better code generation
> for simple spinlocks.
> 
> Where is that horribly buggy recent code? It's not in that exact
> format, certainly, since 'grep' doesn't find it.

It would be convenient for that "gfp =" code to work, as this would
allow better cache locality while invoking RCU callbacks, and would
further provide better robustness to callback floods.  The full story
is quite long, but here are alternatives have not yet been proven to be
abject failures:

1.	Use workqueues to do the allocations in a clean context.
	While waiting for the allocations, the callbacks are queued
	in the old cache-busting manner.  This functions correctly,
	but in the meantime (which on busy systems can be some time)
	the cache locality and robustness are lost.

2.	Provide the ability to allocate memory in raw atomic context.
	This is extremely effective, especially when used in combination
	with #1 above, but as you might suspect, the MM guys don't like
	it much.

In contrast, with Thomas's patch series, call_rcu() and kvfree_rcu()
could just look at preemptible() to see whether or not it was safe to
allocate memory, even in !PREEMPT kernels -- and in the common case,
it almost always would be safe.  It is quite possible that this approach
would work in isolation, or failing that, that adding #1 above would do
the trick.

I understand that this is all very hand-wavy, and I do apologize for that.
If you really want the full sad story with performance numbers and the
works, let me know!

							Thanx, Paul
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-15 17:25     ` Paul E. McKenney
  0 siblings, 0 replies; 282+ messages in thread
From: Paul E. McKenney @ 2020-09-15 17:25 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Joonas Lahtinen, Lai Jiangshan, dri-devel, Ben Segall, Linux-MM,
	open list:KERNEL SELFTEST FRAMEWORK, linux-hexagon, Will Deacon,
	Ingo Molnar, Anton Ivanov, linux-arch, Vincent Guittot,
	Brian Cain, Richard Weinberger, Russell King, David Airlie,
	Ingo Molnar, Geert Uytterhoeven, Mel Gorman, intel-

On Mon, Sep 14, 2020 at 01:59:15PM -0700, Linus Torvalds wrote:
> On Mon, Sep 14, 2020 at 1:45 PM Thomas Gleixner <tglx@linutronix.de> wrote:
> >
> > Recently merged code does:
> >
> >          gfp = preemptible() ? GFP_KERNEL : GFP_ATOMIC;
> >
> > Looks obviously correct, except for the fact that preemptible() is
> > unconditionally false for CONFIF_PREEMPT_COUNT=n, i.e. all allocations in
> > that code use GFP_ATOMIC on such kernels.
> 
> I don't think this is a good reason to entirely get rid of the no-preempt thing.
> 
> The above is just garbage. It's bogus. You can't do it.
> 
> Blaming the no-preempt code for this bug is extremely unfair, imho.
> 
> And the no-preempt code does help make for much better code generation
> for simple spinlocks.
> 
> Where is that horribly buggy recent code? It's not in that exact
> format, certainly, since 'grep' doesn't find it.

It would be convenient for that "gfp =" code to work, as this would
allow better cache locality while invoking RCU callbacks, and would
further provide better robustness to callback floods.  The full story
is quite long, but here are alternatives have not yet been proven to be
abject failures:

1.	Use workqueues to do the allocations in a clean context.
	While waiting for the allocations, the callbacks are queued
	in the old cache-busting manner.  This functions correctly,
	but in the meantime (which on busy systems can be some time)
	the cache locality and robustness are lost.

2.	Provide the ability to allocate memory in raw atomic context.
	This is extremely effective, especially when used in combination
	with #1 above, but as you might suspect, the MM guys don't like
	it much.

In contrast, with Thomas's patch series, call_rcu() and kvfree_rcu()
could just look at preemptible() to see whether or not it was safe to
allocate memory, even in !PREEMPT kernels -- and in the common case,
it almost always would be safe.  It is quite possible that this approach
would work in isolation, or failing that, that adding #1 above would do
the trick.

I understand that this is all very hand-wavy, and I do apologize for that.
If you really want the full sad story with performance numbers and the
works, let me know!

							Thanx, Paul

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
  2020-09-15  7:24                 ` [Intel-gfx] " Thomas Gleixner
                                     ` (2 preceding siblings ...)
  (?)
@ 2020-09-15 17:29                   ` Linus Torvalds
  -1 siblings, 0 replies; 282+ messages in thread
From: Linus Torvalds @ 2020-09-15 17:29 UTC (permalink / raw)
  To: Thomas Gleixner
  Cc: Herbert Xu, Ard Biesheuvel, LKML, linux-arch,
	Sebastian Andrzej Siewior, Valentin Schneider, Richard Henderson,
	Ivan Kokshaysky, Matt Turner, alpha, Jeff Dike,
	Richard Weinberger, Anton Ivanov, linux-um, Brian Cain,
	linux-hexagon, Geert Uytterhoeven, linux-m68k, Ingo Molnar,
	Peter Zijlstra, Juri Lelli, Vincent Guittot, Dietmar Eggemann,
	Steven Rostedt, Ben Segall, Mel Gorman,
	Daniel Bristot de Oliveira, Will Deacon, Andrew Morton, Linux-MM,
	Ingo Molnar, Russell King, Linux ARM, Chris Zankel, Max Filippov,
	linux-xtensa, Jani Nikula, Joonas Lahtinen, Rodrigo Vivi,
	David Airlie, Daniel Vetter, intel-gfx, dri-devel,
	Paul E. McKenney, Josh Triplett, Mathieu Desnoyers,
	Lai Jiangshan, Shuah Khan, rcu

On Tue, Sep 15, 2020 at 12:24 AM Thomas Gleixner <tglx@linutronix.de> wrote:
>
> Alternatively we just make highmem a bit more expensive by making these
> maps preemptible. RT is doing this for a long time and it's not that
> horrible.

Ack.

In fact, I've wanted to start just removing kmap support entirely. At
some point it's not so much about "I have an old machine that wants
HIGHMEM" but about "I have an old CPU, and I'll just run an old
kernel".

It's not that 32-bit is irrelevant, it's that 32-bit with large
amounts of memory is irrelevant.

Last time this was discussed, iirc the main issue was some
questionable old ARM chips that were still very common in embedded
environments, even with large memory.

But we could definitely start de-emphasizing HIGHMEM.

                     Linus

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-15 17:29                   ` Linus Torvalds
  0 siblings, 0 replies; 282+ messages in thread
From: Linus Torvalds @ 2020-09-15 17:29 UTC (permalink / raw)
  To: Thomas Gleixner
  Cc: Herbert Xu, Ard Biesheuvel, LKML, linux-arch,
	Sebastian Andrzej Siewior, Valentin Schneider, Richard Henderson,
	Ivan Kokshaysky, Matt Turner, alpha, Jeff Dike,
	Richard Weinberger, Anton Ivanov, linux-um, Brian Cain,
	linux-hexagon, Geert Uytterhoeven, linux-m68k, Ingo Molnar,
	Peter Zijlstra, Juri Lelli, Vincent Guittot, Dietmar Eggemann,
	Steven Rostedt, Ben Segall, Mel Gorman,
	Daniel Bristot de Oliveira, Will Deacon, Andrew Morton, Linux-MM,
	Ingo Molnar, Russell King, Linux ARM, Chris Zankel, Max Filippov,
	linux-xtensa, Jani Nikula, Joonas Lahtinen, Rodrigo Vivi,
	David Airlie, Daniel Vetter, intel-gfx, dri-devel,
	Paul E. McKenney, Josh Triplett, Mathieu Desnoyers,
	Lai Jiangshan, Shuah Khan, rcu

On Tue, Sep 15, 2020 at 12:24 AM Thomas Gleixner <tglx@linutronix.de> wrote:
>
> Alternatively we just make highmem a bit more expensive by making these
> maps preemptible. RT is doing this for a long time and it's not that
> horrible.

Ack.

In fact, I've wanted to start just removing kmap support entirely. At
some point it's not so much about "I have an old machine that wants
HIGHMEM" but about "I have an old CPU, and I'll just run an old
kernel".

It's not that 32-bit is irrelevant, it's that 32-bit with large
amounts of memory is irrelevant.

Last time this was discussed, iirc the main issue was some
questionable old ARM chips that were still very common in embedded
environments, even with large memory.

But we could definitely start de-emphasizing HIGHMEM.

                     Linus


^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-15 17:29                   ` Linus Torvalds
  0 siblings, 0 replies; 282+ messages in thread
From: Linus Torvalds @ 2020-09-15 17:29 UTC (permalink / raw)
  To: Thomas Gleixner
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, Linux-MM, linux-hexagon,
	Will Deacon, Ingo Molnar, Anton Ivanov, linux-arch,
	Vincent Guittot, Herbert Xu, Brian Cain, Richard Weinberger,
	Russell King, Ard Biesheuvel, David Airlie, Ingo Molnar,
	Geert Uytterhoeven, Mel Gorman, intel-gfx, Matt Turner,
	Valentin Schneider, linux-xtensa, Shuah Khan, Paul E. McKenney,
	Jeff Dike, linux-um, Josh Triplett, Steven Rostedt, rcu,
	linux-m68k, Ivan Kokshaysky, Rodrigo Vivi, Dietmar Eggemann,
	Linux ARM, Richard Henderson, Chris Zankel, Max Filippov, LKML,
	alpha, Mathieu Desnoyers, Andrew Morton,
	Daniel Bristot de Oliveira

On Tue, Sep 15, 2020 at 12:24 AM Thomas Gleixner <tglx@linutronix.de> wrote:
>
> Alternatively we just make highmem a bit more expensive by making these
> maps preemptible. RT is doing this for a long time and it's not that
> horrible.

Ack.

In fact, I've wanted to start just removing kmap support entirely. At
some point it's not so much about "I have an old machine that wants
HIGHMEM" but about "I have an old CPU, and I'll just run an old
kernel".

It's not that 32-bit is irrelevant, it's that 32-bit with large
amounts of memory is irrelevant.

Last time this was discussed, iirc the main issue was some
questionable old ARM chips that were still very common in embedded
environments, even with large memory.

But we could definitely start de-emphasizing HIGHMEM.

                     Linus
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [Intel-gfx] [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-15 17:29                   ` Linus Torvalds
  0 siblings, 0 replies; 282+ messages in thread
From: Linus Torvalds @ 2020-09-15 17:29 UTC (permalink / raw)
  To: Thomas Gleixner
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, Linux-MM, linux-hexagon,
	Will Deacon, Ingo Molnar, Anton Ivanov, linux-arch, Herbert Xu,
	Brian Cain, Richard Weinberger, Russell King, Ard Biesheuvel,
	David Airlie, Ingo Molnar, Geert Uytterhoeven, Mel Gorman,
	intel-gfx, Matt Turner, Valentin Schneider, linux-xtensa,
	Shuah Khan, Paul E. McKenney, Jeff Dike, linux-um, Josh Triplett,
	Steven Rostedt, rcu, linux-m68k, Ivan Kokshaysky,
	Dietmar Eggemann, Linux ARM, Richard Henderson, Chris Zankel,
	Max Filippov, LKML, alpha, Mathieu Desnoyers, Andrew Morton,
	Daniel Bristot de Oliveira

On Tue, Sep 15, 2020 at 12:24 AM Thomas Gleixner <tglx@linutronix.de> wrote:
>
> Alternatively we just make highmem a bit more expensive by making these
> maps preemptible. RT is doing this for a long time and it's not that
> horrible.

Ack.

In fact, I've wanted to start just removing kmap support entirely. At
some point it's not so much about "I have an old machine that wants
HIGHMEM" but about "I have an old CPU, and I'll just run an old
kernel".

It's not that 32-bit is irrelevant, it's that 32-bit with large
amounts of memory is irrelevant.

Last time this was discussed, iirc the main issue was some
questionable old ARM chips that were still very common in embedded
environments, even with large memory.

But we could definitely start de-emphasizing HIGHMEM.

                     Linus
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-15 17:29                   ` Linus Torvalds
  0 siblings, 0 replies; 282+ messages in thread
From: Linus Torvalds @ 2020-09-15 17:29 UTC (permalink / raw)
  To: Thomas Gleixner
  Cc: Herbert Xu, Ard Biesheuvel, LKML, linux-arch,
	Sebastian Andrzej Siewior, Valentin Schneider, Richard Henderson,
	Ivan Kokshaysky, Matt Turner, alpha, Jeff Dike,
	Richard Weinberger, Anton Ivanov, linux-um, Brian Cain,
	linux-hexagon, Geert Uytterhoeven, linux-m68k, Ingo Molnar

On Tue, Sep 15, 2020 at 12:24 AM Thomas Gleixner <tglx@linutronix.de> wrote:
>
> Alternatively we just make highmem a bit more expensive by making these
> maps preemptible. RT is doing this for a long time and it's not that
> horrible.

Ack.

In fact, I've wanted to start just removing kmap support entirely. At
some point it's not so much about "I have an old machine that wants
HIGHMEM" but about "I have an old CPU, and I'll just run an old
kernel".

It's not that 32-bit is irrelevant, it's that 32-bit with large
amounts of memory is irrelevant.

Last time this was discussed, iirc the main issue was some
questionable old ARM chips that were still very common in embedded
environments, even with large memory.

But we could definitely start de-emphasizing HIGHMEM.

                     Linus

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
  2020-09-15  8:39         ` Thomas Gleixner
                             ` (2 preceding siblings ...)
  (?)
@ 2020-09-15 17:35           ` Linus Torvalds
  -1 siblings, 0 replies; 282+ messages in thread
From: Linus Torvalds @ 2020-09-15 17:35 UTC (permalink / raw)
  To: Thomas Gleixner
  Cc: Ard Biesheuvel, Herbert Xu, LKML, linux-arch,
	Sebastian Andrzej Siewior, Valentin Schneider, Richard Henderson,
	Ivan Kokshaysky, Matt Turner, alpha, Jeff Dike,
	Richard Weinberger, Anton Ivanov, linux-um, Brian Cain,
	linux-hexagon, Geert Uytterhoeven, linux-m68k, Ingo Molnar,
	Peter Zijlstra, Juri Lelli, Vincent Guittot, Dietmar Eggemann,
	Steven Rostedt, Ben Segall, Mel Gorman,
	Daniel Bristot de Oliveira, Will Deacon, Andrew Morton, Linux-MM,
	Ingo Molnar, Russell King, Linux ARM, Chris Zankel, Max Filippov,
	linux-xtensa, Jani Nikula, Joonas Lahtinen, Rodrigo Vivi,
	David Airlie, Daniel Vetter, intel-gfx, dri-devel,
	Paul E. McKenney, Josh Triplett, Mathieu Desnoyers,
	Lai Jiangshan, Shuah Khan, rcu,
	open list:KERNEL SELFTEST FRAMEWORK

On Tue, Sep 15, 2020 at 1:39 AM Thomas Gleixner <tglx@linutronix.de> wrote:
>
> OTOH, having a working 'preemptible()' or maybe better named
> 'can_schedule()' check makes tons of sense to make decisions about
> allocation modes or other things.

No. I think that those kinds of decisions about actual behavior are
always simply fundamentally wrong.

Note that this is very different from having warnings about invalid
use. THAT is correct. It may not warn in all configurations, but that
doesn't matter: what matters is that it warns in common enough
configurations that developers will catch it.

So having a warning in "might_sleep()" that doesn't always trigger,
because you have a limited configuration that can't even detect the
situation, that's fine and dandy and intentional.

But having code like

       if (can_schedule())
           .. do something different ..

is fundamentally complete and utter garbage.

It's one thing if you test for "am I in hardware interrupt context".
Those tests aren't great either, but at least they make sense.

But a driver - or some library routine - making a difference based on
some nebulous "can I schedule" is fundamentally and basically WRONG.

If some code changes behavior, it needs to be explicit to the *caller*
of that code.

So this is why GFP_ATOMIC is fine, but "if (!can_schedule())
do_something_atomic()" is pure shite.

And I am not IN THE LEAST interested in trying to help people doing
pure shite. We need to fix them. Like the crypto code is getting
fixed.

                   Linus

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-15 17:35           ` Linus Torvalds
  0 siblings, 0 replies; 282+ messages in thread
From: Linus Torvalds @ 2020-09-15 17:35 UTC (permalink / raw)
  To: Thomas Gleixner
  Cc: Ard Biesheuvel, Herbert Xu, LKML, linux-arch,
	Sebastian Andrzej Siewior, Valentin Schneider, Richard Henderson,
	Ivan Kokshaysky, Matt Turner, alpha, Jeff Dike,
	Richard Weinberger, Anton Ivanov, linux-um, Brian Cain,
	linux-hexagon, Geert Uytterhoeven, linux-m68k, Ingo Molnar,
	Peter Zijlstra, Juri Lelli, Vincent Guittot, Dietmar Eggemann,
	Steven Rostedt, Ben Segall, Mel Gorman,
	Daniel Bristot de Oliveira, Will Deacon, Andrew Morton, Linux-MM,
	Ingo Molnar, Russell King, Linux ARM, Chris Zankel, Max Filippov,
	linux-xtensa, Jani Nikula, Joonas Lahtinen, Rodrigo Vivi,
	David Airlie, Daniel Vetter, intel-gfx, dri-devel,
	Paul E. McKenney, Josh Triplett, Mathieu Desnoyers,
	Lai Jiangshan, Shuah Khan, rcu,
	open list:KERNEL SELFTEST FRAMEWORK

On Tue, Sep 15, 2020 at 1:39 AM Thomas Gleixner <tglx@linutronix.de> wrote:
>
> OTOH, having a working 'preemptible()' or maybe better named
> 'can_schedule()' check makes tons of sense to make decisions about
> allocation modes or other things.

No. I think that those kinds of decisions about actual behavior are
always simply fundamentally wrong.

Note that this is very different from having warnings about invalid
use. THAT is correct. It may not warn in all configurations, but that
doesn't matter: what matters is that it warns in common enough
configurations that developers will catch it.

So having a warning in "might_sleep()" that doesn't always trigger,
because you have a limited configuration that can't even detect the
situation, that's fine and dandy and intentional.

But having code like

       if (can_schedule())
           .. do something different ..

is fundamentally complete and utter garbage.

It's one thing if you test for "am I in hardware interrupt context".
Those tests aren't great either, but at least they make sense.

But a driver - or some library routine - making a difference based on
some nebulous "can I schedule" is fundamentally and basically WRONG.

If some code changes behavior, it needs to be explicit to the *caller*
of that code.

So this is why GFP_ATOMIC is fine, but "if (!can_schedule())
do_something_atomic()" is pure shite.

And I am not IN THE LEAST interested in trying to help people doing
pure shite. We need to fix them. Like the crypto code is getting
fixed.

                   Linus


^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-15 17:35           ` Linus Torvalds
  0 siblings, 0 replies; 282+ messages in thread
From: Linus Torvalds @ 2020-09-15 17:35 UTC (permalink / raw)
  To: Thomas Gleixner
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, Linux-MM,
	open list:KERNEL SELFTEST FRAMEWORK, linux-hexagon, Will Deacon,
	Ingo Molnar, Anton Ivanov, linux-arch, Vincent Guittot,
	Herbert Xu, Brian Cain, Richard Weinberger, Russell King,
	Ard Biesheuvel, David Airlie, Ingo Molnar, Geert Uytterhoeven,
	Mel Gorman, intel-gfx, Matt Turner, Valentin Schneider,
	linux-xtensa, Shuah Khan, Paul E. McKenney, Jeff Dike, linux-um,
	Josh Triplett, Steven Rostedt, rcu, linux-m68k, Ivan Kokshaysky,
	Rodrigo Vivi, Dietmar Eggemann, Linux ARM, Richard Henderson,
	Chris Zankel, Max Filippov, LKML, alpha, Mathieu Desnoyers,
	Andrew Morton, Daniel Bristot de Oliveira

On Tue, Sep 15, 2020 at 1:39 AM Thomas Gleixner <tglx@linutronix.de> wrote:
>
> OTOH, having a working 'preemptible()' or maybe better named
> 'can_schedule()' check makes tons of sense to make decisions about
> allocation modes or other things.

No. I think that those kinds of decisions about actual behavior are
always simply fundamentally wrong.

Note that this is very different from having warnings about invalid
use. THAT is correct. It may not warn in all configurations, but that
doesn't matter: what matters is that it warns in common enough
configurations that developers will catch it.

So having a warning in "might_sleep()" that doesn't always trigger,
because you have a limited configuration that can't even detect the
situation, that's fine and dandy and intentional.

But having code like

       if (can_schedule())
           .. do something different ..

is fundamentally complete and utter garbage.

It's one thing if you test for "am I in hardware interrupt context".
Those tests aren't great either, but at least they make sense.

But a driver - or some library routine - making a difference based on
some nebulous "can I schedule" is fundamentally and basically WRONG.

If some code changes behavior, it needs to be explicit to the *caller*
of that code.

So this is why GFP_ATOMIC is fine, but "if (!can_schedule())
do_something_atomic()" is pure shite.

And I am not IN THE LEAST interested in trying to help people doing
pure shite. We need to fix them. Like the crypto code is getting
fixed.

                   Linus
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [Intel-gfx] [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-15 17:35           ` Linus Torvalds
  0 siblings, 0 replies; 282+ messages in thread
From: Linus Torvalds @ 2020-09-15 17:35 UTC (permalink / raw)
  To: Thomas Gleixner
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, Linux-MM,
	open list:KERNEL SELFTEST FRAMEWORK, linux-hexagon, Will Deacon,
	Ingo Molnar, Anton Ivanov, linux-arch, Herbert Xu, Brian Cain,
	Richard Weinberger, Russell King, Ard Biesheuvel, David Airlie,
	Ingo Molnar, Geert Uytterhoeven, Mel Gorman, intel-gfx,
	Matt Turner, Valentin Schneider, linux-xtensa, Shuah Khan,
	Paul E. McKenney, Jeff Dike, linux-um, Josh Triplett,
	Steven Rostedt, rcu, linux-m68k, Ivan Kokshaysky,
	Dietmar Eggemann, Linux ARM, Richard Henderson, Chris Zankel,
	Max Filippov, LKML, alpha, Mathieu Desnoyers, Andrew Morton,
	Daniel Bristot de Oliveira

On Tue, Sep 15, 2020 at 1:39 AM Thomas Gleixner <tglx@linutronix.de> wrote:
>
> OTOH, having a working 'preemptible()' or maybe better named
> 'can_schedule()' check makes tons of sense to make decisions about
> allocation modes or other things.

No. I think that those kinds of decisions about actual behavior are
always simply fundamentally wrong.

Note that this is very different from having warnings about invalid
use. THAT is correct. It may not warn in all configurations, but that
doesn't matter: what matters is that it warns in common enough
configurations that developers will catch it.

So having a warning in "might_sleep()" that doesn't always trigger,
because you have a limited configuration that can't even detect the
situation, that's fine and dandy and intentional.

But having code like

       if (can_schedule())
           .. do something different ..

is fundamentally complete and utter garbage.

It's one thing if you test for "am I in hardware interrupt context".
Those tests aren't great either, but at least they make sense.

But a driver - or some library routine - making a difference based on
some nebulous "can I schedule" is fundamentally and basically WRONG.

If some code changes behavior, it needs to be explicit to the *caller*
of that code.

So this is why GFP_ATOMIC is fine, but "if (!can_schedule())
do_something_atomic()" is pure shite.

And I am not IN THE LEAST interested in trying to help people doing
pure shite. We need to fix them. Like the crypto code is getting
fixed.

                   Linus
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-15 17:35           ` Linus Torvalds
  0 siblings, 0 replies; 282+ messages in thread
From: Linus Torvalds @ 2020-09-15 17:35 UTC (permalink / raw)
  To: Thomas Gleixner
  Cc: Ard Biesheuvel, Herbert Xu, LKML, linux-arch,
	Sebastian Andrzej Siewior, Valentin Schneider, Richard Henderson,
	Ivan Kokshaysky, Matt Turner, alpha, Jeff Dike,
	Richard Weinberger, Anton Ivanov, linux-um, Brian Cain,
	linux-hexagon, Geert Uytterhoeven, linux-m68k, Ingo Molnar

On Tue, Sep 15, 2020 at 1:39 AM Thomas Gleixner <tglx@linutronix.de> wrote:
>
> OTOH, having a working 'preemptible()' or maybe better named
> 'can_schedule()' check makes tons of sense to make decisions about
> allocation modes or other things.

No. I think that those kinds of decisions about actual behavior are
always simply fundamentally wrong.

Note that this is very different from having warnings about invalid
use. THAT is correct. It may not warn in all configurations, but that
doesn't matter: what matters is that it warns in common enough
configurations that developers will catch it.

So having a warning in "might_sleep()" that doesn't always trigger,
because you have a limited configuration that can't even detect the
situation, that's fine and dandy and intentional.

But having code like

       if (can_schedule())
           .. do something different ..

is fundamentally complete and utter garbage.

It's one thing if you test for "am I in hardware interrupt context".
Those tests aren't great either, but at least they make sense.

But a driver - or some library routine - making a difference based on
some nebulous "can I schedule" is fundamentally and basically WRONG.

If some code changes behavior, it needs to be explicit to the *caller*
of that code.

So this is why GFP_ATOMIC is fine, but "if (!can_schedule())
do_something_atomic()" is pure shite.

And I am not IN THE LEAST interested in trying to help people doing
pure shite. We need to fix them. Like the crypto code is getting
fixed.

                   Linus

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [PATCH] crypto: lib/chacha20poly1305 - Set SG_MITER_ATOMIC unconditionally
  2020-09-15 10:10                         ` Herbert Xu
@ 2020-09-15 19:04                           ` Thomas Gleixner
  0 siblings, 0 replies; 282+ messages in thread
From: Thomas Gleixner @ 2020-09-15 19:04 UTC (permalink / raw)
  To: Herbert Xu, Ard Biesheuvel
  Cc: Linus Torvalds, LKML, Linux Crypto Mailing List

On Tue, Sep 15 2020 at 20:10, Herbert Xu wrote:
> On Tue, Sep 15, 2020 at 01:08:31PM +0300, Ard Biesheuvel wrote:
>>
>> But making atomic kmap preemptible/sleepable creates the exact same
>> problem, i.e., that we have no idea which existing callers are
>> currently relying on those preemption disabling semantics, so we can't
>> just take them away. Or am I missing something?
>
> Good point.
>
> Thomas mentioned that RT has been doing this for a while now so
> perhaps someone has studied this problem already? Thomas?

RT is substituting preempt_disable() with migrate_disable() which pins
the task on the CPU so that per CPU stuff still works. And we did quite
some staring whether there is code which purely relies on the
preempt_disable() to prevent reentrancy, but there is almost none.

Though we don't have migrate disable on !RT and PeterZ is not a great
fan of making it available as it wreckages schedulability - though IMO
not much more than preempt disable :)

Thanks,

        tglx

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
  2020-09-15 17:35           ` Linus Torvalds
                               ` (3 preceding siblings ...)
  (?)
@ 2020-09-15 19:57             ` Thomas Gleixner
  -1 siblings, 0 replies; 282+ messages in thread
From: Thomas Gleixner @ 2020-09-15 19:57 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Ard Biesheuvel, Herbert Xu, LKML, linux-arch,
	Sebastian Andrzej Siewior, Valentin Schneider, Richard Henderson,
	Ivan Kokshaysky, Matt Turner, alpha, Jeff Dike,
	Richard Weinberger, Anton Ivanov, linux-um, Brian Cain,
	linux-hexagon, Geert Uytterhoeven, linux-m68k, Ingo Molnar,
	Peter Zijlstra, Juri Lelli, Vincent Guittot, Dietmar Eggemann,
	Steven Rostedt, Ben Segall, Mel Gorman,
	Daniel Bristot de Oliveira, Will Deacon, Andrew Morton, Linux-MM,
	Ingo Molnar, Russell King, Linux ARM, Chris Zankel, Max Filippov,
	linux-xtensa, Jani Nikula, Joonas Lahtinen, Rodrigo Vivi,
	David Airlie, Daniel Vetter, intel-gfx, dri-devel,
	Paul E. McKenney, Josh Triplett, Mathieu Desnoyers,
	Lai Jiangshan, Shuah Khan, rcu,
	open list:KERNEL SELFTEST FRAMEWORK

On Tue, Sep 15 2020 at 10:35, Linus Torvalds wrote:
> On Tue, Sep 15, 2020 at 1:39 AM Thomas Gleixner <tglx@linutronix.de> wrote:
>>
>> OTOH, having a working 'preemptible()' or maybe better named
>> 'can_schedule()' check makes tons of sense to make decisions about
>> allocation modes or other things.
>
> No. I think that those kinds of decisions about actual behavior are
> always simply fundamentally wrong.
>
> Note that this is very different from having warnings about invalid
> use. THAT is correct. It may not warn in all configurations, but that
> doesn't matter: what matters is that it warns in common enough
> configurations that developers will catch it.

You wish. I just found a 7 year old bug in a 10G network driver which
surely would have been found if people would enable debug configs and
not just run the crap on their PREEMPT_NONE, all debug off kernel. And
that driver is not subject to bitrot, it gets regular bug fixes from
people who seem to care (distro folks).

> So having a warning in "might_sleep()" that doesn't always trigger,
> because you have a limited configuration that can't even detect the
> situation, that's fine and dandy and intentional.

and lets people get away with their crap.

> But having code like
>
>        if (can_schedule())
>            .. do something different ..
>
> is fundamentally complete and utter garbage.
>
> It's one thing if you test for "am I in hardware interrupt context".
> Those tests aren't great either, but at least they make sense.

They make sense in limited situations like exception handlers and such
which really have to know from which context an exception was raised.

But with the above reasoning such checks do not make sense in any other
general code. 'in hard interrupt context' is just another context where
you can't do stuff which you can do when in preemptible task context.

Most tests are way broader than a single context. in_interrupt() is true
for hard interrupt, soft interrupt delivery and all BH disabled
contexts, which is completely ill defined.

> But a driver - or some library routine - making a difference based on
> some nebulous "can I schedule" is fundamentally and basically WRONG.
>
> If some code changes behavior, it needs to be explicit to the *caller*
> of that code.

I'm fine with that, but then we have to be consequent and ban _all_ of
these and not just declare can_schedule() to be a bad one.

Thanks,

        tglx

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-15 19:57             ` Thomas Gleixner
  0 siblings, 0 replies; 282+ messages in thread
From: Thomas Gleixner @ 2020-09-15 19:57 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Ard Biesheuvel, Herbert Xu, LKML, linux-arch,
	Sebastian Andrzej Siewior, Valentin Schneider, Richard Henderson,
	Ivan Kokshaysky, Matt Turner, alpha, Jeff Dike,
	Richard Weinberger, Anton Ivanov, linux-um, Brian Cain,
	linux-hexagon, Geert Uytterhoeven, linux-m68k, Ingo Molnar,
	Peter Zijlstra, Juri Lelli, Vincent Guittot, Dietmar Eggemann,
	Steven Rostedt, Ben Segall, Mel Gorman,
	Daniel Bristot de Oliveira, Will Deacon, Andrew Morton, Linux-MM,
	Ingo Molnar, Russell King, Linux ARM, Chris Zankel, Max Filippov,
	linux-xtensa, Jani Nikula, Joonas Lahtinen, Rodrigo Vivi,
	David Airlie, Daniel Vetter, intel-gfx, dri-devel,
	Paul E. McKenney, Josh Triplett, Mathieu Desnoyers,
	Lai Jiangshan, Shuah Khan, rcu,
	open list:KERNEL SELFTEST FRAMEWORK

On Tue, Sep 15 2020 at 10:35, Linus Torvalds wrote:
> On Tue, Sep 15, 2020 at 1:39 AM Thomas Gleixner <tglx@linutronix.de> wrote:
>>
>> OTOH, having a working 'preemptible()' or maybe better named
>> 'can_schedule()' check makes tons of sense to make decisions about
>> allocation modes or other things.
>
> No. I think that those kinds of decisions about actual behavior are
> always simply fundamentally wrong.
>
> Note that this is very different from having warnings about invalid
> use. THAT is correct. It may not warn in all configurations, but that
> doesn't matter: what matters is that it warns in common enough
> configurations that developers will catch it.

You wish. I just found a 7 year old bug in a 10G network driver which
surely would have been found if people would enable debug configs and
not just run the crap on their PREEMPT_NONE, all debug off kernel. And
that driver is not subject to bitrot, it gets regular bug fixes from
people who seem to care (distro folks).

> So having a warning in "might_sleep()" that doesn't always trigger,
> because you have a limited configuration that can't even detect the
> situation, that's fine and dandy and intentional.

and lets people get away with their crap.

> But having code like
>
>        if (can_schedule())
>            .. do something different ..
>
> is fundamentally complete and utter garbage.
>
> It's one thing if you test for "am I in hardware interrupt context".
> Those tests aren't great either, but at least they make sense.

They make sense in limited situations like exception handlers and such
which really have to know from which context an exception was raised.

But with the above reasoning such checks do not make sense in any other
general code. 'in hard interrupt context' is just another context where
you can't do stuff which you can do when in preemptible task context.

Most tests are way broader than a single context. in_interrupt() is true
for hard interrupt, soft interrupt delivery and all BH disabled
contexts, which is completely ill defined.

> But a driver - or some library routine - making a difference based on
> some nebulous "can I schedule" is fundamentally and basically WRONG.
>
> If some code changes behavior, it needs to be explicit to the *caller*
> of that code.

I'm fine with that, but then we have to be consequent and ban _all_ of
these and not just declare can_schedule() to be a bad one.

Thanks,

        tglx


^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-15 19:57             ` Thomas Gleixner
  0 siblings, 0 replies; 282+ messages in thread
From: Thomas Gleixner @ 2020-09-15 19:57 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, Linux-MM,
	open list:KERNEL SELFTEST FRAMEWORK, linux-hexagon, Will Deacon,
	Ingo Molnar, Anton Ivanov, linux-arch, Vincent Guittot,
	Herbert Xu, Brian Cain, Richard Weinberger, Russell King,
	Ard Biesheuvel, David Airlie, Ingo Molnar, Geert Uytterhoeven,
	Mel Gorman, intel-gfx, Matt Turner, Valentin Schneider,
	linux-xtensa, Shuah Khan, Paul E. McKenney, Jeff Dike, linux-um,
	Josh Triplett, Steven Rostedt, rcu, linux-m68k, Ivan Kokshaysky,
	Rodrigo Vivi, Dietmar Eggemann, Linux ARM, Richard Henderson,
	Chris Zankel, Max Filippov, LKML, alpha, Mathieu Desnoyers,
	Andrew Morton, Daniel Bristot de Oliveira

On Tue, Sep 15 2020 at 10:35, Linus Torvalds wrote:
> On Tue, Sep 15, 2020 at 1:39 AM Thomas Gleixner <tglx@linutronix.de> wrote:
>>
>> OTOH, having a working 'preemptible()' or maybe better named
>> 'can_schedule()' check makes tons of sense to make decisions about
>> allocation modes or other things.
>
> No. I think that those kinds of decisions about actual behavior are
> always simply fundamentally wrong.
>
> Note that this is very different from having warnings about invalid
> use. THAT is correct. It may not warn in all configurations, but that
> doesn't matter: what matters is that it warns in common enough
> configurations that developers will catch it.

You wish. I just found a 7 year old bug in a 10G network driver which
surely would have been found if people would enable debug configs and
not just run the crap on their PREEMPT_NONE, all debug off kernel. And
that driver is not subject to bitrot, it gets regular bug fixes from
people who seem to care (distro folks).

> So having a warning in "might_sleep()" that doesn't always trigger,
> because you have a limited configuration that can't even detect the
> situation, that's fine and dandy and intentional.

and lets people get away with their crap.

> But having code like
>
>        if (can_schedule())
>            .. do something different ..
>
> is fundamentally complete and utter garbage.
>
> It's one thing if you test for "am I in hardware interrupt context".
> Those tests aren't great either, but at least they make sense.

They make sense in limited situations like exception handlers and such
which really have to know from which context an exception was raised.

But with the above reasoning such checks do not make sense in any other
general code. 'in hard interrupt context' is just another context where
you can't do stuff which you can do when in preemptible task context.

Most tests are way broader than a single context. in_interrupt() is true
for hard interrupt, soft interrupt delivery and all BH disabled
contexts, which is completely ill defined.

> But a driver - or some library routine - making a difference based on
> some nebulous "can I schedule" is fundamentally and basically WRONG.
>
> If some code changes behavior, it needs to be explicit to the *caller*
> of that code.

I'm fine with that, but then we have to be consequent and ban _all_ of
these and not just declare can_schedule() to be a bad one.

Thanks,

        tglx
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [Intel-gfx] [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-15 19:57             ` Thomas Gleixner
  0 siblings, 0 replies; 282+ messages in thread
From: Thomas Gleixner @ 2020-09-15 19:57 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, Linux-MM,
	open list:KERNEL SELFTEST FRAMEWORK, linux-hexagon, Will Deacon,
	Ingo Molnar, Anton Ivanov, linux-arch, Herbert Xu, Brian Cain,
	Richard Weinberger, Russell King, Ard Biesheuvel, David Airlie,
	Ingo Molnar, Geert Uytterhoeven, Mel Gorman, intel-gfx,
	Matt Turner, Valentin Schneider, linux-xtensa, Shuah Khan,
	Paul E. McKenney, Jeff Dike, linux-um, Josh Triplett,
	Steven Rostedt, rcu, linux-m68k, Ivan Kokshaysky,
	Dietmar Eggemann, Linux ARM, Richard Henderson, Chris Zankel,
	Max Filippov, LKML, alpha, Mathieu Desnoyers, Andrew Morton,
	Daniel Bristot de Oliveira

On Tue, Sep 15 2020 at 10:35, Linus Torvalds wrote:
> On Tue, Sep 15, 2020 at 1:39 AM Thomas Gleixner <tglx@linutronix.de> wrote:
>>
>> OTOH, having a working 'preemptible()' or maybe better named
>> 'can_schedule()' check makes tons of sense to make decisions about
>> allocation modes or other things.
>
> No. I think that those kinds of decisions about actual behavior are
> always simply fundamentally wrong.
>
> Note that this is very different from having warnings about invalid
> use. THAT is correct. It may not warn in all configurations, but that
> doesn't matter: what matters is that it warns in common enough
> configurations that developers will catch it.

You wish. I just found a 7 year old bug in a 10G network driver which
surely would have been found if people would enable debug configs and
not just run the crap on their PREEMPT_NONE, all debug off kernel. And
that driver is not subject to bitrot, it gets regular bug fixes from
people who seem to care (distro folks).

> So having a warning in "might_sleep()" that doesn't always trigger,
> because you have a limited configuration that can't even detect the
> situation, that's fine and dandy and intentional.

and lets people get away with their crap.

> But having code like
>
>        if (can_schedule())
>            .. do something different ..
>
> is fundamentally complete and utter garbage.
>
> It's one thing if you test for "am I in hardware interrupt context".
> Those tests aren't great either, but at least they make sense.

They make sense in limited situations like exception handlers and such
which really have to know from which context an exception was raised.

But with the above reasoning such checks do not make sense in any other
general code. 'in hard interrupt context' is just another context where
you can't do stuff which you can do when in preemptible task context.

Most tests are way broader than a single context. in_interrupt() is true
for hard interrupt, soft interrupt delivery and all BH disabled
contexts, which is completely ill defined.

> But a driver - or some library routine - making a difference based on
> some nebulous "can I schedule" is fundamentally and basically WRONG.
>
> If some code changes behavior, it needs to be explicit to the *caller*
> of that code.

I'm fine with that, but then we have to be consequent and ban _all_ of
these and not just declare can_schedule() to be a bad one.

Thanks,

        tglx
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-15 19:57             ` Thomas Gleixner
  0 siblings, 0 replies; 282+ messages in thread
From: Thomas Gleixner @ 2020-09-15 19:57 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Ard Biesheuvel, Herbert Xu, LKML, linux-arch,
	Sebastian Andrzej Siewior, Valentin Schneider, Richard Henderson,
	Ivan Kokshaysky, Matt Turner, alpha, Jeff Dike,
	Richard Weinberger, Anton Ivanov, linux-um, Brian Cain,
	linux-hexagon, Geert Uytterhoeven, linux-m68k, Ingo Molnar,
	Peter Zijlstra, Juri Lelli, Vincent Guittot, Dietmar Eggemann,
	Steven Rostedt, Ben Segall, Mel Gorman,
	Daniel Bristot de Oliveira, Will Deacon, Andrew Morton, Linux-MM,
	Ingo Molnar, Russell King, Linux ARM, Chris Zankel, Max Filippov,
	linux-xtensa, Jani Nikula, Joonas Lahtinen, Rodrigo Vivi,
	David Airlie, Daniel Vetter, intel-gfx, dri-devel,
	Paul E. McKenney, Josh Triplett, Mathieu Desnoyers,
	Lai Jiangshan, Shuah Khan, rcu,
	open list:KERNEL SELFTEST FRAMEWORK

On Tue, Sep 15 2020 at 10:35, Linus Torvalds wrote:
> On Tue, Sep 15, 2020 at 1:39 AM Thomas Gleixner <tglx@linutronix.de> wrote:
>>
>> OTOH, having a working 'preemptible()' or maybe better named
>> 'can_schedule()' check makes tons of sense to make decisions about
>> allocation modes or other things.
>
> No. I think that those kinds of decisions about actual behavior are
> always simply fundamentally wrong.
>
> Note that this is very different from having warnings about invalid
> use. THAT is correct. It may not warn in all configurations, but that
> doesn't matter: what matters is that it warns in common enough
> configurations that developers will catch it.

You wish. I just found a 7 year old bug in a 10G network driver which
surely would have been found if people would enable debug configs and
not just run the crap on their PREEMPT_NONE, all debug off kernel. And
that driver is not subject to bitrot, it gets regular bug fixes from
people who seem to care (distro folks).

> So having a warning in "might_sleep()" that doesn't always trigger,
> because you have a limited configuration that can't even detect the
> situation, that's fine and dandy and intentional.

and lets people get away with their crap.

> But having code like
>
>        if (can_schedule())
>            .. do something different ..
>
> is fundamentally complete and utter garbage.
>
> It's one thing if you test for "am I in hardware interrupt context".
> Those tests aren't great either, but at least they make sense.

They make sense in limited situations like exception handlers and such
which really have to know from which context an exception was raised.

But with the above reasoning such checks do not make sense in any other
general code. 'in hard interrupt context' is just another context where
you can't do stuff which you can do when in preemptible task context.

Most tests are way broader than a single context. in_interrupt() is true
for hard interrupt, soft interrupt delivery and all BH disabled
contexts, which is completely ill defined.

> But a driver - or some library routine - making a difference based on
> some nebulous "can I schedule" is fundamentally and basically WRONG.
>
> If some code changes behavior, it needs to be explicit to the *caller*
> of that code.

I'm fine with that, but then we have to be consequent and ban _all_ of
these and not just declare can_schedule() to be a bad one.

Thanks,

        tglx


^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-15 19:57             ` Thomas Gleixner
  0 siblings, 0 replies; 282+ messages in thread
From: Thomas Gleixner @ 2020-09-15 19:57 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, Linux-MM,
	open list:KERNEL SELFTEST FRAMEWORK, linux-hexagon, Will Deacon,
	Ingo Molnar, Anton Ivanov, linux-arch, Herbert Xu, Brian Cain,
	Richard Weinberger, Russell King, Ard Biesheuvel, David Airlie,
	Ingo Molnar, Geert Uytterhoeven, Mel Gorman, intel-gfx

On Tue, Sep 15 2020 at 10:35, Linus Torvalds wrote:
> On Tue, Sep 15, 2020 at 1:39 AM Thomas Gleixner <tglx@linutronix.de> wrote:
>>
>> OTOH, having a working 'preemptible()' or maybe better named
>> 'can_schedule()' check makes tons of sense to make decisions about
>> allocation modes or other things.
>
> No. I think that those kinds of decisions about actual behavior are
> always simply fundamentally wrong.
>
> Note that this is very different from having warnings about invalid
> use. THAT is correct. It may not warn in all configurations, but that
> doesn't matter: what matters is that it warns in common enough
> configurations that developers will catch it.

You wish. I just found a 7 year old bug in a 10G network driver which
surely would have been found if people would enable debug configs and
not just run the crap on their PREEMPT_NONE, all debug off kernel. And
that driver is not subject to bitrot, it gets regular bug fixes from
people who seem to care (distro folks).

> So having a warning in "might_sleep()" that doesn't always trigger,
> because you have a limited configuration that can't even detect the
> situation, that's fine and dandy and intentional.

and lets people get away with their crap.

> But having code like
>
>        if (can_schedule())
>            .. do something different ..
>
> is fundamentally complete and utter garbage.
>
> It's one thing if you test for "am I in hardware interrupt context".
> Those tests aren't great either, but at least they make sense.

They make sense in limited situations like exception handlers and such
which really have to know from which context an exception was raised.

But with the above reasoning such checks do not make sense in any other
general code. 'in hard interrupt context' is just another context where
you can't do stuff which you can do when in preemptible task context.

Most tests are way broader than a single context. in_interrupt() is true
for hard interrupt, soft interrupt delivery and all BH disabled
contexts, which is completely ill defined.

> But a driver - or some library routine - making a difference based on
> some nebulous "can I schedule" is fundamentally and basically WRONG.
>
> If some code changes behavior, it needs to be explicit to the *caller*
> of that code.

I'm fine with that, but then we have to be consequent and ban _all_ of
these and not just declare can_schedule() to be a bad one.

Thanks,

        tglx

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
  2020-09-15 17:35           ` Linus Torvalds
                               ` (2 preceding siblings ...)
  (?)
@ 2020-09-16  7:37             ` Daniel Vetter
  -1 siblings, 0 replies; 282+ messages in thread
From: Daniel Vetter @ 2020-09-16  7:37 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Thomas Gleixner, Ard Biesheuvel, Herbert Xu, LKML, linux-arch,
	Sebastian Andrzej Siewior, Valentin Schneider, Richard Henderson,
	Ivan Kokshaysky, Matt Turner, alpha, Jeff Dike,
	Richard Weinberger, Anton Ivanov, linux-um, Brian Cain,
	linux-hexagon, Geert Uytterhoeven, linux-m68k, Ingo Molnar,
	Peter Zijlstra, Juri Lelli, Vincent Guittot, Dietmar Eggemann,
	Steven Rostedt, Ben Segall, Mel Gorman,
	Daniel Bristot de Oliveira, Will Deacon, Andrew Morton, Linux-MM,
	Ingo Molnar, Russell King, Linux ARM, Chris Zankel, Max Filippov,
	linux-xtensa, Jani Nikula, Joonas Lahtinen, Rodrigo Vivi,
	David Airlie, intel-gfx, dri-devel, Paul E. McKenney,
	Josh Triplett, Mathieu Desnoyers, Lai Jiangshan, Shuah Khan, rcu,
	open list:KERNEL SELFTEST FRAMEWORK

On Tue, Sep 15, 2020 at 7:35 PM Linus Torvalds
<torvalds@linux-foundation.org> wrote:
>
> On Tue, Sep 15, 2020 at 1:39 AM Thomas Gleixner <tglx@linutronix.de> wrote:
> >
> > OTOH, having a working 'preemptible()' or maybe better named
> > 'can_schedule()' check makes tons of sense to make decisions about
> > allocation modes or other things.
>
> No. I think that those kinds of decisions about actual behavior are
> always simply fundamentally wrong.
>
> Note that this is very different from having warnings about invalid
> use. THAT is correct. It may not warn in all configurations, but that
> doesn't matter: what matters is that it warns in common enough
> configurations that developers will catch it.
>
> So having a warning in "might_sleep()" that doesn't always trigger,
> because you have a limited configuration that can't even detect the
> situation, that's fine and dandy and intentional.
>
> But having code like
>
>        if (can_schedule())
>            .. do something different ..
>
> is fundamentally complete and utter garbage.
>
> It's one thing if you test for "am I in hardware interrupt context".
> Those tests aren't great either, but at least they make sense.
>
> But a driver - or some library routine - making a difference based on
> some nebulous "can I schedule" is fundamentally and basically WRONG.
>
> If some code changes behavior, it needs to be explicit to the *caller*
> of that code.
>
> So this is why GFP_ATOMIC is fine, but "if (!can_schedule())
> do_something_atomic()" is pure shite.
>
> And I am not IN THE LEAST interested in trying to help people doing
> pure shite. We need to fix them. Like the crypto code is getting
> fixed.

Just figured I'll throw my +1 in from reading too many (gpu) drivers.
Code that tries to cleverly adjust its behaviour depending upon the
context it's running in is harder to understand and blows up in more
interesting ways. We still have drm_can_sleep() and it's mostly just
used for debug code, and I've largely ended up just deleting
everything that used it because when you're driver is blowing up the
last thing you want is to realize your debug code and output can't be
relied upon. Or worse, that the only Oops you have is the one in the
debug code, because the real one scrolled away - the original idea
behind drm_can_sleep was to make all the modeset code work
automagically both in normal ioctl/kworker context and in the panic
handlers or kgdb callbacks. Wishful thinking at best.

Also at least for me that extends to everything, e.g. I much prefer
explicit spin_lock and spin_lock_irq vs magic spin_lock_irqsave for
locks shared with interrupt handlers, since the former two gives me
clear information from which contexts such function can be called.
Other end is the memalloc_no*_save/restore functions, where I recently
made a real big fool of myself because I didn't realize how much that
impacts everything that's run within - suddenly "GFP_KERNEL for small
stuff never fails" is wrong everywhere.

It's all great for debugging and sanity checks (and we run with all
that stuff enabled in our CI), but really semantic changes depending
upon magic context checks freak my out :-)
-Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-16  7:37             ` Daniel Vetter
  0 siblings, 0 replies; 282+ messages in thread
From: Daniel Vetter @ 2020-09-16  7:37 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Thomas Gleixner, Ard Biesheuvel, Herbert Xu, LKML, linux-arch,
	Sebastian Andrzej Siewior, Valentin Schneider, Richard Henderson,
	Ivan Kokshaysky, Matt Turner, alpha, Jeff Dike,
	Richard Weinberger, Anton Ivanov, linux-um, Brian Cain,
	linux-hexagon, Geert Uytterhoeven, linux-m68k, Ingo Molnar,
	Peter Zijlstra, Juri Lelli, Vincent Guittot, Dietmar Eggemann,
	Steven Rostedt, Ben Segall, Mel Gorman,
	Daniel Bristot de Oliveira, Will Deacon, Andrew Morton, Linux-MM,
	Ingo Molnar, Russell King, Linux ARM, Chris Zankel, Max Filippov,
	linux-xtensa, Jani Nikula, Joonas Lahtinen, Rodrigo Vivi,
	David Airlie, intel-gfx, dri-devel, Paul E. McKenney,
	Josh Triplett, Mathieu Desnoyers, Lai Jiangshan, Shuah Khan, rcu,
	open list:KERNEL SELFTEST FRAMEWORK

On Tue, Sep 15, 2020 at 7:35 PM Linus Torvalds
<torvalds@linux-foundation.org> wrote:
>
> On Tue, Sep 15, 2020 at 1:39 AM Thomas Gleixner <tglx@linutronix.de> wrote:
> >
> > OTOH, having a working 'preemptible()' or maybe better named
> > 'can_schedule()' check makes tons of sense to make decisions about
> > allocation modes or other things.
>
> No. I think that those kinds of decisions about actual behavior are
> always simply fundamentally wrong.
>
> Note that this is very different from having warnings about invalid
> use. THAT is correct. It may not warn in all configurations, but that
> doesn't matter: what matters is that it warns in common enough
> configurations that developers will catch it.
>
> So having a warning in "might_sleep()" that doesn't always trigger,
> because you have a limited configuration that can't even detect the
> situation, that's fine and dandy and intentional.
>
> But having code like
>
>        if (can_schedule())
>            .. do something different ..
>
> is fundamentally complete and utter garbage.
>
> It's one thing if you test for "am I in hardware interrupt context".
> Those tests aren't great either, but at least they make sense.
>
> But a driver - or some library routine - making a difference based on
> some nebulous "can I schedule" is fundamentally and basically WRONG.
>
> If some code changes behavior, it needs to be explicit to the *caller*
> of that code.
>
> So this is why GFP_ATOMIC is fine, but "if (!can_schedule())
> do_something_atomic()" is pure shite.
>
> And I am not IN THE LEAST interested in trying to help people doing
> pure shite. We need to fix them. Like the crypto code is getting
> fixed.

Just figured I'll throw my +1 in from reading too many (gpu) drivers.
Code that tries to cleverly adjust its behaviour depending upon the
context it's running in is harder to understand and blows up in more
interesting ways. We still have drm_can_sleep() and it's mostly just
used for debug code, and I've largely ended up just deleting
everything that used it because when you're driver is blowing up the
last thing you want is to realize your debug code and output can't be
relied upon. Or worse, that the only Oops you have is the one in the
debug code, because the real one scrolled away - the original idea
behind drm_can_sleep was to make all the modeset code work
automagically both in normal ioctl/kworker context and in the panic
handlers or kgdb callbacks. Wishful thinking at best.

Also at least for me that extends to everything, e.g. I much prefer
explicit spin_lock and spin_lock_irq vs magic spin_lock_irqsave for
locks shared with interrupt handlers, since the former two gives me
clear information from which contexts such function can be called.
Other end is the memalloc_no*_save/restore functions, where I recently
made a real big fool of myself because I didn't realize how much that
impacts everything that's run within - suddenly "GFP_KERNEL for small
stuff never fails" is wrong everywhere.

It's all great for debugging and sanity checks (and we run with all
that stuff enabled in our CI), but really semantic changes depending
upon magic context checks freak my out :-)
-Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch


^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-16  7:37             ` Daniel Vetter
  0 siblings, 0 replies; 282+ messages in thread
From: Daniel Vetter @ 2020-09-16  7:37 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, Linux-MM,
	open list:KERNEL SELFTEST FRAMEWORK, linux-hexagon, Will Deacon,
	Ingo Molnar, Anton Ivanov, linux-arch, Vincent Guittot,
	Herbert Xu, Brian Cain, Richard Weinberger, Russell King,
	Ard Biesheuvel, David Airlie, Ingo Molnar, Geert Uytterhoeven,
	Mel Gorman, intel-gfx, Matt Turner, Valentin Schneider,
	linux-xtensa, Shuah Khan, Paul E. McKenney, Jeff Dike, linux-um,
	Josh Triplett, Steven Rostedt, rcu, linux-m68k, Ivan Kokshaysky,
	Rodrigo Vivi, Thomas Gleixner, Dietmar Eggemann, Linux ARM,
	Richard Henderson, Chris Zankel, Max Filippov, LKML, alpha,
	Mathieu Desnoyers, Andrew Morton, Daniel Bristot de Oliveira

On Tue, Sep 15, 2020 at 7:35 PM Linus Torvalds
<torvalds@linux-foundation.org> wrote:
>
> On Tue, Sep 15, 2020 at 1:39 AM Thomas Gleixner <tglx@linutronix.de> wrote:
> >
> > OTOH, having a working 'preemptible()' or maybe better named
> > 'can_schedule()' check makes tons of sense to make decisions about
> > allocation modes or other things.
>
> No. I think that those kinds of decisions about actual behavior are
> always simply fundamentally wrong.
>
> Note that this is very different from having warnings about invalid
> use. THAT is correct. It may not warn in all configurations, but that
> doesn't matter: what matters is that it warns in common enough
> configurations that developers will catch it.
>
> So having a warning in "might_sleep()" that doesn't always trigger,
> because you have a limited configuration that can't even detect the
> situation, that's fine and dandy and intentional.
>
> But having code like
>
>        if (can_schedule())
>            .. do something different ..
>
> is fundamentally complete and utter garbage.
>
> It's one thing if you test for "am I in hardware interrupt context".
> Those tests aren't great either, but at least they make sense.
>
> But a driver - or some library routine - making a difference based on
> some nebulous "can I schedule" is fundamentally and basically WRONG.
>
> If some code changes behavior, it needs to be explicit to the *caller*
> of that code.
>
> So this is why GFP_ATOMIC is fine, but "if (!can_schedule())
> do_something_atomic()" is pure shite.
>
> And I am not IN THE LEAST interested in trying to help people doing
> pure shite. We need to fix them. Like the crypto code is getting
> fixed.

Just figured I'll throw my +1 in from reading too many (gpu) drivers.
Code that tries to cleverly adjust its behaviour depending upon the
context it's running in is harder to understand and blows up in more
interesting ways. We still have drm_can_sleep() and it's mostly just
used for debug code, and I've largely ended up just deleting
everything that used it because when you're driver is blowing up the
last thing you want is to realize your debug code and output can't be
relied upon. Or worse, that the only Oops you have is the one in the
debug code, because the real one scrolled away - the original idea
behind drm_can_sleep was to make all the modeset code work
automagically both in normal ioctl/kworker context and in the panic
handlers or kgdb callbacks. Wishful thinking at best.

Also at least for me that extends to everything, e.g. I much prefer
explicit spin_lock and spin_lock_irq vs magic spin_lock_irqsave for
locks shared with interrupt handlers, since the former two gives me
clear information from which contexts such function can be called.
Other end is the memalloc_no*_save/restore functions, where I recently
made a real big fool of myself because I didn't realize how much that
impacts everything that's run within - suddenly "GFP_KERNEL for small
stuff never fails" is wrong everywhere.

It's all great for debugging and sanity checks (and we run with all
that stuff enabled in our CI), but really semantic changes depending
upon magic context checks freak my out :-)
-Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [Intel-gfx] [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-16  7:37             ` Daniel Vetter
  0 siblings, 0 replies; 282+ messages in thread
From: Daniel Vetter @ 2020-09-16  7:37 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, Linux-MM,
	open list:KERNEL SELFTEST FRAMEWORK, linux-hexagon, Will Deacon,
	Ingo Molnar, Anton Ivanov, linux-arch, Herbert Xu, Brian Cain,
	Richard Weinberger, Russell King, Ard Biesheuvel, David Airlie,
	Ingo Molnar, Geert Uytterhoeven, Mel Gorman, intel-gfx,
	Matt Turner, Valentin Schneider, linux-xtensa, Shuah Khan,
	Paul E. McKenney, Jeff Dike, linux-um, Josh Triplett,
	Steven Rostedt, rcu, linux-m68k, Ivan Kokshaysky,
	Thomas Gleixner, Dietmar Eggemann, Linux ARM, Richard Henderson,
	Chris Zankel, Max Filippov, LKML, alpha, Mathieu Desnoyers,
	Andrew Morton, Daniel Bristot de Oliveira

On Tue, Sep 15, 2020 at 7:35 PM Linus Torvalds
<torvalds@linux-foundation.org> wrote:
>
> On Tue, Sep 15, 2020 at 1:39 AM Thomas Gleixner <tglx@linutronix.de> wrote:
> >
> > OTOH, having a working 'preemptible()' or maybe better named
> > 'can_schedule()' check makes tons of sense to make decisions about
> > allocation modes or other things.
>
> No. I think that those kinds of decisions about actual behavior are
> always simply fundamentally wrong.
>
> Note that this is very different from having warnings about invalid
> use. THAT is correct. It may not warn in all configurations, but that
> doesn't matter: what matters is that it warns in common enough
> configurations that developers will catch it.
>
> So having a warning in "might_sleep()" that doesn't always trigger,
> because you have a limited configuration that can't even detect the
> situation, that's fine and dandy and intentional.
>
> But having code like
>
>        if (can_schedule())
>            .. do something different ..
>
> is fundamentally complete and utter garbage.
>
> It's one thing if you test for "am I in hardware interrupt context".
> Those tests aren't great either, but at least they make sense.
>
> But a driver - or some library routine - making a difference based on
> some nebulous "can I schedule" is fundamentally and basically WRONG.
>
> If some code changes behavior, it needs to be explicit to the *caller*
> of that code.
>
> So this is why GFP_ATOMIC is fine, but "if (!can_schedule())
> do_something_atomic()" is pure shite.
>
> And I am not IN THE LEAST interested in trying to help people doing
> pure shite. We need to fix them. Like the crypto code is getting
> fixed.

Just figured I'll throw my +1 in from reading too many (gpu) drivers.
Code that tries to cleverly adjust its behaviour depending upon the
context it's running in is harder to understand and blows up in more
interesting ways. We still have drm_can_sleep() and it's mostly just
used for debug code, and I've largely ended up just deleting
everything that used it because when you're driver is blowing up the
last thing you want is to realize your debug code and output can't be
relied upon. Or worse, that the only Oops you have is the one in the
debug code, because the real one scrolled away - the original idea
behind drm_can_sleep was to make all the modeset code work
automagically both in normal ioctl/kworker context and in the panic
handlers or kgdb callbacks. Wishful thinking at best.

Also at least for me that extends to everything, e.g. I much prefer
explicit spin_lock and spin_lock_irq vs magic spin_lock_irqsave for
locks shared with interrupt handlers, since the former two gives me
clear information from which contexts such function can be called.
Other end is the memalloc_no*_save/restore functions, where I recently
made a real big fool of myself because I didn't realize how much that
impacts everything that's run within - suddenly "GFP_KERNEL for small
stuff never fails" is wrong everywhere.

It's all great for debugging and sanity checks (and we run with all
that stuff enabled in our CI), but really semantic changes depending
upon magic context checks freak my out :-)
-Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-16  7:37             ` Daniel Vetter
  0 siblings, 0 replies; 282+ messages in thread
From: Daniel Vetter @ 2020-09-16  7:37 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, Linux-MM,
	open list:KERNEL SELFTEST FRAMEWORK, linux-hexagon, Will Deacon,
	Ingo Molnar, Anton Ivanov, linux-arch, Herbert Xu, Brian Cain,
	Richard Weinberger, Russell King, Ard Biesheuvel, David Airlie,
	Ingo Molnar, Geert Uytterhoeven, Mel Gorman, intel-gfx

On Tue, Sep 15, 2020 at 7:35 PM Linus Torvalds
<torvalds@linux-foundation.org> wrote:
>
> On Tue, Sep 15, 2020 at 1:39 AM Thomas Gleixner <tglx@linutronix.de> wrote:
> >
> > OTOH, having a working 'preemptible()' or maybe better named
> > 'can_schedule()' check makes tons of sense to make decisions about
> > allocation modes or other things.
>
> No. I think that those kinds of decisions about actual behavior are
> always simply fundamentally wrong.
>
> Note that this is very different from having warnings about invalid
> use. THAT is correct. It may not warn in all configurations, but that
> doesn't matter: what matters is that it warns in common enough
> configurations that developers will catch it.
>
> So having a warning in "might_sleep()" that doesn't always trigger,
> because you have a limited configuration that can't even detect the
> situation, that's fine and dandy and intentional.
>
> But having code like
>
>        if (can_schedule())
>            .. do something different ..
>
> is fundamentally complete and utter garbage.
>
> It's one thing if you test for "am I in hardware interrupt context".
> Those tests aren't great either, but at least they make sense.
>
> But a driver - or some library routine - making a difference based on
> some nebulous "can I schedule" is fundamentally and basically WRONG.
>
> If some code changes behavior, it needs to be explicit to the *caller*
> of that code.
>
> So this is why GFP_ATOMIC is fine, but "if (!can_schedule())
> do_something_atomic()" is pure shite.
>
> And I am not IN THE LEAST interested in trying to help people doing
> pure shite. We need to fix them. Like the crypto code is getting
> fixed.

Just figured I'll throw my +1 in from reading too many (gpu) drivers.
Code that tries to cleverly adjust its behaviour depending upon the
context it's running in is harder to understand and blows up in more
interesting ways. We still have drm_can_sleep() and it's mostly just
used for debug code, and I've largely ended up just deleting
everything that used it because when you're driver is blowing up the
last thing you want is to realize your debug code and output can't be
relied upon. Or worse, that the only Oops you have is the one in the
debug code, because the real one scrolled away - the original idea
behind drm_can_sleep was to make all the modeset code work
automagically both in normal ioctl/kworker context and in the panic
handlers or kgdb callbacks. Wishful thinking at best.

Also at least for me that extends to everything, e.g. I much prefer
explicit spin_lock and spin_lock_irq vs magic spin_lock_irqsave for
locks shared with interrupt handlers, since the former two gives me
clear information from which contexts such function can be called.
Other end is the memalloc_no*_save/restore functions, where I recently
made a real big fool of myself because I didn't realize how much that
impacts everything that's run within - suddenly "GFP_KERNEL for small
stuff never fails" is wrong everywhere.

It's all great for debugging and sanity checks (and we run with all
that stuff enabled in our CI), but really semantic changes depending
upon magic context checks freak my out :-)
-Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 03/13] preempt: Clenaup PREEMPT_COUNT leftovers
  2020-09-14 20:42   ` Thomas Gleixner
  (?)
  (?)
@ 2020-09-16 10:56     ` Valentin Schneider
  -1 siblings, 0 replies; 282+ messages in thread
From: Valentin Schneider @ 2020-09-16 10:56 UTC (permalink / raw)
  To: Thomas Gleixner
  Cc: LKML, linux-arch, Linus Torvalds, Sebastian Andrzej Siewior,
	Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
	Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman,
	Daniel Bristot de Oliveira, Richard Henderson, Ivan Kokshaysky,
	Matt Turner, linux-alpha, Jeff Dike, Richard Weinberger,
	Anton Ivanov, linux-um, Brian Cain, linux-hexagon,
	Geert Uytterhoeven, linux-m68k, Will Deacon, Andrew Morton,
	linux-mm, Ingo Molnar, Russell King, linux-arm-kernel,
	Chris Zankel, Max Filippov, linux-xtensa, Jani Nikula,
	Joonas Lahtinen, Rodrigo Vivi, David Airlie, Daniel Vetter,
	intel-gfx, dri-devel, Paul E. McKenney, Josh Triplett,
	Mathieu Desnoyers, Lai Jiangshan, Shuah Khan, rcu,
	linux-kselftest


On 14/09/20 21:42, Thomas Gleixner wrote:
> CONFIG_PREEMPT_COUNT is now unconditionally enabled and will be
> removed. Cleanup the leftovers before doing so.
>
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> Cc: Ingo Molnar <mingo@kernel.org>
> Cc: Peter Zijlstra <peterz@infradead.org>
> Cc: Juri Lelli <juri.lelli@redhat.com>
> Cc: Vincent Guittot <vincent.guittot@linaro.org>
> Cc: Dietmar Eggemann <dietmar.eggemann@arm.com>
> Cc: Steven Rostedt <rostedt@goodmis.org>
> Cc: Ben Segall <bsegall@google.com>
> Cc: Mel Gorman <mgorman@suse.de>
> Cc: Daniel Bristot de Oliveira <bristot@redhat.com>

Reviewed-by: Valentin Schneider <valentin.schneider@arm.com>

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 03/13] preempt: Clenaup PREEMPT_COUNT leftovers
@ 2020-09-16 10:56     ` Valentin Schneider
  0 siblings, 0 replies; 282+ messages in thread
From: Valentin Schneider @ 2020-09-16 10:56 UTC (permalink / raw)
  To: Thomas Gleixner
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Joonas Lahtinen, Lai Jiangshan, dri-devel, Ben Segall, linux-mm,
	linux-kselftest, linux-hexagon, Will Deacon, Ingo Molnar,
	Anton Ivanov, linux-arch, Vincent Guittot, Brian Cain,
	Richard Weinberger, Russell King, David Airlie, Ingo Molnar,
	Geert Uytterhoeven, Mel Gorman, Matt Turner, intel-gfx,
	linux-xtensa, Shuah Khan, Paul E. McKenney, Jeff Dike, linux-um,
	Josh Triplett, Steven Rostedt, rcu, Linus Torvalds,
	Ivan Kokshaysky, Jani Nikula, Rodrigo Vivi, Dietmar Eggemann,
	linux-arm-kernel, Richard Henderson, Chris Zankel, Max Filippov,
	linux-m68k, LKML, Daniel Vetter, linux-alpha, Mathieu Desnoyers,
	Andrew Morton, Daniel Bristot de Oliveira


On 14/09/20 21:42, Thomas Gleixner wrote:
> CONFIG_PREEMPT_COUNT is now unconditionally enabled and will be
> removed. Cleanup the leftovers before doing so.
>
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> Cc: Ingo Molnar <mingo@kernel.org>
> Cc: Peter Zijlstra <peterz@infradead.org>
> Cc: Juri Lelli <juri.lelli@redhat.com>
> Cc: Vincent Guittot <vincent.guittot@linaro.org>
> Cc: Dietmar Eggemann <dietmar.eggemann@arm.com>
> Cc: Steven Rostedt <rostedt@goodmis.org>
> Cc: Ben Segall <bsegall@google.com>
> Cc: Mel Gorman <mgorman@suse.de>
> Cc: Daniel Bristot de Oliveira <bristot@redhat.com>

Reviewed-by: Valentin Schneider <valentin.schneider@arm.com>

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [Intel-gfx] [patch 03/13] preempt: Clenaup PREEMPT_COUNT leftovers
@ 2020-09-16 10:56     ` Valentin Schneider
  0 siblings, 0 replies; 282+ messages in thread
From: Valentin Schneider @ 2020-09-16 10:56 UTC (permalink / raw)
  To: Thomas Gleixner
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, linux-mm, linux-kselftest,
	linux-hexagon, Will Deacon, Ingo Molnar, Anton Ivanov,
	linux-arch, Brian Cain, Richard Weinberger, Russell King,
	David Airlie, Ingo Molnar, Geert Uytterhoeven, Mel Gorman,
	Matt Turner, intel-gfx, linux-xtensa, Shuah Khan,
	Paul E. McKenney, Jeff Dike, linux-um, Josh Triplett,
	Steven Rostedt, rcu, Linus Torvalds, Ivan Kokshaysky,
	Dietmar Eggemann, linux-arm-kernel, Richard Henderson,
	Chris Zankel, Max Filippov, linux-m68k, LKML, linux-alpha,
	Mathieu Desnoyers, Andrew Morton, Daniel Bristot de Oliveira


On 14/09/20 21:42, Thomas Gleixner wrote:
> CONFIG_PREEMPT_COUNT is now unconditionally enabled and will be
> removed. Cleanup the leftovers before doing so.
>
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> Cc: Ingo Molnar <mingo@kernel.org>
> Cc: Peter Zijlstra <peterz@infradead.org>
> Cc: Juri Lelli <juri.lelli@redhat.com>
> Cc: Vincent Guittot <vincent.guittot@linaro.org>
> Cc: Dietmar Eggemann <dietmar.eggemann@arm.com>
> Cc: Steven Rostedt <rostedt@goodmis.org>
> Cc: Ben Segall <bsegall@google.com>
> Cc: Mel Gorman <mgorman@suse.de>
> Cc: Daniel Bristot de Oliveira <bristot@redhat.com>

Reviewed-by: Valentin Schneider <valentin.schneider@arm.com>
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 03/13] preempt: Clenaup PREEMPT_COUNT leftovers
@ 2020-09-16 10:56     ` Valentin Schneider
  0 siblings, 0 replies; 282+ messages in thread
From: Valentin Schneider @ 2020-09-16 10:56 UTC (permalink / raw)
  To: Thomas Gleixner
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Joonas Lahtinen, Lai Jiangshan, dri-devel, Ben Segall, linux-mm,
	linux-kselftest, linux-hexagon, Will Deacon, Ingo Molnar,
	Anton Ivanov, linux-arch, Vincent Guittot, Brian Cain,
	Richard Weinberger, Russell King, David Airlie, Ingo Molnar,
	Geert Uytterhoeven, Mel Gorman, Matt Turner, intel-gfx,
	linux-xtensa


On 14/09/20 21:42, Thomas Gleixner wrote:
> CONFIG_PREEMPT_COUNT is now unconditionally enabled and will be
> removed. Cleanup the leftovers before doing so.
>
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> Cc: Ingo Molnar <mingo@kernel.org>
> Cc: Peter Zijlstra <peterz@infradead.org>
> Cc: Juri Lelli <juri.lelli@redhat.com>
> Cc: Vincent Guittot <vincent.guittot@linaro.org>
> Cc: Dietmar Eggemann <dietmar.eggemann@arm.com>
> Cc: Steven Rostedt <rostedt@goodmis.org>
> Cc: Ben Segall <bsegall@google.com>
> Cc: Mel Gorman <mgorman@suse.de>
> Cc: Daniel Bristot de Oliveira <bristot@redhat.com>

Reviewed-by: Valentin Schneider <valentin.schneider@arm.com>

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 08/13] sched: Clenaup PREEMPT_COUNT leftovers
  2020-09-14 20:42   ` Thomas Gleixner
  (?)
  (?)
@ 2020-09-16 10:56     ` Valentin Schneider
  -1 siblings, 0 replies; 282+ messages in thread
From: Valentin Schneider @ 2020-09-16 10:56 UTC (permalink / raw)
  To: Thomas Gleixner
  Cc: LKML, linux-arch, Linus Torvalds, Sebastian Andrzej Siewior,
	Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
	Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman,
	Daniel Bristot de Oliveira, Richard Henderson, Ivan Kokshaysky,
	Matt Turner, linux-alpha, Jeff Dike, Richard Weinberger,
	Anton Ivanov, linux-um, Brian Cain, linux-hexagon,
	Geert Uytterhoeven, linux-m68k, Ingo Molnar, Will Deacon,
	Andrew Morton, linux-mm, Russell King, linux-arm-kernel,
	Chris Zankel, Max Filippov, linux-xtensa, Jani Nikula,
	Joonas Lahtinen, Rodrigo Vivi, David Airlie, Daniel Vetter,
	intel-gfx, dri-devel, Paul E. McKenney, Josh Triplett,
	Mathieu Desnoyers, Lai Jiangshan, Shuah Khan, rcu,
	linux-kselftest


On 14/09/20 21:42, Thomas Gleixner wrote:
> CONFIG_PREEMPT_COUNT is now unconditionally enabled and will be
> removed. Cleanup the leftovers before doing so.
>
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> Cc: Ingo Molnar <mingo@redhat.com>
> Cc: Peter Zijlstra <peterz@infradead.org>
> Cc: Juri Lelli <juri.lelli@redhat.com>
> Cc: Vincent Guittot <vincent.guittot@linaro.org>
> Cc: Dietmar Eggemann <dietmar.eggemann@arm.com>
> Cc: Steven Rostedt <rostedt@goodmis.org>
> Cc: Ben Segall <bsegall@google.com>
> Cc: Mel Gorman <mgorman@suse.de>
> Cc: Daniel Bristot de Oliveira <bristot@redhat.com>

Small nit below;

Reviewed-by: Valentin Schneider <valentin.schneider@arm.com>

> ---
>  kernel/sched/core.c |    6 +-----
>  lib/Kconfig.debug   |    1 -
>  2 files changed, 1 insertion(+), 6 deletions(-)
>
> --- a/kernel/sched/core.c
> +++ b/kernel/sched/core.c
> @@ -3706,8 +3706,7 @@ asmlinkage __visible void schedule_tail(
>        * finish_task_switch() for details.
>        *
>        * finish_task_switch() will drop rq->lock() and lower preempt_count
> -	 * and the preempt_enable() will end up enabling preemption (on
> -	 * PREEMPT_COUNT kernels).

I suppose this wanted to be s/PREEMPT_COUNT/PREEMPT/ in the first place,
which ought to be still relevant.

> +	 * and the preempt_enable() will end up enabling preemption.
>        */
>
>       rq = finish_task_switch(prev);

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 08/13] sched: Clenaup PREEMPT_COUNT leftovers
@ 2020-09-16 10:56     ` Valentin Schneider
  0 siblings, 0 replies; 282+ messages in thread
From: Valentin Schneider @ 2020-09-16 10:56 UTC (permalink / raw)
  To: Thomas Gleixner
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Joonas Lahtinen, Lai Jiangshan, dri-devel, Ben Segall, linux-mm,
	linux-kselftest, linux-hexagon, Will Deacon, Ingo Molnar,
	Anton Ivanov, linux-arch, Vincent Guittot, Brian Cain,
	Richard Weinberger, Russell King, David Airlie, Ingo Molnar,
	Geert Uytterhoeven, Mel Gorman, Matt Turner, intel-gfx,
	linux-xtensa, Shuah Khan, Paul E. McKenney, Jeff Dike, linux-um,
	Josh Triplett, Steven Rostedt, rcu, Linus Torvalds,
	Ivan Kokshaysky, Jani Nikula, Rodrigo Vivi, Dietmar Eggemann,
	linux-arm-kernel, Richard Henderson, Chris Zankel, Max Filippov,
	linux-m68k, LKML, Daniel Vetter, linux-alpha, Mathieu Desnoyers,
	Andrew Morton, Daniel Bristot de Oliveira


On 14/09/20 21:42, Thomas Gleixner wrote:
> CONFIG_PREEMPT_COUNT is now unconditionally enabled and will be
> removed. Cleanup the leftovers before doing so.
>
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> Cc: Ingo Molnar <mingo@redhat.com>
> Cc: Peter Zijlstra <peterz@infradead.org>
> Cc: Juri Lelli <juri.lelli@redhat.com>
> Cc: Vincent Guittot <vincent.guittot@linaro.org>
> Cc: Dietmar Eggemann <dietmar.eggemann@arm.com>
> Cc: Steven Rostedt <rostedt@goodmis.org>
> Cc: Ben Segall <bsegall@google.com>
> Cc: Mel Gorman <mgorman@suse.de>
> Cc: Daniel Bristot de Oliveira <bristot@redhat.com>

Small nit below;

Reviewed-by: Valentin Schneider <valentin.schneider@arm.com>

> ---
>  kernel/sched/core.c |    6 +-----
>  lib/Kconfig.debug   |    1 -
>  2 files changed, 1 insertion(+), 6 deletions(-)
>
> --- a/kernel/sched/core.c
> +++ b/kernel/sched/core.c
> @@ -3706,8 +3706,7 @@ asmlinkage __visible void schedule_tail(
>        * finish_task_switch() for details.
>        *
>        * finish_task_switch() will drop rq->lock() and lower preempt_count
> -	 * and the preempt_enable() will end up enabling preemption (on
> -	 * PREEMPT_COUNT kernels).

I suppose this wanted to be s/PREEMPT_COUNT/PREEMPT/ in the first place,
which ought to be still relevant.

> +	 * and the preempt_enable() will end up enabling preemption.
>        */
>
>       rq = finish_task_switch(prev);

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [Intel-gfx] [patch 08/13] sched: Clenaup PREEMPT_COUNT leftovers
@ 2020-09-16 10:56     ` Valentin Schneider
  0 siblings, 0 replies; 282+ messages in thread
From: Valentin Schneider @ 2020-09-16 10:56 UTC (permalink / raw)
  To: Thomas Gleixner
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, linux-mm, linux-kselftest,
	linux-hexagon, Will Deacon, Ingo Molnar, Anton Ivanov,
	linux-arch, Brian Cain, Richard Weinberger, Russell King,
	David Airlie, Ingo Molnar, Geert Uytterhoeven, Mel Gorman,
	Matt Turner, intel-gfx, linux-xtensa, Shuah Khan,
	Paul E. McKenney, Jeff Dike, linux-um, Josh Triplett,
	Steven Rostedt, rcu, Linus Torvalds, Ivan Kokshaysky,
	Dietmar Eggemann, linux-arm-kernel, Richard Henderson,
	Chris Zankel, Max Filippov, linux-m68k, LKML, linux-alpha,
	Mathieu Desnoyers, Andrew Morton, Daniel Bristot de Oliveira


On 14/09/20 21:42, Thomas Gleixner wrote:
> CONFIG_PREEMPT_COUNT is now unconditionally enabled and will be
> removed. Cleanup the leftovers before doing so.
>
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> Cc: Ingo Molnar <mingo@redhat.com>
> Cc: Peter Zijlstra <peterz@infradead.org>
> Cc: Juri Lelli <juri.lelli@redhat.com>
> Cc: Vincent Guittot <vincent.guittot@linaro.org>
> Cc: Dietmar Eggemann <dietmar.eggemann@arm.com>
> Cc: Steven Rostedt <rostedt@goodmis.org>
> Cc: Ben Segall <bsegall@google.com>
> Cc: Mel Gorman <mgorman@suse.de>
> Cc: Daniel Bristot de Oliveira <bristot@redhat.com>

Small nit below;

Reviewed-by: Valentin Schneider <valentin.schneider@arm.com>

> ---
>  kernel/sched/core.c |    6 +-----
>  lib/Kconfig.debug   |    1 -
>  2 files changed, 1 insertion(+), 6 deletions(-)
>
> --- a/kernel/sched/core.c
> +++ b/kernel/sched/core.c
> @@ -3706,8 +3706,7 @@ asmlinkage __visible void schedule_tail(
>        * finish_task_switch() for details.
>        *
>        * finish_task_switch() will drop rq->lock() and lower preempt_count
> -	 * and the preempt_enable() will end up enabling preemption (on
> -	 * PREEMPT_COUNT kernels).

I suppose this wanted to be s/PREEMPT_COUNT/PREEMPT/ in the first place,
which ought to be still relevant.

> +	 * and the preempt_enable() will end up enabling preemption.
>        */
>
>       rq = finish_task_switch(prev);
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 08/13] sched: Clenaup PREEMPT_COUNT leftovers
@ 2020-09-16 10:56     ` Valentin Schneider
  0 siblings, 0 replies; 282+ messages in thread
From: Valentin Schneider @ 2020-09-16 10:56 UTC (permalink / raw)
  To: Thomas Gleixner
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Joonas Lahtinen, Lai Jiangshan, dri-devel, Ben Segall, linux-mm,
	linux-kselftest, linux-hexagon, Will Deacon, Ingo Molnar,
	Anton Ivanov, linux-arch, Vincent Guittot, Brian Cain,
	Richard Weinberger, Russell King, David Airlie, Ingo Molnar,
	Geert Uytterhoeven, Mel Gorman, Matt Turner, intel-gfx,
	linux-xtensa


On 14/09/20 21:42, Thomas Gleixner wrote:
> CONFIG_PREEMPT_COUNT is now unconditionally enabled and will be
> removed. Cleanup the leftovers before doing so.
>
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> Cc: Ingo Molnar <mingo@redhat.com>
> Cc: Peter Zijlstra <peterz@infradead.org>
> Cc: Juri Lelli <juri.lelli@redhat.com>
> Cc: Vincent Guittot <vincent.guittot@linaro.org>
> Cc: Dietmar Eggemann <dietmar.eggemann@arm.com>
> Cc: Steven Rostedt <rostedt@goodmis.org>
> Cc: Ben Segall <bsegall@google.com>
> Cc: Mel Gorman <mgorman@suse.de>
> Cc: Daniel Bristot de Oliveira <bristot@redhat.com>

Small nit below;

Reviewed-by: Valentin Schneider <valentin.schneider@arm.com>

> ---
>  kernel/sched/core.c |    6 +-----
>  lib/Kconfig.debug   |    1 -
>  2 files changed, 1 insertion(+), 6 deletions(-)
>
> --- a/kernel/sched/core.c
> +++ b/kernel/sched/core.c
> @@ -3706,8 +3706,7 @@ asmlinkage __visible void schedule_tail(
>        * finish_task_switch() for details.
>        *
>        * finish_task_switch() will drop rq->lock() and lower preempt_count
> -	 * and the preempt_enable() will end up enabling preemption (on
> -	 * PREEMPT_COUNT kernels).

I suppose this wanted to be s/PREEMPT_COUNT/PREEMPT/ in the first place,
which ought to be still relevant.

> +	 * and the preempt_enable() will end up enabling preemption.
>        */
>
>       rq = finish_task_switch(prev);

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
  2020-09-16  7:37             ` Daniel Vetter
                                 ` (2 preceding siblings ...)
  (?)
@ 2020-09-16 15:29               ` Paul E. McKenney
  -1 siblings, 0 replies; 282+ messages in thread
From: Paul E. McKenney @ 2020-09-16 15:29 UTC (permalink / raw)
  To: Daniel Vetter
  Cc: Linus Torvalds, Thomas Gleixner, Ard Biesheuvel, Herbert Xu,
	LKML, linux-arch, Sebastian Andrzej Siewior, Valentin Schneider,
	Richard Henderson, Ivan Kokshaysky, Matt Turner, alpha,
	Jeff Dike, Richard Weinberger, Anton Ivanov, linux-um,
	Brian Cain, linux-hexagon, Geert Uytterhoeven, linux-m68k,
	Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
	Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman,
	Daniel Bristot de Oliveira, Will Deacon, Andrew Morton, Linux-MM,
	Ingo Molnar, Russell King, Linux ARM, Chris Zankel, Max Filippov,
	linux-xtensa, Jani Nikula, Joonas Lahtinen, Rodrigo Vivi,
	David Airlie, intel-gfx, dri-devel, Josh Triplett,
	Mathieu Desnoyers, Lai Jiangshan, Shuah Khan, rcu,
	open list:KERNEL SELFTEST FRAMEWORK

On Wed, Sep 16, 2020 at 09:37:17AM +0200, Daniel Vetter wrote:
> On Tue, Sep 15, 2020 at 7:35 PM Linus Torvalds
> <torvalds@linux-foundation.org> wrote:
> >
> > On Tue, Sep 15, 2020 at 1:39 AM Thomas Gleixner <tglx@linutronix.de> wrote:
> > >
> > > OTOH, having a working 'preemptible()' or maybe better named
> > > 'can_schedule()' check makes tons of sense to make decisions about
> > > allocation modes or other things.
> >
> > No. I think that those kinds of decisions about actual behavior are
> > always simply fundamentally wrong.
> >
> > Note that this is very different from having warnings about invalid
> > use. THAT is correct. It may not warn in all configurations, but that
> > doesn't matter: what matters is that it warns in common enough
> > configurations that developers will catch it.
> >
> > So having a warning in "might_sleep()" that doesn't always trigger,
> > because you have a limited configuration that can't even detect the
> > situation, that's fine and dandy and intentional.
> >
> > But having code like
> >
> >        if (can_schedule())
> >            .. do something different ..
> >
> > is fundamentally complete and utter garbage.
> >
> > It's one thing if you test for "am I in hardware interrupt context".
> > Those tests aren't great either, but at least they make sense.
> >
> > But a driver - or some library routine - making a difference based on
> > some nebulous "can I schedule" is fundamentally and basically WRONG.
> >
> > If some code changes behavior, it needs to be explicit to the *caller*
> > of that code.
> >
> > So this is why GFP_ATOMIC is fine, but "if (!can_schedule())
> > do_something_atomic()" is pure shite.
> >
> > And I am not IN THE LEAST interested in trying to help people doing
> > pure shite. We need to fix them. Like the crypto code is getting
> > fixed.
> 
> Just figured I'll throw my +1 in from reading too many (gpu) drivers.
> Code that tries to cleverly adjust its behaviour depending upon the
> context it's running in is harder to understand and blows up in more
> interesting ways. We still have drm_can_sleep() and it's mostly just
> used for debug code, and I've largely ended up just deleting
> everything that used it because when you're driver is blowing up the
> last thing you want is to realize your debug code and output can't be
> relied upon. Or worse, that the only Oops you have is the one in the
> debug code, because the real one scrolled away - the original idea
> behind drm_can_sleep was to make all the modeset code work
> automagically both in normal ioctl/kworker context and in the panic
> handlers or kgdb callbacks. Wishful thinking at best.
> 
> Also at least for me that extends to everything, e.g. I much prefer
> explicit spin_lock and spin_lock_irq vs magic spin_lock_irqsave for
> locks shared with interrupt handlers, since the former two gives me
> clear information from which contexts such function can be called.
> Other end is the memalloc_no*_save/restore functions, where I recently
> made a real big fool of myself because I didn't realize how much that
> impacts everything that's run within - suddenly "GFP_KERNEL for small
> stuff never fails" is wrong everywhere.
> 
> It's all great for debugging and sanity checks (and we run with all
> that stuff enabled in our CI), but really semantic changes depending
> upon magic context checks freak my out :-)

All fair, but some of us need to write code that must handle being
invoked from a wide variety of contexts.  Now perhaps you like the idea of
call_rcu() for schedulable contexts, call_rcu_nosched() when preemption
is disabled, call_rcu_irqs_are_disabled() when interrupts are disabled,
call_rcu_raw_atomic() from contexts where (for example) raw spinlocks
are held, and so on.  However, from what I can see, most people instead
consistently prefer that the RCU API instead be consolidated.

Some in-flight cache-efficiency work for kvfree_rcu() and call_rcu()
needs to be able to allocate memory occasionally.  It can do that when
invoked from some contexts, but not when invoked from others.  Right now,
in !PREEMPT kernels, it cannot tell, and must either do things to the
memory allocators that some of the MM hate or must unnecessarily invoke
workqueues.  Thomas's patches would allow the code to just allocate in
the common case when these primitives are invoked from contexts where
allocation is permitted.

If we want to restrict access to the can_schedule() or whatever primitive,
fine and good.  We can add a check to checkpatch.pl, for example.  Maybe
we can go back to the old brlock approach of requiring certain people's
review for each addition to the kernel.

But there really are use cases that it would greatly help.

							Thanx, Paul

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-16 15:29               ` Paul E. McKenney
  0 siblings, 0 replies; 282+ messages in thread
From: Paul E. McKenney @ 2020-09-16 15:29 UTC (permalink / raw)
  To: Daniel Vetter
  Cc: Linus Torvalds, Thomas Gleixner, Ard Biesheuvel, Herbert Xu,
	LKML, linux-arch, Sebastian Andrzej Siewior, Valentin Schneider,
	Richard Henderson, Ivan Kokshaysky, Matt Turner, alpha,
	Jeff Dike, Richard Weinberger, Anton Ivanov, linux-um,
	Brian Cain, linux-hexagon, Geert Uytterhoeven, linux-m68k,
	Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
	Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman,
	Daniel Bristot de Oliveira, Will Deacon, Andrew Morton, Linux-MM,
	Ingo Molnar, Russell King, Linux ARM, Chris Zankel, Max Filippov,
	linux-xtensa, Jani Nikula, Joonas Lahtinen, Rodrigo Vivi,
	David Airlie, intel-gfx, dri-devel, Josh Triplett,
	Mathieu Desnoyers, Lai Jiangshan, Shuah Khan, rcu,
	open list:KERNEL SELFTEST FRAMEWORK

On Wed, Sep 16, 2020 at 09:37:17AM +0200, Daniel Vetter wrote:
> On Tue, Sep 15, 2020 at 7:35 PM Linus Torvalds
> <torvalds@linux-foundation.org> wrote:
> >
> > On Tue, Sep 15, 2020 at 1:39 AM Thomas Gleixner <tglx@linutronix.de> wrote:
> > >
> > > OTOH, having a working 'preemptible()' or maybe better named
> > > 'can_schedule()' check makes tons of sense to make decisions about
> > > allocation modes or other things.
> >
> > No. I think that those kinds of decisions about actual behavior are
> > always simply fundamentally wrong.
> >
> > Note that this is very different from having warnings about invalid
> > use. THAT is correct. It may not warn in all configurations, but that
> > doesn't matter: what matters is that it warns in common enough
> > configurations that developers will catch it.
> >
> > So having a warning in "might_sleep()" that doesn't always trigger,
> > because you have a limited configuration that can't even detect the
> > situation, that's fine and dandy and intentional.
> >
> > But having code like
> >
> >        if (can_schedule())
> >            .. do something different ..
> >
> > is fundamentally complete and utter garbage.
> >
> > It's one thing if you test for "am I in hardware interrupt context".
> > Those tests aren't great either, but at least they make sense.
> >
> > But a driver - or some library routine - making a difference based on
> > some nebulous "can I schedule" is fundamentally and basically WRONG.
> >
> > If some code changes behavior, it needs to be explicit to the *caller*
> > of that code.
> >
> > So this is why GFP_ATOMIC is fine, but "if (!can_schedule())
> > do_something_atomic()" is pure shite.
> >
> > And I am not IN THE LEAST interested in trying to help people doing
> > pure shite. We need to fix them. Like the crypto code is getting
> > fixed.
> 
> Just figured I'll throw my +1 in from reading too many (gpu) drivers.
> Code that tries to cleverly adjust its behaviour depending upon the
> context it's running in is harder to understand and blows up in more
> interesting ways. We still have drm_can_sleep() and it's mostly just
> used for debug code, and I've largely ended up just deleting
> everything that used it because when you're driver is blowing up the
> last thing you want is to realize your debug code and output can't be
> relied upon. Or worse, that the only Oops you have is the one in the
> debug code, because the real one scrolled away - the original idea
> behind drm_can_sleep was to make all the modeset code work
> automagically both in normal ioctl/kworker context and in the panic
> handlers or kgdb callbacks. Wishful thinking at best.
> 
> Also at least for me that extends to everything, e.g. I much prefer
> explicit spin_lock and spin_lock_irq vs magic spin_lock_irqsave for
> locks shared with interrupt handlers, since the former two gives me
> clear information from which contexts such function can be called.
> Other end is the memalloc_no*_save/restore functions, where I recently
> made a real big fool of myself because I didn't realize how much that
> impacts everything that's run within - suddenly "GFP_KERNEL for small
> stuff never fails" is wrong everywhere.
> 
> It's all great for debugging and sanity checks (and we run with all
> that stuff enabled in our CI), but really semantic changes depending
> upon magic context checks freak my out :-)

All fair, but some of us need to write code that must handle being
invoked from a wide variety of contexts.  Now perhaps you like the idea of
call_rcu() for schedulable contexts, call_rcu_nosched() when preemption
is disabled, call_rcu_irqs_are_disabled() when interrupts are disabled,
call_rcu_raw_atomic() from contexts where (for example) raw spinlocks
are held, and so on.  However, from what I can see, most people instead
consistently prefer that the RCU API instead be consolidated.

Some in-flight cache-efficiency work for kvfree_rcu() and call_rcu()
needs to be able to allocate memory occasionally.  It can do that when
invoked from some contexts, but not when invoked from others.  Right now,
in !PREEMPT kernels, it cannot tell, and must either do things to the
memory allocators that some of the MM hate or must unnecessarily invoke
workqueues.  Thomas's patches would allow the code to just allocate in
the common case when these primitives are invoked from contexts where
allocation is permitted.

If we want to restrict access to the can_schedule() or whatever primitive,
fine and good.  We can add a check to checkpatch.pl, for example.  Maybe
we can go back to the old brlock approach of requiring certain people's
review for each addition to the kernel.

But there really are use cases that it would greatly help.

							Thanx, Paul


^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-16 15:29               ` Paul E. McKenney
  0 siblings, 0 replies; 282+ messages in thread
From: Paul E. McKenney @ 2020-09-16 15:29 UTC (permalink / raw)
  To: Daniel Vetter
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, Linux-MM,
	open list:KERNEL SELFTEST FRAMEWORK, linux-hexagon, Will Deacon,
	Ingo Molnar, Anton Ivanov, linux-arch, Vincent Guittot,
	Herbert Xu, Brian Cain, Richard Weinberger, Russell King,
	Ard Biesheuvel, David Airlie, Ingo Molnar, Geert Uytterhoeven,
	Mel Gorman, intel-gfx, Matt Turner, Valentin Schneider,
	linux-xtensa, Shuah Khan, Jeff Dike, linux-um, Josh Triplett,
	Steven Rostedt, rcu, linux-m68k, Ivan Kokshaysky, Rodrigo Vivi,
	Thomas Gleixner, Dietmar Eggemann, Linux ARM, Richard Henderson,
	Chris Zankel, Max Filippov, Daniel Bristot de Oliveira, LKML,
	alpha, Mathieu Desnoyers, Andrew Morton, Linus Torvalds

On Wed, Sep 16, 2020 at 09:37:17AM +0200, Daniel Vetter wrote:
> On Tue, Sep 15, 2020 at 7:35 PM Linus Torvalds
> <torvalds@linux-foundation.org> wrote:
> >
> > On Tue, Sep 15, 2020 at 1:39 AM Thomas Gleixner <tglx@linutronix.de> wrote:
> > >
> > > OTOH, having a working 'preemptible()' or maybe better named
> > > 'can_schedule()' check makes tons of sense to make decisions about
> > > allocation modes or other things.
> >
> > No. I think that those kinds of decisions about actual behavior are
> > always simply fundamentally wrong.
> >
> > Note that this is very different from having warnings about invalid
> > use. THAT is correct. It may not warn in all configurations, but that
> > doesn't matter: what matters is that it warns in common enough
> > configurations that developers will catch it.
> >
> > So having a warning in "might_sleep()" that doesn't always trigger,
> > because you have a limited configuration that can't even detect the
> > situation, that's fine and dandy and intentional.
> >
> > But having code like
> >
> >        if (can_schedule())
> >            .. do something different ..
> >
> > is fundamentally complete and utter garbage.
> >
> > It's one thing if you test for "am I in hardware interrupt context".
> > Those tests aren't great either, but at least they make sense.
> >
> > But a driver - or some library routine - making a difference based on
> > some nebulous "can I schedule" is fundamentally and basically WRONG.
> >
> > If some code changes behavior, it needs to be explicit to the *caller*
> > of that code.
> >
> > So this is why GFP_ATOMIC is fine, but "if (!can_schedule())
> > do_something_atomic()" is pure shite.
> >
> > And I am not IN THE LEAST interested in trying to help people doing
> > pure shite. We need to fix them. Like the crypto code is getting
> > fixed.
> 
> Just figured I'll throw my +1 in from reading too many (gpu) drivers.
> Code that tries to cleverly adjust its behaviour depending upon the
> context it's running in is harder to understand and blows up in more
> interesting ways. We still have drm_can_sleep() and it's mostly just
> used for debug code, and I've largely ended up just deleting
> everything that used it because when you're driver is blowing up the
> last thing you want is to realize your debug code and output can't be
> relied upon. Or worse, that the only Oops you have is the one in the
> debug code, because the real one scrolled away - the original idea
> behind drm_can_sleep was to make all the modeset code work
> automagically both in normal ioctl/kworker context and in the panic
> handlers or kgdb callbacks. Wishful thinking at best.
> 
> Also at least for me that extends to everything, e.g. I much prefer
> explicit spin_lock and spin_lock_irq vs magic spin_lock_irqsave for
> locks shared with interrupt handlers, since the former two gives me
> clear information from which contexts such function can be called.
> Other end is the memalloc_no*_save/restore functions, where I recently
> made a real big fool of myself because I didn't realize how much that
> impacts everything that's run within - suddenly "GFP_KERNEL for small
> stuff never fails" is wrong everywhere.
> 
> It's all great for debugging and sanity checks (and we run with all
> that stuff enabled in our CI), but really semantic changes depending
> upon magic context checks freak my out :-)

All fair, but some of us need to write code that must handle being
invoked from a wide variety of contexts.  Now perhaps you like the idea of
call_rcu() for schedulable contexts, call_rcu_nosched() when preemption
is disabled, call_rcu_irqs_are_disabled() when interrupts are disabled,
call_rcu_raw_atomic() from contexts where (for example) raw spinlocks
are held, and so on.  However, from what I can see, most people instead
consistently prefer that the RCU API instead be consolidated.

Some in-flight cache-efficiency work for kvfree_rcu() and call_rcu()
needs to be able to allocate memory occasionally.  It can do that when
invoked from some contexts, but not when invoked from others.  Right now,
in !PREEMPT kernels, it cannot tell, and must either do things to the
memory allocators that some of the MM hate or must unnecessarily invoke
workqueues.  Thomas's patches would allow the code to just allocate in
the common case when these primitives are invoked from contexts where
allocation is permitted.

If we want to restrict access to the can_schedule() or whatever primitive,
fine and good.  We can add a check to checkpatch.pl, for example.  Maybe
we can go back to the old brlock approach of requiring certain people's
review for each addition to the kernel.

But there really are use cases that it would greatly help.

							Thanx, Paul
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [Intel-gfx] [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-16 15:29               ` Paul E. McKenney
  0 siblings, 0 replies; 282+ messages in thread
From: Paul E. McKenney @ 2020-09-16 15:29 UTC (permalink / raw)
  To: Daniel Vetter
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, Linux-MM,
	open list:KERNEL SELFTEST FRAMEWORK, linux-hexagon, Will Deacon,
	Ingo Molnar, Anton Ivanov, linux-arch, Herbert Xu, Brian Cain,
	Richard Weinberger, Russell King, Ard Biesheuvel, David Airlie,
	Ingo Molnar, Geert Uytterhoeven, Mel Gorman, intel-gfx,
	Matt Turner, Valentin Schneider, linux-xtensa, Shuah Khan,
	Jeff Dike, linux-um, Josh Triplett, Steven Rostedt, rcu,
	linux-m68k, Ivan Kokshaysky, Thomas Gleixner, Dietmar Eggemann,
	Linux ARM, Richard Henderson, Chris Zankel, Max Filippov,
	Daniel Bristot de Oliveira, LKML, alpha, Mathieu Desnoyers,
	Andrew Morton, Linus Torvalds

On Wed, Sep 16, 2020 at 09:37:17AM +0200, Daniel Vetter wrote:
> On Tue, Sep 15, 2020 at 7:35 PM Linus Torvalds
> <torvalds@linux-foundation.org> wrote:
> >
> > On Tue, Sep 15, 2020 at 1:39 AM Thomas Gleixner <tglx@linutronix.de> wrote:
> > >
> > > OTOH, having a working 'preemptible()' or maybe better named
> > > 'can_schedule()' check makes tons of sense to make decisions about
> > > allocation modes or other things.
> >
> > No. I think that those kinds of decisions about actual behavior are
> > always simply fundamentally wrong.
> >
> > Note that this is very different from having warnings about invalid
> > use. THAT is correct. It may not warn in all configurations, but that
> > doesn't matter: what matters is that it warns in common enough
> > configurations that developers will catch it.
> >
> > So having a warning in "might_sleep()" that doesn't always trigger,
> > because you have a limited configuration that can't even detect the
> > situation, that's fine and dandy and intentional.
> >
> > But having code like
> >
> >        if (can_schedule())
> >            .. do something different ..
> >
> > is fundamentally complete and utter garbage.
> >
> > It's one thing if you test for "am I in hardware interrupt context".
> > Those tests aren't great either, but at least they make sense.
> >
> > But a driver - or some library routine - making a difference based on
> > some nebulous "can I schedule" is fundamentally and basically WRONG.
> >
> > If some code changes behavior, it needs to be explicit to the *caller*
> > of that code.
> >
> > So this is why GFP_ATOMIC is fine, but "if (!can_schedule())
> > do_something_atomic()" is pure shite.
> >
> > And I am not IN THE LEAST interested in trying to help people doing
> > pure shite. We need to fix them. Like the crypto code is getting
> > fixed.
> 
> Just figured I'll throw my +1 in from reading too many (gpu) drivers.
> Code that tries to cleverly adjust its behaviour depending upon the
> context it's running in is harder to understand and blows up in more
> interesting ways. We still have drm_can_sleep() and it's mostly just
> used for debug code, and I've largely ended up just deleting
> everything that used it because when you're driver is blowing up the
> last thing you want is to realize your debug code and output can't be
> relied upon. Or worse, that the only Oops you have is the one in the
> debug code, because the real one scrolled away - the original idea
> behind drm_can_sleep was to make all the modeset code work
> automagically both in normal ioctl/kworker context and in the panic
> handlers or kgdb callbacks. Wishful thinking at best.
> 
> Also at least for me that extends to everything, e.g. I much prefer
> explicit spin_lock and spin_lock_irq vs magic spin_lock_irqsave for
> locks shared with interrupt handlers, since the former two gives me
> clear information from which contexts such function can be called.
> Other end is the memalloc_no*_save/restore functions, where I recently
> made a real big fool of myself because I didn't realize how much that
> impacts everything that's run within - suddenly "GFP_KERNEL for small
> stuff never fails" is wrong everywhere.
> 
> It's all great for debugging and sanity checks (and we run with all
> that stuff enabled in our CI), but really semantic changes depending
> upon magic context checks freak my out :-)

All fair, but some of us need to write code that must handle being
invoked from a wide variety of contexts.  Now perhaps you like the idea of
call_rcu() for schedulable contexts, call_rcu_nosched() when preemption
is disabled, call_rcu_irqs_are_disabled() when interrupts are disabled,
call_rcu_raw_atomic() from contexts where (for example) raw spinlocks
are held, and so on.  However, from what I can see, most people instead
consistently prefer that the RCU API instead be consolidated.

Some in-flight cache-efficiency work for kvfree_rcu() and call_rcu()
needs to be able to allocate memory occasionally.  It can do that when
invoked from some contexts, but not when invoked from others.  Right now,
in !PREEMPT kernels, it cannot tell, and must either do things to the
memory allocators that some of the MM hate or must unnecessarily invoke
workqueues.  Thomas's patches would allow the code to just allocate in
the common case when these primitives are invoked from contexts where
allocation is permitted.

If we want to restrict access to the can_schedule() or whatever primitive,
fine and good.  We can add a check to checkpatch.pl, for example.  Maybe
we can go back to the old brlock approach of requiring certain people's
review for each addition to the kernel.

But there really are use cases that it would greatly help.

							Thanx, Paul
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-16 15:29               ` Paul E. McKenney
  0 siblings, 0 replies; 282+ messages in thread
From: Paul E. McKenney @ 2020-09-16 15:29 UTC (permalink / raw)
  To: Daniel Vetter
  Cc: Linus Torvalds, Thomas Gleixner, Ard Biesheuvel, Herbert Xu,
	LKML, linux-arch, Sebastian Andrzej Siewior, Valentin Schneider,
	Richard Henderson, Ivan Kokshaysky, Matt Turner, alpha,
	Jeff Dike, Richard Weinberger, Anton Ivanov, linux-um,
	Brian Cain, linux-hexagon, Geert Uytterhoeven

On Wed, Sep 16, 2020 at 09:37:17AM +0200, Daniel Vetter wrote:
> On Tue, Sep 15, 2020 at 7:35 PM Linus Torvalds
> <torvalds@linux-foundation.org> wrote:
> >
> > On Tue, Sep 15, 2020 at 1:39 AM Thomas Gleixner <tglx@linutronix.de> wrote:
> > >
> > > OTOH, having a working 'preemptible()' or maybe better named
> > > 'can_schedule()' check makes tons of sense to make decisions about
> > > allocation modes or other things.
> >
> > No. I think that those kinds of decisions about actual behavior are
> > always simply fundamentally wrong.
> >
> > Note that this is very different from having warnings about invalid
> > use. THAT is correct. It may not warn in all configurations, but that
> > doesn't matter: what matters is that it warns in common enough
> > configurations that developers will catch it.
> >
> > So having a warning in "might_sleep()" that doesn't always trigger,
> > because you have a limited configuration that can't even detect the
> > situation, that's fine and dandy and intentional.
> >
> > But having code like
> >
> >        if (can_schedule())
> >            .. do something different ..
> >
> > is fundamentally complete and utter garbage.
> >
> > It's one thing if you test for "am I in hardware interrupt context".
> > Those tests aren't great either, but at least they make sense.
> >
> > But a driver - or some library routine - making a difference based on
> > some nebulous "can I schedule" is fundamentally and basically WRONG.
> >
> > If some code changes behavior, it needs to be explicit to the *caller*
> > of that code.
> >
> > So this is why GFP_ATOMIC is fine, but "if (!can_schedule())
> > do_something_atomic()" is pure shite.
> >
> > And I am not IN THE LEAST interested in trying to help people doing
> > pure shite. We need to fix them. Like the crypto code is getting
> > fixed.
> 
> Just figured I'll throw my +1 in from reading too many (gpu) drivers.
> Code that tries to cleverly adjust its behaviour depending upon the
> context it's running in is harder to understand and blows up in more
> interesting ways. We still have drm_can_sleep() and it's mostly just
> used for debug code, and I've largely ended up just deleting
> everything that used it because when you're driver is blowing up the
> last thing you want is to realize your debug code and output can't be
> relied upon. Or worse, that the only Oops you have is the one in the
> debug code, because the real one scrolled away - the original idea
> behind drm_can_sleep was to make all the modeset code work
> automagically both in normal ioctl/kworker context and in the panic
> handlers or kgdb callbacks. Wishful thinking at best.
> 
> Also at least for me that extends to everything, e.g. I much prefer
> explicit spin_lock and spin_lock_irq vs magic spin_lock_irqsave for
> locks shared with interrupt handlers, since the former two gives me
> clear information from which contexts such function can be called.
> Other end is the memalloc_no*_save/restore functions, where I recently
> made a real big fool of myself because I didn't realize how much that
> impacts everything that's run within - suddenly "GFP_KERNEL for small
> stuff never fails" is wrong everywhere.
> 
> It's all great for debugging and sanity checks (and we run with all
> that stuff enabled in our CI), but really semantic changes depending
> upon magic context checks freak my out :-)

All fair, but some of us need to write code that must handle being
invoked from a wide variety of contexts.  Now perhaps you like the idea of
call_rcu() for schedulable contexts, call_rcu_nosched() when preemption
is disabled, call_rcu_irqs_are_disabled() when interrupts are disabled,
call_rcu_raw_atomic() from contexts where (for example) raw spinlocks
are held, and so on.  However, from what I can see, most people instead
consistently prefer that the RCU API instead be consolidated.

Some in-flight cache-efficiency work for kvfree_rcu() and call_rcu()
needs to be able to allocate memory occasionally.  It can do that when
invoked from some contexts, but not when invoked from others.  Right now,
in !PREEMPT kernels, it cannot tell, and must either do things to the
memory allocators that some of the MM hate or must unnecessarily invoke
workqueues.  Thomas's patches would allow the code to just allocate in
the common case when these primitives are invoked from contexts where
allocation is permitted.

If we want to restrict access to the can_schedule() or whatever primitive,
fine and good.  We can add a check to checkpatch.pl, for example.  Maybe
we can go back to the old brlock approach of requiring certain people's
review for each addition to the kernel.

But there really are use cases that it would greatly help.

							Thanx, Paul

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
  2020-09-16 15:29               ` Paul E. McKenney
                                   ` (2 preceding siblings ...)
  (?)
@ 2020-09-16 18:32                 ` Linus Torvalds
  -1 siblings, 0 replies; 282+ messages in thread
From: Linus Torvalds @ 2020-09-16 18:32 UTC (permalink / raw)
  To: Paul E. McKenney
  Cc: Daniel Vetter, Thomas Gleixner, Ard Biesheuvel, Herbert Xu, LKML,
	linux-arch, Sebastian Andrzej Siewior, Valentin Schneider,
	Richard Henderson, Ivan Kokshaysky, Matt Turner, alpha,
	Jeff Dike, Richard Weinberger, Anton Ivanov, linux-um,
	Brian Cain, linux-hexagon, Geert Uytterhoeven, linux-m68k,
	Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
	Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman,
	Daniel Bristot de Oliveira, Will Deacon, Andrew Morton, Linux-MM,
	Ingo Molnar, Russell King, Linux ARM, Chris Zankel, Max Filippov,
	linux-xtensa, Jani Nikula, Joonas Lahtinen, Rodrigo Vivi,
	David Airlie, intel-gfx, dri-devel, Josh Triplett,
	Mathieu Desnoyers, Lai Jiangshan, Shuah Khan, rcu,
	open list:KERNEL SELFTEST FRAMEWORK

On Wed, Sep 16, 2020 at 8:29 AM Paul E. McKenney <paulmck@kernel.org> wrote:
>
> All fair, but some of us need to write code that must handle being
> invoked from a wide variety of contexts.

Note that I think that core functionality is different from random drivers.

Of course core code can (and will) look at things like

        if (in_interrupt())
            .. schedule work asynchronously ..

because core code ends up being called from odd places, and code like
that is expected to have understanding of the rules it plays with.

But something like RCU is a very different beast from some "walk the
scatter-gather list" code.

RCU does its work in the background, and works with lots of different
things. And it's so core and used everywhere that it knows about these
things. I mean, we literally have special code explicitly to let RCU
know "we entered kernel context now".

But something like a driver list walking thing should not be doing
different things behind peoples back depending on whether they hold
spinlocks or not. It should either just work regardless, or there
should be a flag (or special interface) for the "you're being called
in a crtitical region".

Because dynamically changing behavior really is very confusing.

               Linus

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-16 18:32                 ` Linus Torvalds
  0 siblings, 0 replies; 282+ messages in thread
From: Linus Torvalds @ 2020-09-16 18:32 UTC (permalink / raw)
  To: Paul E. McKenney
  Cc: Daniel Vetter, Thomas Gleixner, Ard Biesheuvel, Herbert Xu, LKML,
	linux-arch, Sebastian Andrzej Siewior, Valentin Schneider,
	Richard Henderson, Ivan Kokshaysky, Matt Turner, alpha,
	Jeff Dike, Richard Weinberger, Anton Ivanov, linux-um,
	Brian Cain, linux-hexagon, Geert Uytterhoeven, linux-m68k,
	Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
	Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman,
	Daniel Bristot de Oliveira, Will Deacon, Andrew Morton, Linux-MM,
	Ingo Molnar, Russell King, Linux ARM, Chris Zankel, Max Filippov,
	linux-xtensa, Jani Nikula, Joonas Lahtinen, Rodrigo Vivi,
	David Airlie, intel-gfx, dri-devel, Josh Triplett,
	Mathieu Desnoyers, Lai Jiangshan, Shuah Khan, rcu,
	open list:KERNEL SELFTEST FRAMEWORK

On Wed, Sep 16, 2020 at 8:29 AM Paul E. McKenney <paulmck@kernel.org> wrote:
>
> All fair, but some of us need to write code that must handle being
> invoked from a wide variety of contexts.

Note that I think that core functionality is different from random drivers.

Of course core code can (and will) look at things like

        if (in_interrupt())
            .. schedule work asynchronously ..

because core code ends up being called from odd places, and code like
that is expected to have understanding of the rules it plays with.

But something like RCU is a very different beast from some "walk the
scatter-gather list" code.

RCU does its work in the background, and works with lots of different
things. And it's so core and used everywhere that it knows about these
things. I mean, we literally have special code explicitly to let RCU
know "we entered kernel context now".

But something like a driver list walking thing should not be doing
different things behind peoples back depending on whether they hold
spinlocks or not. It should either just work regardless, or there
should be a flag (or special interface) for the "you're being called
in a crtitical region".

Because dynamically changing behavior really is very confusing.

               Linus


^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-16 18:32                 ` Linus Torvalds
  0 siblings, 0 replies; 282+ messages in thread
From: Linus Torvalds @ 2020-09-16 18:32 UTC (permalink / raw)
  To: Paul E. McKenney
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, Linux-MM,
	open list:KERNEL SELFTEST FRAMEWORK, linux-hexagon, Will Deacon,
	Ingo Molnar, Anton Ivanov, linux-arch, Vincent Guittot,
	Herbert Xu, Brian Cain, Richard Weinberger, Russell King,
	Ard Biesheuvel, David Airlie, Ingo Molnar, Geert Uytterhoeven,
	Mel Gorman, intel-gfx, Matt Turner, Valentin Schneider,
	linux-xtensa, Shuah Khan, Jeff Dike, linux-um, Josh Triplett,
	Steven Rostedt, rcu, linux-m68k, Ivan Kokshaysky, Rodrigo Vivi,
	Thomas Gleixner, Dietmar Eggemann, Linux ARM, Richard Henderson,
	Chris Zankel, Max Filippov, LKML, alpha, Mathieu Desnoyers,
	Andrew Morton, Daniel Bristot de Oliveira

On Wed, Sep 16, 2020 at 8:29 AM Paul E. McKenney <paulmck@kernel.org> wrote:
>
> All fair, but some of us need to write code that must handle being
> invoked from a wide variety of contexts.

Note that I think that core functionality is different from random drivers.

Of course core code can (and will) look at things like

        if (in_interrupt())
            .. schedule work asynchronously ..

because core code ends up being called from odd places, and code like
that is expected to have understanding of the rules it plays with.

But something like RCU is a very different beast from some "walk the
scatter-gather list" code.

RCU does its work in the background, and works with lots of different
things. And it's so core and used everywhere that it knows about these
things. I mean, we literally have special code explicitly to let RCU
know "we entered kernel context now".

But something like a driver list walking thing should not be doing
different things behind peoples back depending on whether they hold
spinlocks or not. It should either just work regardless, or there
should be a flag (or special interface) for the "you're being called
in a crtitical region".

Because dynamically changing behavior really is very confusing.

               Linus
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [Intel-gfx] [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-16 18:32                 ` Linus Torvalds
  0 siblings, 0 replies; 282+ messages in thread
From: Linus Torvalds @ 2020-09-16 18:32 UTC (permalink / raw)
  To: Paul E. McKenney
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, Linux-MM,
	open list:KERNEL SELFTEST FRAMEWORK, linux-hexagon, Will Deacon,
	Ingo Molnar, Anton Ivanov, linux-arch, Herbert Xu, Brian Cain,
	Richard Weinberger, Russell King, Ard Biesheuvel, David Airlie,
	Ingo Molnar, Geert Uytterhoeven, Mel Gorman, intel-gfx,
	Matt Turner, Valentin Schneider, linux-xtensa, Shuah Khan,
	Jeff Dike, linux-um, Josh Triplett, Steven Rostedt, rcu,
	linux-m68k, Ivan Kokshaysky, Thomas Gleixner, Dietmar Eggemann,
	Linux ARM, Richard Henderson, Chris Zankel, Max Filippov, LKML,
	alpha, Mathieu Desnoyers, Andrew Morton,
	Daniel Bristot de Oliveira

On Wed, Sep 16, 2020 at 8:29 AM Paul E. McKenney <paulmck@kernel.org> wrote:
>
> All fair, but some of us need to write code that must handle being
> invoked from a wide variety of contexts.

Note that I think that core functionality is different from random drivers.

Of course core code can (and will) look at things like

        if (in_interrupt())
            .. schedule work asynchronously ..

because core code ends up being called from odd places, and code like
that is expected to have understanding of the rules it plays with.

But something like RCU is a very different beast from some "walk the
scatter-gather list" code.

RCU does its work in the background, and works with lots of different
things. And it's so core and used everywhere that it knows about these
things. I mean, we literally have special code explicitly to let RCU
know "we entered kernel context now".

But something like a driver list walking thing should not be doing
different things behind peoples back depending on whether they hold
spinlocks or not. It should either just work regardless, or there
should be a flag (or special interface) for the "you're being called
in a crtitical region".

Because dynamically changing behavior really is very confusing.

               Linus
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-16 18:32                 ` Linus Torvalds
  0 siblings, 0 replies; 282+ messages in thread
From: Linus Torvalds @ 2020-09-16 18:32 UTC (permalink / raw)
  To: Paul E. McKenney
  Cc: Daniel Vetter, Thomas Gleixner, Ard Biesheuvel, Herbert Xu, LKML,
	linux-arch, Sebastian Andrzej Siewior, Valentin Schneider,
	Richard Henderson, Ivan Kokshaysky, Matt Turner, alpha,
	Jeff Dike, Richard Weinberger, Anton Ivanov, linux-um,
	Brian Cain, linux-hexagon, Geert Uytterhoeven, l

On Wed, Sep 16, 2020 at 8:29 AM Paul E. McKenney <paulmck@kernel.org> wrote:
>
> All fair, but some of us need to write code that must handle being
> invoked from a wide variety of contexts.

Note that I think that core functionality is different from random drivers.

Of course core code can (and will) look at things like

        if (in_interrupt())
            .. schedule work asynchronously ..

because core code ends up being called from odd places, and code like
that is expected to have understanding of the rules it plays with.

But something like RCU is a very different beast from some "walk the
scatter-gather list" code.

RCU does its work in the background, and works with lots of different
things. And it's so core and used everywhere that it knows about these
things. I mean, we literally have special code explicitly to let RCU
know "we entered kernel context now".

But something like a driver list walking thing should not be doing
different things behind peoples back depending on whether they hold
spinlocks or not. It should either just work regardless, or there
should be a flag (or special interface) for the "you're being called
in a crtitical region".

Because dynamically changing behavior really is very confusing.

               Linus

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
  2020-09-15 19:57             ` Thomas Gleixner
                                 ` (2 preceding siblings ...)
  (?)
@ 2020-09-16 18:34               ` Linus Torvalds
  -1 siblings, 0 replies; 282+ messages in thread
From: Linus Torvalds @ 2020-09-16 18:34 UTC (permalink / raw)
  To: Thomas Gleixner
  Cc: Ard Biesheuvel, Herbert Xu, LKML, linux-arch,
	Sebastian Andrzej Siewior, Valentin Schneider, Richard Henderson,
	Ivan Kokshaysky, Matt Turner, alpha, Jeff Dike,
	Richard Weinberger, Anton Ivanov, linux-um, Brian Cain,
	linux-hexagon, Geert Uytterhoeven, linux-m68k, Ingo Molnar,
	Peter Zijlstra, Juri Lelli, Vincent Guittot, Dietmar Eggemann,
	Steven Rostedt, Ben Segall, Mel Gorman,
	Daniel Bristot de Oliveira, Will Deacon, Andrew Morton, Linux-MM,
	Ingo Molnar, Russell King, Linux ARM, Chris Zankel, Max Filippov,
	linux-xtensa, Jani Nikula, Joonas Lahtinen, Rodrigo Vivi,
	David Airlie, Daniel Vetter, intel-gfx, dri-devel,
	Paul E. McKenney, Josh Triplett, Mathieu Desnoyers,
	Lai Jiangshan, Shuah Khan, rcu,
	open list:KERNEL SELFTEST FRAMEWORK

On Tue, Sep 15, 2020 at 12:57 PM Thomas Gleixner <tglx@linutronix.de> wrote:
>
> You wish. I just found a 7 year old bug in a 10G network driver which
> surely would have been found if people would enable debug configs and
> not just run the crap on their PREEMPT_NONE, all debug off kernel. And
> that driver is not subject to bitrot, it gets regular bug fixes from
> people who seem to care (distro folks).

That driver clearly cannot be very well maintained. All the distro
kernels have the basic debug checks in place, afaik.

Is it some wonderful "enterprise hardware" garbage again that only
gets used in special data centers?

Becasue the "enterprise" people really are special. Very much in the
"short bus" special kind of way. The fact that they have fooled so
much of the industry into thinking that they are the competent and
serious people is a disgrace.

              Linus

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-16 18:34               ` Linus Torvalds
  0 siblings, 0 replies; 282+ messages in thread
From: Linus Torvalds @ 2020-09-16 18:34 UTC (permalink / raw)
  To: Thomas Gleixner
  Cc: Ard Biesheuvel, Herbert Xu, LKML, linux-arch,
	Sebastian Andrzej Siewior, Valentin Schneider, Richard Henderson,
	Ivan Kokshaysky, Matt Turner, alpha, Jeff Dike,
	Richard Weinberger, Anton Ivanov, linux-um, Brian Cain,
	linux-hexagon, Geert Uytterhoeven, linux-m68k, Ingo Molnar,
	Peter Zijlstra, Juri Lelli, Vincent Guittot, Dietmar Eggemann,
	Steven Rostedt, Ben Segall, Mel Gorman,
	Daniel Bristot de Oliveira, Will Deacon, Andrew Morton, Linux-MM,
	Ingo Molnar, Russell King, Linux ARM, Chris Zankel, Max Filippov,
	linux-xtensa, Jani Nikula, Joonas Lahtinen, Rodrigo Vivi,
	David Airlie, Daniel Vetter, intel-gfx, dri-devel,
	Paul E. McKenney, Josh Triplett, Mathieu Desnoyers,
	Lai Jiangshan, Shuah Khan, rcu,
	open list:KERNEL SELFTEST FRAMEWORK

On Tue, Sep 15, 2020 at 12:57 PM Thomas Gleixner <tglx@linutronix.de> wrote:
>
> You wish. I just found a 7 year old bug in a 10G network driver which
> surely would have been found if people would enable debug configs and
> not just run the crap on their PREEMPT_NONE, all debug off kernel. And
> that driver is not subject to bitrot, it gets regular bug fixes from
> people who seem to care (distro folks).

That driver clearly cannot be very well maintained. All the distro
kernels have the basic debug checks in place, afaik.

Is it some wonderful "enterprise hardware" garbage again that only
gets used in special data centers?

Becasue the "enterprise" people really are special. Very much in the
"short bus" special kind of way. The fact that they have fooled so
much of the industry into thinking that they are the competent and
serious people is a disgrace.

              Linus


^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-16 18:34               ` Linus Torvalds
  0 siblings, 0 replies; 282+ messages in thread
From: Linus Torvalds @ 2020-09-16 18:34 UTC (permalink / raw)
  To: Thomas Gleixner
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, Linux-MM,
	open list:KERNEL SELFTEST FRAMEWORK, linux-hexagon, Will Deacon,
	Ingo Molnar, Anton Ivanov, linux-arch, Vincent Guittot,
	Herbert Xu, Brian Cain, Richard Weinberger, Russell King,
	Ard Biesheuvel, David Airlie, Ingo Molnar, Geert Uytterhoeven,
	Mel Gorman, intel-gfx, Matt Turner, Valentin Schneider,
	linux-xtensa, Shuah Khan, Paul E. McKenney, Jeff Dike, linux-um,
	Josh Triplett, Steven Rostedt, rcu, linux-m68k, Ivan Kokshaysky,
	Rodrigo Vivi, Dietmar Eggemann, Linux ARM, Richard Henderson,
	Chris Zankel, Max Filippov, LKML, alpha, Mathieu Desnoyers,
	Andrew Morton, Daniel Bristot de Oliveira

On Tue, Sep 15, 2020 at 12:57 PM Thomas Gleixner <tglx@linutronix.de> wrote:
>
> You wish. I just found a 7 year old bug in a 10G network driver which
> surely would have been found if people would enable debug configs and
> not just run the crap on their PREEMPT_NONE, all debug off kernel. And
> that driver is not subject to bitrot, it gets regular bug fixes from
> people who seem to care (distro folks).

That driver clearly cannot be very well maintained. All the distro
kernels have the basic debug checks in place, afaik.

Is it some wonderful "enterprise hardware" garbage again that only
gets used in special data centers?

Becasue the "enterprise" people really are special. Very much in the
"short bus" special kind of way. The fact that they have fooled so
much of the industry into thinking that they are the competent and
serious people is a disgrace.

              Linus
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [Intel-gfx] [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-16 18:34               ` Linus Torvalds
  0 siblings, 0 replies; 282+ messages in thread
From: Linus Torvalds @ 2020-09-16 18:34 UTC (permalink / raw)
  To: Thomas Gleixner
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, Linux-MM,
	open list:KERNEL SELFTEST FRAMEWORK, linux-hexagon, Will Deacon,
	Ingo Molnar, Anton Ivanov, linux-arch, Herbert Xu, Brian Cain,
	Richard Weinberger, Russell King, Ard Biesheuvel, David Airlie,
	Ingo Molnar, Geert Uytterhoeven, Mel Gorman, intel-gfx,
	Matt Turner, Valentin Schneider, linux-xtensa, Shuah Khan,
	Paul E. McKenney, Jeff Dike, linux-um, Josh Triplett,
	Steven Rostedt, rcu, linux-m68k, Ivan Kokshaysky,
	Dietmar Eggemann, Linux ARM, Richard Henderson, Chris Zankel,
	Max Filippov, LKML, alpha, Mathieu Desnoyers, Andrew Morton,
	Daniel Bristot de Oliveira

On Tue, Sep 15, 2020 at 12:57 PM Thomas Gleixner <tglx@linutronix.de> wrote:
>
> You wish. I just found a 7 year old bug in a 10G network driver which
> surely would have been found if people would enable debug configs and
> not just run the crap on their PREEMPT_NONE, all debug off kernel. And
> that driver is not subject to bitrot, it gets regular bug fixes from
> people who seem to care (distro folks).

That driver clearly cannot be very well maintained. All the distro
kernels have the basic debug checks in place, afaik.

Is it some wonderful "enterprise hardware" garbage again that only
gets used in special data centers?

Becasue the "enterprise" people really are special. Very much in the
"short bus" special kind of way. The fact that they have fooled so
much of the industry into thinking that they are the competent and
serious people is a disgrace.

              Linus
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-16 18:34               ` Linus Torvalds
  0 siblings, 0 replies; 282+ messages in thread
From: Linus Torvalds @ 2020-09-16 18:34 UTC (permalink / raw)
  To: Thomas Gleixner
  Cc: Ard Biesheuvel, Herbert Xu, LKML, linux-arch,
	Sebastian Andrzej Siewior, Valentin Schneider, Richard Henderson,
	Ivan Kokshaysky, Matt Turner, alpha, Jeff Dike,
	Richard Weinberger, Anton Ivanov, linux-um, Brian Cain,
	linux-hexagon, Geert Uytterhoeven, linux-m68k, Ingo Molnar

On Tue, Sep 15, 2020 at 12:57 PM Thomas Gleixner <tglx@linutronix.de> wrote:
>
> You wish. I just found a 7 year old bug in a 10G network driver which
> surely would have been found if people would enable debug configs and
> not just run the crap on their PREEMPT_NONE, all debug off kernel. And
> that driver is not subject to bitrot, it gets regular bug fixes from
> people who seem to care (distro folks).

That driver clearly cannot be very well maintained. All the distro
kernels have the basic debug checks in place, afaik.

Is it some wonderful "enterprise hardware" garbage again that only
gets used in special data centers?

Becasue the "enterprise" people really are special. Very much in the
"short bus" special kind of way. The fact that they have fooled so
much of the industry into thinking that they are the competent and
serious people is a disgrace.

              Linus

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
  2020-09-14 21:55     ` Thomas Gleixner
                         ` (2 preceding siblings ...)
  (?)
@ 2020-09-16 19:23       ` Matthew Wilcox
  -1 siblings, 0 replies; 282+ messages in thread
From: Matthew Wilcox @ 2020-09-16 19:23 UTC (permalink / raw)
  To: Thomas Gleixner
  Cc: Linus Torvalds, LKML, linux-arch, Sebastian Andrzej Siewior,
	Valentin Schneider, Richard Henderson, Ivan Kokshaysky,
	Matt Turner, alpha, Jeff Dike, Richard Weinberger, Anton Ivanov,
	linux-um, Brian Cain, linux-hexagon, Geert Uytterhoeven,
	linux-m68k, Ingo Molnar, Peter Zijlstra, Juri Lelli,
	Vincent Guittot, Dietmar Eggemann, Steven Rostedt, Ben Segall,
	Mel Gorman, Daniel Bristot de Oliveira, Will Deacon,
	Andrew Morton, Linux-MM, Ingo Molnar, Russell King, Linux ARM,
	Chris Zankel, Max Filippov, linux-xtensa, Jani Nikula,
	Joonas Lahtinen, Rodrigo Vivi, David Airlie, Daniel Vetter,
	intel-gfx, dri-devel, Paul E. McKenney, Josh Triplett,
	Mathieu Desnoyers, Lai Jiangshan, Shuah Khan, rcu,
	open list:KERNEL SELFTEST FRAMEWORK

On Mon, Sep 14, 2020 at 11:55:24PM +0200, Thomas Gleixner wrote:
> But just look at any check which uses preemptible(), especially those
> which check !preemptible():

hmm.

+++ b/include/linux/preempt.h
@@ -180,7 +180,9 @@ do { \
 
 #define preempt_enable_no_resched() sched_preempt_enable_no_resched()
 
+#ifndef MODULE
 #define preemptible()  (preempt_count() == 0 && !irqs_disabled())
+#endif
 
 #ifdef CONFIG_PREEMPTION
 #define preempt_enable() \


$ git grep -w preemptible drivers
(slightly trimmed by hand to remove, eg, comments)
drivers/firmware/arm_sdei.c:    WARN_ON_ONCE(preemptible());
drivers/firmware/arm_sdei.c:    WARN_ON_ONCE(preemptible());
drivers/firmware/arm_sdei.c:    WARN_ON_ONCE(preemptible());
drivers/firmware/arm_sdei.c:    WARN_ON_ONCE(preemptible());
drivers/firmware/arm_sdei.c:    WARN_ON(preemptible());
drivers/firmware/efi/efi-pstore.c:                            preemptible(), record->size, record->psi->buf);
drivers/irqchip/irq-gic-v4.c:   WARN_ON(preemptible());
drivers/irqchip/irq-gic-v4.c:   WARN_ON(preemptible());
drivers/scsi/hisi_sas/hisi_sas_main.c:          if (!preemptible())
drivers/xen/time.c:     BUG_ON(preemptible());

That only looks like two drivers that need more than WARNectomies.

Although maybe rcu_read_load_sched_held() or rcu_read_lock_any_held()
might get called from a module ...

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-16 19:23       ` Matthew Wilcox
  0 siblings, 0 replies; 282+ messages in thread
From: Matthew Wilcox @ 2020-09-16 19:23 UTC (permalink / raw)
  To: Thomas Gleixner
  Cc: Linus Torvalds, LKML, linux-arch, Sebastian Andrzej Siewior,
	Valentin Schneider, Richard Henderson, Ivan Kokshaysky,
	Matt Turner, alpha, Jeff Dike, Richard Weinberger, Anton Ivanov,
	linux-um, Brian Cain, linux-hexagon, Geert Uytterhoeven,
	linux-m68k, Ingo Molnar, Peter Zijlstra, Juri Lelli,
	Vincent Guittot, Dietmar Eggemann, Steven Rostedt, Ben Segall,
	Mel Gorman, Daniel Bristot de Oliveira, Will Deacon,
	Andrew Morton, Linux-MM, Ingo Molnar, Russell King, Linux ARM,
	Chris Zankel, Max Filippov, linux-xtensa, Jani Nikula,
	Joonas Lahtinen, Rodrigo Vivi, David Airlie, Daniel Vetter,
	intel-gfx, dri-devel, Paul E. McKenney, Josh Triplett,
	Mathieu Desnoyers, Lai Jiangshan, Shuah Khan, rcu,
	open list:KERNEL SELFTEST FRAMEWORK

On Mon, Sep 14, 2020 at 11:55:24PM +0200, Thomas Gleixner wrote:
> But just look at any check which uses preemptible(), especially those
> which check !preemptible():

hmm.

+++ b/include/linux/preempt.h
@@ -180,7 +180,9 @@ do { \
 
 #define preempt_enable_no_resched() sched_preempt_enable_no_resched()
 
+#ifndef MODULE
 #define preemptible()  (preempt_count() == 0 && !irqs_disabled())
+#endif
 
 #ifdef CONFIG_PREEMPTION
 #define preempt_enable() \


$ git grep -w preemptible drivers
(slightly trimmed by hand to remove, eg, comments)
drivers/firmware/arm_sdei.c:    WARN_ON_ONCE(preemptible());
drivers/firmware/arm_sdei.c:    WARN_ON_ONCE(preemptible());
drivers/firmware/arm_sdei.c:    WARN_ON_ONCE(preemptible());
drivers/firmware/arm_sdei.c:    WARN_ON_ONCE(preemptible());
drivers/firmware/arm_sdei.c:    WARN_ON(preemptible());
drivers/firmware/efi/efi-pstore.c:                            preemptible(), record->size, record->psi->buf);
drivers/irqchip/irq-gic-v4.c:   WARN_ON(preemptible());
drivers/irqchip/irq-gic-v4.c:   WARN_ON(preemptible());
drivers/scsi/hisi_sas/hisi_sas_main.c:          if (!preemptible())
drivers/xen/time.c:     BUG_ON(preemptible());

That only looks like two drivers that need more than WARNectomies.

Although maybe rcu_read_load_sched_held() or rcu_read_lock_any_held()
might get called from a module ...


^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-16 19:23       ` Matthew Wilcox
  0 siblings, 0 replies; 282+ messages in thread
From: Matthew Wilcox @ 2020-09-16 19:23 UTC (permalink / raw)
  To: Thomas Gleixner
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, Linux-MM,
	open list:KERNEL SELFTEST FRAMEWORK, linux-hexagon, Will Deacon,
	Ingo Molnar, Anton Ivanov, linux-arch, Vincent Guittot,
	Brian Cain, Richard Weinberger, Russell King, David Airlie,
	Ingo Molnar, Geert Uytterhoeven, Mel Gorman, intel-gfx,
	Matt Turner, Valentin Schneider, linux-xtensa, Shuah Khan,
	Paul E. McKenney, Jeff Dike, linux-um, Josh Triplett,
	Steven Rostedt, rcu, linux-m68k, Ivan Kokshaysky, Rodrigo Vivi,
	Dietmar Eggemann, Linux ARM, Richard Henderson, Chris Zankel,
	Max Filippov, Daniel Bristot de Oliveira, LKML, alpha,
	Mathieu Desnoyers, Andrew Morton, Linus Torvalds

On Mon, Sep 14, 2020 at 11:55:24PM +0200, Thomas Gleixner wrote:
> But just look at any check which uses preemptible(), especially those
> which check !preemptible():

hmm.

+++ b/include/linux/preempt.h
@@ -180,7 +180,9 @@ do { \
 
 #define preempt_enable_no_resched() sched_preempt_enable_no_resched()
 
+#ifndef MODULE
 #define preemptible()  (preempt_count() == 0 && !irqs_disabled())
+#endif
 
 #ifdef CONFIG_PREEMPTION
 #define preempt_enable() \


$ git grep -w preemptible drivers
(slightly trimmed by hand to remove, eg, comments)
drivers/firmware/arm_sdei.c:    WARN_ON_ONCE(preemptible());
drivers/firmware/arm_sdei.c:    WARN_ON_ONCE(preemptible());
drivers/firmware/arm_sdei.c:    WARN_ON_ONCE(preemptible());
drivers/firmware/arm_sdei.c:    WARN_ON_ONCE(preemptible());
drivers/firmware/arm_sdei.c:    WARN_ON(preemptible());
drivers/firmware/efi/efi-pstore.c:                            preemptible(), record->size, record->psi->buf);
drivers/irqchip/irq-gic-v4.c:   WARN_ON(preemptible());
drivers/irqchip/irq-gic-v4.c:   WARN_ON(preemptible());
drivers/scsi/hisi_sas/hisi_sas_main.c:          if (!preemptible())
drivers/xen/time.c:     BUG_ON(preemptible());

That only looks like two drivers that need more than WARNectomies.

Although maybe rcu_read_load_sched_held() or rcu_read_lock_any_held()
might get called from a module ...
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [Intel-gfx] [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-16 19:23       ` Matthew Wilcox
  0 siblings, 0 replies; 282+ messages in thread
From: Matthew Wilcox @ 2020-09-16 19:23 UTC (permalink / raw)
  To: Thomas Gleixner
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, Linux-MM,
	open list:KERNEL SELFTEST FRAMEWORK, linux-hexagon, Will Deacon,
	Ingo Molnar, Anton Ivanov, linux-arch, Brian Cain,
	Richard Weinberger, Russell King, David Airlie, Ingo Molnar,
	Geert Uytterhoeven, Mel Gorman, intel-gfx, Matt Turner,
	Valentin Schneider, linux-xtensa, Shuah Khan, Paul E. McKenney,
	Jeff Dike, linux-um, Josh Triplett, Steven Rostedt, rcu,
	linux-m68k, Ivan Kokshaysky, Dietmar Eggemann, Linux ARM,
	Richard Henderson, Chris Zankel, Max Filippov,
	Daniel Bristot de Oliveira, LKML, alpha, Mathieu Desnoyers,
	Andrew Morton, Linus Torvalds

On Mon, Sep 14, 2020 at 11:55:24PM +0200, Thomas Gleixner wrote:
> But just look at any check which uses preemptible(), especially those
> which check !preemptible():

hmm.

+++ b/include/linux/preempt.h
@@ -180,7 +180,9 @@ do { \
 
 #define preempt_enable_no_resched() sched_preempt_enable_no_resched()
 
+#ifndef MODULE
 #define preemptible()  (preempt_count() == 0 && !irqs_disabled())
+#endif
 
 #ifdef CONFIG_PREEMPTION
 #define preempt_enable() \


$ git grep -w preemptible drivers
(slightly trimmed by hand to remove, eg, comments)
drivers/firmware/arm_sdei.c:    WARN_ON_ONCE(preemptible());
drivers/firmware/arm_sdei.c:    WARN_ON_ONCE(preemptible());
drivers/firmware/arm_sdei.c:    WARN_ON_ONCE(preemptible());
drivers/firmware/arm_sdei.c:    WARN_ON_ONCE(preemptible());
drivers/firmware/arm_sdei.c:    WARN_ON(preemptible());
drivers/firmware/efi/efi-pstore.c:                            preemptible(), record->size, record->psi->buf);
drivers/irqchip/irq-gic-v4.c:   WARN_ON(preemptible());
drivers/irqchip/irq-gic-v4.c:   WARN_ON(preemptible());
drivers/scsi/hisi_sas/hisi_sas_main.c:          if (!preemptible())
drivers/xen/time.c:     BUG_ON(preemptible());

That only looks like two drivers that need more than WARNectomies.

Although maybe rcu_read_load_sched_held() or rcu_read_lock_any_held()
might get called from a module ...
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-16 19:23       ` Matthew Wilcox
  0 siblings, 0 replies; 282+ messages in thread
From: Matthew Wilcox @ 2020-09-16 19:23 UTC (permalink / raw)
  To: Thomas Gleixner
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, Linux-MM,
	open list:KERNEL SELFTEST FRAMEWORK, linux-hexagon, Will Deacon,
	Ingo Molnar, Anton Ivanov, linux-arch, Brian Cain,
	Richard Weinberger, Russell King, David Airlie, Ingo Molnar,
	Geert Uytterhoeven, Mel Gorman, intel-gfx, Matt Turner,
	Valentin Schneider

On Mon, Sep 14, 2020 at 11:55:24PM +0200, Thomas Gleixner wrote:
> But just look at any check which uses preemptible(), especially those
> which check !preemptible():

hmm.

+++ b/include/linux/preempt.h
@@ -180,7 +180,9 @@ do { \
 
 #define preempt_enable_no_resched() sched_preempt_enable_no_resched()
 
+#ifndef MODULE
 #define preemptible()  (preempt_count() == 0 && !irqs_disabled())
+#endif
 
 #ifdef CONFIG_PREEMPTION
 #define preempt_enable() \


$ git grep -w preemptible drivers
(slightly trimmed by hand to remove, eg, comments)
drivers/firmware/arm_sdei.c:    WARN_ON_ONCE(preemptible());
drivers/firmware/arm_sdei.c:    WARN_ON_ONCE(preemptible());
drivers/firmware/arm_sdei.c:    WARN_ON_ONCE(preemptible());
drivers/firmware/arm_sdei.c:    WARN_ON_ONCE(preemptible());
drivers/firmware/arm_sdei.c:    WARN_ON(preemptible());
drivers/firmware/efi/efi-pstore.c:                            preemptible(), record->size, record->psi->buf);
drivers/irqchip/irq-gic-v4.c:   WARN_ON(preemptible());
drivers/irqchip/irq-gic-v4.c:   WARN_ON(preemptible());
drivers/scsi/hisi_sas/hisi_sas_main.c:          if (!preemptible())
drivers/xen/time.c:     BUG_ON(preemptible());

That only looks like two drivers that need more than WARNectomies.

Although maybe rcu_read_load_sched_held() or rcu_read_lock_any_held()
might get called from a module ...

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
  2020-09-16 15:29               ` Paul E. McKenney
                                   ` (2 preceding siblings ...)
  (?)
@ 2020-09-16 20:29                 ` Daniel Vetter
  -1 siblings, 0 replies; 282+ messages in thread
From: Daniel Vetter @ 2020-09-16 20:29 UTC (permalink / raw)
  To: Paul E. McKenney
  Cc: Linus Torvalds, Thomas Gleixner, Ard Biesheuvel, Herbert Xu,
	LKML, linux-arch, Sebastian Andrzej Siewior, Valentin Schneider,
	Richard Henderson, Ivan Kokshaysky, Matt Turner, alpha,
	Jeff Dike, Richard Weinberger, Anton Ivanov, linux-um,
	Brian Cain, linux-hexagon, Geert Uytterhoeven, linux-m68k,
	Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
	Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman,
	Daniel Bristot de Oliveira, Will Deacon, Andrew Morton, Linux-MM,
	Ingo Molnar, Russell King, Linux ARM, Chris Zankel, Max Filippov,
	linux-xtensa, Jani Nikula, Joonas Lahtinen, Rodrigo Vivi,
	David Airlie, intel-gfx, dri-devel, Josh Triplett,
	Mathieu Desnoyers, Lai Jiangshan, Shuah Khan, rcu,
	open list:KERNEL SELFTEST FRAMEWORK

On Wed, Sep 16, 2020 at 5:29 PM Paul E. McKenney <paulmck@kernel.org> wrote:
>
> On Wed, Sep 16, 2020 at 09:37:17AM +0200, Daniel Vetter wrote:
> > On Tue, Sep 15, 2020 at 7:35 PM Linus Torvalds
> > <torvalds@linux-foundation.org> wrote:
> > >
> > > On Tue, Sep 15, 2020 at 1:39 AM Thomas Gleixner <tglx@linutronix.de> wrote:
> > > >
> > > > OTOH, having a working 'preemptible()' or maybe better named
> > > > 'can_schedule()' check makes tons of sense to make decisions about
> > > > allocation modes or other things.
> > >
> > > No. I think that those kinds of decisions about actual behavior are
> > > always simply fundamentally wrong.
> > >
> > > Note that this is very different from having warnings about invalid
> > > use. THAT is correct. It may not warn in all configurations, but that
> > > doesn't matter: what matters is that it warns in common enough
> > > configurations that developers will catch it.
> > >
> > > So having a warning in "might_sleep()" that doesn't always trigger,
> > > because you have a limited configuration that can't even detect the
> > > situation, that's fine and dandy and intentional.
> > >
> > > But having code like
> > >
> > >        if (can_schedule())
> > >            .. do something different ..
> > >
> > > is fundamentally complete and utter garbage.
> > >
> > > It's one thing if you test for "am I in hardware interrupt context".
> > > Those tests aren't great either, but at least they make sense.
> > >
> > > But a driver - or some library routine - making a difference based on
> > > some nebulous "can I schedule" is fundamentally and basically WRONG.
> > >
> > > If some code changes behavior, it needs to be explicit to the *caller*
> > > of that code.
> > >
> > > So this is why GFP_ATOMIC is fine, but "if (!can_schedule())
> > > do_something_atomic()" is pure shite.
> > >
> > > And I am not IN THE LEAST interested in trying to help people doing
> > > pure shite. We need to fix them. Like the crypto code is getting
> > > fixed.
> >
> > Just figured I'll throw my +1 in from reading too many (gpu) drivers.
> > Code that tries to cleverly adjust its behaviour depending upon the
> > context it's running in is harder to understand and blows up in more
> > interesting ways. We still have drm_can_sleep() and it's mostly just
> > used for debug code, and I've largely ended up just deleting
> > everything that used it because when you're driver is blowing up the
> > last thing you want is to realize your debug code and output can't be
> > relied upon. Or worse, that the only Oops you have is the one in the
> > debug code, because the real one scrolled away - the original idea
> > behind drm_can_sleep was to make all the modeset code work
> > automagically both in normal ioctl/kworker context and in the panic
> > handlers or kgdb callbacks. Wishful thinking at best.
> >
> > Also at least for me that extends to everything, e.g. I much prefer
> > explicit spin_lock and spin_lock_irq vs magic spin_lock_irqsave for
> > locks shared with interrupt handlers, since the former two gives me
> > clear information from which contexts such function can be called.
> > Other end is the memalloc_no*_save/restore functions, where I recently
> > made a real big fool of myself because I didn't realize how much that
> > impacts everything that's run within - suddenly "GFP_KERNEL for small
> > stuff never fails" is wrong everywhere.
> >
> > It's all great for debugging and sanity checks (and we run with all
> > that stuff enabled in our CI), but really semantic changes depending
> > upon magic context checks freak my out :-)
>
> All fair, but some of us need to write code that must handle being
> invoked from a wide variety of contexts.  Now perhaps you like the idea of
> call_rcu() for schedulable contexts, call_rcu_nosched() when preemption
> is disabled, call_rcu_irqs_are_disabled() when interrupts are disabled,
> call_rcu_raw_atomic() from contexts where (for example) raw spinlocks
> are held, and so on.  However, from what I can see, most people instead
> consistently prefer that the RCU API instead be consolidated.
>
> Some in-flight cache-efficiency work for kvfree_rcu() and call_rcu()
> needs to be able to allocate memory occasionally.  It can do that when
> invoked from some contexts, but not when invoked from others.  Right now,
> in !PREEMPT kernels, it cannot tell, and must either do things to the
> memory allocators that some of the MM hate or must unnecessarily invoke
> workqueues.  Thomas's patches would allow the code to just allocate in
> the common case when these primitives are invoked from contexts where
> allocation is permitted.
>
> If we want to restrict access to the can_schedule() or whatever primitive,
> fine and good.  We can add a check to checkpatch.pl, for example.  Maybe
> we can go back to the old brlock approach of requiring certain people's
> review for each addition to the kernel.
>
> But there really are use cases that it would greatly help.

We can deadlock in random fun places if random stuff we're calling
suddenly starts allocating. Sometimes. Maybe once in a blue moon, to
make it extra fun to reproduce. Maybe most driver subsystems are less
brittle, but gpu drivers definitely need to know about the details for
exactly this example. And yes gpu drivers use rcu for freeing
dma_fence structures, and that tends to happen in code that we only
recently figured out should really not allocate memory.

I think minimally you need to throw in an unconditional
fs_reclaim_acquire();fs_reclaim_release(); so that everyone who runs
with full debugging knows what might happen. It's kinda like
might_sleep, but a lot more specific. might_sleep() alone is not
enough, because in the specific code paths I'm thinking of (and
created special lockdep annotations for just recently) sleeping is
allowed, but any memory allocations with GFP_RECLAIM set are no-go.

Cheers, Daniel




--
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-16 20:29                 ` Daniel Vetter
  0 siblings, 0 replies; 282+ messages in thread
From: Daniel Vetter @ 2020-09-16 20:29 UTC (permalink / raw)
  To: Paul E. McKenney
  Cc: Linus Torvalds, Thomas Gleixner, Ard Biesheuvel, Herbert Xu,
	LKML, linux-arch, Sebastian Andrzej Siewior, Valentin Schneider,
	Richard Henderson, Ivan Kokshaysky, Matt Turner, alpha,
	Jeff Dike, Richard Weinberger, Anton Ivanov, linux-um,
	Brian Cain, linux-hexagon, Geert Uytterhoeven, linux-m68k,
	Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
	Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman,
	Daniel Bristot de Oliveira, Will Deacon, Andrew Morton, Linux-MM,
	Ingo Molnar, Russell King, Linux ARM, Chris Zankel, Max Filippov,
	linux-xtensa, Jani Nikula, Joonas Lahtinen, Rodrigo Vivi,
	David Airlie, intel-gfx, dri-devel, Josh Triplett,
	Mathieu Desnoyers, Lai Jiangshan, Shuah Khan, rcu,
	open list:KERNEL SELFTEST FRAMEWORK

On Wed, Sep 16, 2020 at 5:29 PM Paul E. McKenney <paulmck@kernel.org> wrote:
>
> On Wed, Sep 16, 2020 at 09:37:17AM +0200, Daniel Vetter wrote:
> > On Tue, Sep 15, 2020 at 7:35 PM Linus Torvalds
> > <torvalds@linux-foundation.org> wrote:
> > >
> > > On Tue, Sep 15, 2020 at 1:39 AM Thomas Gleixner <tglx@linutronix.de> wrote:
> > > >
> > > > OTOH, having a working 'preemptible()' or maybe better named
> > > > 'can_schedule()' check makes tons of sense to make decisions about
> > > > allocation modes or other things.
> > >
> > > No. I think that those kinds of decisions about actual behavior are
> > > always simply fundamentally wrong.
> > >
> > > Note that this is very different from having warnings about invalid
> > > use. THAT is correct. It may not warn in all configurations, but that
> > > doesn't matter: what matters is that it warns in common enough
> > > configurations that developers will catch it.
> > >
> > > So having a warning in "might_sleep()" that doesn't always trigger,
> > > because you have a limited configuration that can't even detect the
> > > situation, that's fine and dandy and intentional.
> > >
> > > But having code like
> > >
> > >        if (can_schedule())
> > >            .. do something different ..
> > >
> > > is fundamentally complete and utter garbage.
> > >
> > > It's one thing if you test for "am I in hardware interrupt context".
> > > Those tests aren't great either, but at least they make sense.
> > >
> > > But a driver - or some library routine - making a difference based on
> > > some nebulous "can I schedule" is fundamentally and basically WRONG.
> > >
> > > If some code changes behavior, it needs to be explicit to the *caller*
> > > of that code.
> > >
> > > So this is why GFP_ATOMIC is fine, but "if (!can_schedule())
> > > do_something_atomic()" is pure shite.
> > >
> > > And I am not IN THE LEAST interested in trying to help people doing
> > > pure shite. We need to fix them. Like the crypto code is getting
> > > fixed.
> >
> > Just figured I'll throw my +1 in from reading too many (gpu) drivers.
> > Code that tries to cleverly adjust its behaviour depending upon the
> > context it's running in is harder to understand and blows up in more
> > interesting ways. We still have drm_can_sleep() and it's mostly just
> > used for debug code, and I've largely ended up just deleting
> > everything that used it because when you're driver is blowing up the
> > last thing you want is to realize your debug code and output can't be
> > relied upon. Or worse, that the only Oops you have is the one in the
> > debug code, because the real one scrolled away - the original idea
> > behind drm_can_sleep was to make all the modeset code work
> > automagically both in normal ioctl/kworker context and in the panic
> > handlers or kgdb callbacks. Wishful thinking at best.
> >
> > Also at least for me that extends to everything, e.g. I much prefer
> > explicit spin_lock and spin_lock_irq vs magic spin_lock_irqsave for
> > locks shared with interrupt handlers, since the former two gives me
> > clear information from which contexts such function can be called.
> > Other end is the memalloc_no*_save/restore functions, where I recently
> > made a real big fool of myself because I didn't realize how much that
> > impacts everything that's run within - suddenly "GFP_KERNEL for small
> > stuff never fails" is wrong everywhere.
> >
> > It's all great for debugging and sanity checks (and we run with all
> > that stuff enabled in our CI), but really semantic changes depending
> > upon magic context checks freak my out :-)
>
> All fair, but some of us need to write code that must handle being
> invoked from a wide variety of contexts.  Now perhaps you like the idea of
> call_rcu() for schedulable contexts, call_rcu_nosched() when preemption
> is disabled, call_rcu_irqs_are_disabled() when interrupts are disabled,
> call_rcu_raw_atomic() from contexts where (for example) raw spinlocks
> are held, and so on.  However, from what I can see, most people instead
> consistently prefer that the RCU API instead be consolidated.
>
> Some in-flight cache-efficiency work for kvfree_rcu() and call_rcu()
> needs to be able to allocate memory occasionally.  It can do that when
> invoked from some contexts, but not when invoked from others.  Right now,
> in !PREEMPT kernels, it cannot tell, and must either do things to the
> memory allocators that some of the MM hate or must unnecessarily invoke
> workqueues.  Thomas's patches would allow the code to just allocate in
> the common case when these primitives are invoked from contexts where
> allocation is permitted.
>
> If we want to restrict access to the can_schedule() or whatever primitive,
> fine and good.  We can add a check to checkpatch.pl, for example.  Maybe
> we can go back to the old brlock approach of requiring certain people's
> review for each addition to the kernel.
>
> But there really are use cases that it would greatly help.

We can deadlock in random fun places if random stuff we're calling
suddenly starts allocating. Sometimes. Maybe once in a blue moon, to
make it extra fun to reproduce. Maybe most driver subsystems are less
brittle, but gpu drivers definitely need to know about the details for
exactly this example. And yes gpu drivers use rcu for freeing
dma_fence structures, and that tends to happen in code that we only
recently figured out should really not allocate memory.

I think minimally you need to throw in an unconditional
fs_reclaim_acquire();fs_reclaim_release(); so that everyone who runs
with full debugging knows what might happen. It's kinda like
might_sleep, but a lot more specific. might_sleep() alone is not
enough, because in the specific code paths I'm thinking of (and
created special lockdep annotations for just recently) sleeping is
allowed, but any memory allocations with GFP_RECLAIM set are no-go.

Cheers, Daniel




--
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch


^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-16 20:29                 ` Daniel Vetter
  0 siblings, 0 replies; 282+ messages in thread
From: Daniel Vetter @ 2020-09-16 20:29 UTC (permalink / raw)
  To: Paul E. McKenney
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, Linux-MM,
	open list:KERNEL SELFTEST FRAMEWORK, linux-hexagon, Will Deacon,
	Ingo Molnar, Anton Ivanov, linux-arch, Vincent Guittot,
	Herbert Xu, Brian Cain, Richard Weinberger, Russell King,
	Ard Biesheuvel, David Airlie, Ingo Molnar, Geert Uytterhoeven,
	Mel Gorman, intel-gfx, Matt Turner, Valentin Schneider,
	linux-xtensa, Shuah Khan, Jeff Dike, linux-um, Josh Triplett,
	Steven Rostedt, rcu, linux-m68k, Ivan Kokshaysky, Rodrigo Vivi,
	Thomas Gleixner, Dietmar Eggemann, Linux ARM, Richard Henderson,
	Chris Zankel, Max Filippov, Daniel Bristot de Oliveira, LKML,
	alpha, Mathieu Desnoyers, Andrew Morton, Linus Torvalds

On Wed, Sep 16, 2020 at 5:29 PM Paul E. McKenney <paulmck@kernel.org> wrote:
>
> On Wed, Sep 16, 2020 at 09:37:17AM +0200, Daniel Vetter wrote:
> > On Tue, Sep 15, 2020 at 7:35 PM Linus Torvalds
> > <torvalds@linux-foundation.org> wrote:
> > >
> > > On Tue, Sep 15, 2020 at 1:39 AM Thomas Gleixner <tglx@linutronix.de> wrote:
> > > >
> > > > OTOH, having a working 'preemptible()' or maybe better named
> > > > 'can_schedule()' check makes tons of sense to make decisions about
> > > > allocation modes or other things.
> > >
> > > No. I think that those kinds of decisions about actual behavior are
> > > always simply fundamentally wrong.
> > >
> > > Note that this is very different from having warnings about invalid
> > > use. THAT is correct. It may not warn in all configurations, but that
> > > doesn't matter: what matters is that it warns in common enough
> > > configurations that developers will catch it.
> > >
> > > So having a warning in "might_sleep()" that doesn't always trigger,
> > > because you have a limited configuration that can't even detect the
> > > situation, that's fine and dandy and intentional.
> > >
> > > But having code like
> > >
> > >        if (can_schedule())
> > >            .. do something different ..
> > >
> > > is fundamentally complete and utter garbage.
> > >
> > > It's one thing if you test for "am I in hardware interrupt context".
> > > Those tests aren't great either, but at least they make sense.
> > >
> > > But a driver - or some library routine - making a difference based on
> > > some nebulous "can I schedule" is fundamentally and basically WRONG.
> > >
> > > If some code changes behavior, it needs to be explicit to the *caller*
> > > of that code.
> > >
> > > So this is why GFP_ATOMIC is fine, but "if (!can_schedule())
> > > do_something_atomic()" is pure shite.
> > >
> > > And I am not IN THE LEAST interested in trying to help people doing
> > > pure shite. We need to fix them. Like the crypto code is getting
> > > fixed.
> >
> > Just figured I'll throw my +1 in from reading too many (gpu) drivers.
> > Code that tries to cleverly adjust its behaviour depending upon the
> > context it's running in is harder to understand and blows up in more
> > interesting ways. We still have drm_can_sleep() and it's mostly just
> > used for debug code, and I've largely ended up just deleting
> > everything that used it because when you're driver is blowing up the
> > last thing you want is to realize your debug code and output can't be
> > relied upon. Or worse, that the only Oops you have is the one in the
> > debug code, because the real one scrolled away - the original idea
> > behind drm_can_sleep was to make all the modeset code work
> > automagically both in normal ioctl/kworker context and in the panic
> > handlers or kgdb callbacks. Wishful thinking at best.
> >
> > Also at least for me that extends to everything, e.g. I much prefer
> > explicit spin_lock and spin_lock_irq vs magic spin_lock_irqsave for
> > locks shared with interrupt handlers, since the former two gives me
> > clear information from which contexts such function can be called.
> > Other end is the memalloc_no*_save/restore functions, where I recently
> > made a real big fool of myself because I didn't realize how much that
> > impacts everything that's run within - suddenly "GFP_KERNEL for small
> > stuff never fails" is wrong everywhere.
> >
> > It's all great for debugging and sanity checks (and we run with all
> > that stuff enabled in our CI), but really semantic changes depending
> > upon magic context checks freak my out :-)
>
> All fair, but some of us need to write code that must handle being
> invoked from a wide variety of contexts.  Now perhaps you like the idea of
> call_rcu() for schedulable contexts, call_rcu_nosched() when preemption
> is disabled, call_rcu_irqs_are_disabled() when interrupts are disabled,
> call_rcu_raw_atomic() from contexts where (for example) raw spinlocks
> are held, and so on.  However, from what I can see, most people instead
> consistently prefer that the RCU API instead be consolidated.
>
> Some in-flight cache-efficiency work for kvfree_rcu() and call_rcu()
> needs to be able to allocate memory occasionally.  It can do that when
> invoked from some contexts, but not when invoked from others.  Right now,
> in !PREEMPT kernels, it cannot tell, and must either do things to the
> memory allocators that some of the MM hate or must unnecessarily invoke
> workqueues.  Thomas's patches would allow the code to just allocate in
> the common case when these primitives are invoked from contexts where
> allocation is permitted.
>
> If we want to restrict access to the can_schedule() or whatever primitive,
> fine and good.  We can add a check to checkpatch.pl, for example.  Maybe
> we can go back to the old brlock approach of requiring certain people's
> review for each addition to the kernel.
>
> But there really are use cases that it would greatly help.

We can deadlock in random fun places if random stuff we're calling
suddenly starts allocating. Sometimes. Maybe once in a blue moon, to
make it extra fun to reproduce. Maybe most driver subsystems are less
brittle, but gpu drivers definitely need to know about the details for
exactly this example. And yes gpu drivers use rcu for freeing
dma_fence structures, and that tends to happen in code that we only
recently figured out should really not allocate memory.

I think minimally you need to throw in an unconditional
fs_reclaim_acquire();fs_reclaim_release(); so that everyone who runs
with full debugging knows what might happen. It's kinda like
might_sleep, but a lot more specific. might_sleep() alone is not
enough, because in the specific code paths I'm thinking of (and
created special lockdep annotations for just recently) sleeping is
allowed, but any memory allocations with GFP_RECLAIM set are no-go.

Cheers, Daniel




--
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [Intel-gfx] [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-16 20:29                 ` Daniel Vetter
  0 siblings, 0 replies; 282+ messages in thread
From: Daniel Vetter @ 2020-09-16 20:29 UTC (permalink / raw)
  To: Paul E. McKenney
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, Linux-MM,
	open list:KERNEL SELFTEST FRAMEWORK, linux-hexagon, Will Deacon,
	Ingo Molnar, Anton Ivanov, linux-arch, Herbert Xu, Brian Cain,
	Richard Weinberger, Russell King, Ard Biesheuvel, David Airlie,
	Ingo Molnar, Geert Uytterhoeven, Mel Gorman, intel-gfx,
	Matt Turner, Valentin Schneider, linux-xtensa, Shuah Khan,
	Jeff Dike, linux-um, Josh Triplett, Steven Rostedt, rcu,
	linux-m68k, Ivan Kokshaysky, Thomas Gleixner, Dietmar Eggemann,
	Linux ARM, Richard Henderson, Chris Zankel, Max Filippov,
	Daniel Bristot de Oliveira, LKML, alpha, Mathieu Desnoyers,
	Andrew Morton, Linus Torvalds

On Wed, Sep 16, 2020 at 5:29 PM Paul E. McKenney <paulmck@kernel.org> wrote:
>
> On Wed, Sep 16, 2020 at 09:37:17AM +0200, Daniel Vetter wrote:
> > On Tue, Sep 15, 2020 at 7:35 PM Linus Torvalds
> > <torvalds@linux-foundation.org> wrote:
> > >
> > > On Tue, Sep 15, 2020 at 1:39 AM Thomas Gleixner <tglx@linutronix.de> wrote:
> > > >
> > > > OTOH, having a working 'preemptible()' or maybe better named
> > > > 'can_schedule()' check makes tons of sense to make decisions about
> > > > allocation modes or other things.
> > >
> > > No. I think that those kinds of decisions about actual behavior are
> > > always simply fundamentally wrong.
> > >
> > > Note that this is very different from having warnings about invalid
> > > use. THAT is correct. It may not warn in all configurations, but that
> > > doesn't matter: what matters is that it warns in common enough
> > > configurations that developers will catch it.
> > >
> > > So having a warning in "might_sleep()" that doesn't always trigger,
> > > because you have a limited configuration that can't even detect the
> > > situation, that's fine and dandy and intentional.
> > >
> > > But having code like
> > >
> > >        if (can_schedule())
> > >            .. do something different ..
> > >
> > > is fundamentally complete and utter garbage.
> > >
> > > It's one thing if you test for "am I in hardware interrupt context".
> > > Those tests aren't great either, but at least they make sense.
> > >
> > > But a driver - or some library routine - making a difference based on
> > > some nebulous "can I schedule" is fundamentally and basically WRONG.
> > >
> > > If some code changes behavior, it needs to be explicit to the *caller*
> > > of that code.
> > >
> > > So this is why GFP_ATOMIC is fine, but "if (!can_schedule())
> > > do_something_atomic()" is pure shite.
> > >
> > > And I am not IN THE LEAST interested in trying to help people doing
> > > pure shite. We need to fix them. Like the crypto code is getting
> > > fixed.
> >
> > Just figured I'll throw my +1 in from reading too many (gpu) drivers.
> > Code that tries to cleverly adjust its behaviour depending upon the
> > context it's running in is harder to understand and blows up in more
> > interesting ways. We still have drm_can_sleep() and it's mostly just
> > used for debug code, and I've largely ended up just deleting
> > everything that used it because when you're driver is blowing up the
> > last thing you want is to realize your debug code and output can't be
> > relied upon. Or worse, that the only Oops you have is the one in the
> > debug code, because the real one scrolled away - the original idea
> > behind drm_can_sleep was to make all the modeset code work
> > automagically both in normal ioctl/kworker context and in the panic
> > handlers or kgdb callbacks. Wishful thinking at best.
> >
> > Also at least for me that extends to everything, e.g. I much prefer
> > explicit spin_lock and spin_lock_irq vs magic spin_lock_irqsave for
> > locks shared with interrupt handlers, since the former two gives me
> > clear information from which contexts such function can be called.
> > Other end is the memalloc_no*_save/restore functions, where I recently
> > made a real big fool of myself because I didn't realize how much that
> > impacts everything that's run within - suddenly "GFP_KERNEL for small
> > stuff never fails" is wrong everywhere.
> >
> > It's all great for debugging and sanity checks (and we run with all
> > that stuff enabled in our CI), but really semantic changes depending
> > upon magic context checks freak my out :-)
>
> All fair, but some of us need to write code that must handle being
> invoked from a wide variety of contexts.  Now perhaps you like the idea of
> call_rcu() for schedulable contexts, call_rcu_nosched() when preemption
> is disabled, call_rcu_irqs_are_disabled() when interrupts are disabled,
> call_rcu_raw_atomic() from contexts where (for example) raw spinlocks
> are held, and so on.  However, from what I can see, most people instead
> consistently prefer that the RCU API instead be consolidated.
>
> Some in-flight cache-efficiency work for kvfree_rcu() and call_rcu()
> needs to be able to allocate memory occasionally.  It can do that when
> invoked from some contexts, but not when invoked from others.  Right now,
> in !PREEMPT kernels, it cannot tell, and must either do things to the
> memory allocators that some of the MM hate or must unnecessarily invoke
> workqueues.  Thomas's patches would allow the code to just allocate in
> the common case when these primitives are invoked from contexts where
> allocation is permitted.
>
> If we want to restrict access to the can_schedule() or whatever primitive,
> fine and good.  We can add a check to checkpatch.pl, for example.  Maybe
> we can go back to the old brlock approach of requiring certain people's
> review for each addition to the kernel.
>
> But there really are use cases that it would greatly help.

We can deadlock in random fun places if random stuff we're calling
suddenly starts allocating. Sometimes. Maybe once in a blue moon, to
make it extra fun to reproduce. Maybe most driver subsystems are less
brittle, but gpu drivers definitely need to know about the details for
exactly this example. And yes gpu drivers use rcu for freeing
dma_fence structures, and that tends to happen in code that we only
recently figured out should really not allocate memory.

I think minimally you need to throw in an unconditional
fs_reclaim_acquire();fs_reclaim_release(); so that everyone who runs
with full debugging knows what might happen. It's kinda like
might_sleep, but a lot more specific. might_sleep() alone is not
enough, because in the specific code paths I'm thinking of (and
created special lockdep annotations for just recently) sleeping is
allowed, but any memory allocations with GFP_RECLAIM set are no-go.

Cheers, Daniel




--
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-16 20:29                 ` Daniel Vetter
  0 siblings, 0 replies; 282+ messages in thread
From: Daniel Vetter @ 2020-09-16 20:29 UTC (permalink / raw)
  To: Paul E. McKenney
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, Linux-MM,
	open list:KERNEL SELFTEST FRAMEWORK, linux-hexagon, Will Deacon,
	Ingo Molnar, Anton Ivanov, linux-arch, Herbert Xu, Brian Cain,
	Richard Weinberger, Russell King, Ard Biesheuvel, David Airlie,
	Ingo Molnar, Geert Uytterhoeven, Mel Gorman, intel-gfx

On Wed, Sep 16, 2020 at 5:29 PM Paul E. McKenney <paulmck@kernel.org> wrote:
>
> On Wed, Sep 16, 2020 at 09:37:17AM +0200, Daniel Vetter wrote:
> > On Tue, Sep 15, 2020 at 7:35 PM Linus Torvalds
> > <torvalds@linux-foundation.org> wrote:
> > >
> > > On Tue, Sep 15, 2020 at 1:39 AM Thomas Gleixner <tglx@linutronix.de> wrote:
> > > >
> > > > OTOH, having a working 'preemptible()' or maybe better named
> > > > 'can_schedule()' check makes tons of sense to make decisions about
> > > > allocation modes or other things.
> > >
> > > No. I think that those kinds of decisions about actual behavior are
> > > always simply fundamentally wrong.
> > >
> > > Note that this is very different from having warnings about invalid
> > > use. THAT is correct. It may not warn in all configurations, but that
> > > doesn't matter: what matters is that it warns in common enough
> > > configurations that developers will catch it.
> > >
> > > So having a warning in "might_sleep()" that doesn't always trigger,
> > > because you have a limited configuration that can't even detect the
> > > situation, that's fine and dandy and intentional.
> > >
> > > But having code like
> > >
> > >        if (can_schedule())
> > >            .. do something different ..
> > >
> > > is fundamentally complete and utter garbage.
> > >
> > > It's one thing if you test for "am I in hardware interrupt context".
> > > Those tests aren't great either, but at least they make sense.
> > >
> > > But a driver - or some library routine - making a difference based on
> > > some nebulous "can I schedule" is fundamentally and basically WRONG.
> > >
> > > If some code changes behavior, it needs to be explicit to the *caller*
> > > of that code.
> > >
> > > So this is why GFP_ATOMIC is fine, but "if (!can_schedule())
> > > do_something_atomic()" is pure shite.
> > >
> > > And I am not IN THE LEAST interested in trying to help people doing
> > > pure shite. We need to fix them. Like the crypto code is getting
> > > fixed.
> >
> > Just figured I'll throw my +1 in from reading too many (gpu) drivers.
> > Code that tries to cleverly adjust its behaviour depending upon the
> > context it's running in is harder to understand and blows up in more
> > interesting ways. We still have drm_can_sleep() and it's mostly just
> > used for debug code, and I've largely ended up just deleting
> > everything that used it because when you're driver is blowing up the
> > last thing you want is to realize your debug code and output can't be
> > relied upon. Or worse, that the only Oops you have is the one in the
> > debug code, because the real one scrolled away - the original idea
> > behind drm_can_sleep was to make all the modeset code work
> > automagically both in normal ioctl/kworker context and in the panic
> > handlers or kgdb callbacks. Wishful thinking at best.
> >
> > Also at least for me that extends to everything, e.g. I much prefer
> > explicit spin_lock and spin_lock_irq vs magic spin_lock_irqsave for
> > locks shared with interrupt handlers, since the former two gives me
> > clear information from which contexts such function can be called.
> > Other end is the memalloc_no*_save/restore functions, where I recently
> > made a real big fool of myself because I didn't realize how much that
> > impacts everything that's run within - suddenly "GFP_KERNEL for small
> > stuff never fails" is wrong everywhere.
> >
> > It's all great for debugging and sanity checks (and we run with all
> > that stuff enabled in our CI), but really semantic changes depending
> > upon magic context checks freak my out :-)
>
> All fair, but some of us need to write code that must handle being
> invoked from a wide variety of contexts.  Now perhaps you like the idea of
> call_rcu() for schedulable contexts, call_rcu_nosched() when preemption
> is disabled, call_rcu_irqs_are_disabled() when interrupts are disabled,
> call_rcu_raw_atomic() from contexts where (for example) raw spinlocks
> are held, and so on.  However, from what I can see, most people instead
> consistently prefer that the RCU API instead be consolidated.
>
> Some in-flight cache-efficiency work for kvfree_rcu() and call_rcu()
> needs to be able to allocate memory occasionally.  It can do that when
> invoked from some contexts, but not when invoked from others.  Right now,
> in !PREEMPT kernels, it cannot tell, and must either do things to the
> memory allocators that some of the MM hate or must unnecessarily invoke
> workqueues.  Thomas's patches would allow the code to just allocate in
> the common case when these primitives are invoked from contexts where
> allocation is permitted.
>
> If we want to restrict access to the can_schedule() or whatever primitive,
> fine and good.  We can add a check to checkpatch.pl, for example.  Maybe
> we can go back to the old brlock approach of requiring certain people's
> review for each addition to the kernel.
>
> But there really are use cases that it would greatly help.

We can deadlock in random fun places if random stuff we're calling
suddenly starts allocating. Sometimes. Maybe once in a blue moon, to
make it extra fun to reproduce. Maybe most driver subsystems are less
brittle, but gpu drivers definitely need to know about the details for
exactly this example. And yes gpu drivers use rcu for freeing
dma_fence structures, and that tends to happen in code that we only
recently figured out should really not allocate memory.

I think minimally you need to throw in an unconditional
fs_reclaim_acquire();fs_reclaim_release(); so that everyone who runs
with full debugging knows what might happen. It's kinda like
might_sleep, but a lot more specific. might_sleep() alone is not
enough, because in the specific code paths I'm thinking of (and
created special lockdep annotations for just recently) sleeping is
allowed, but any memory allocations with GFP_RECLAIM set are no-go.

Cheers, Daniel




--
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
  2020-09-16 18:32                 ` Linus Torvalds
                                     ` (2 preceding siblings ...)
  (?)
@ 2020-09-16 20:43                   ` Paul E. McKenney
  -1 siblings, 0 replies; 282+ messages in thread
From: Paul E. McKenney @ 2020-09-16 20:43 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Daniel Vetter, Thomas Gleixner, Ard Biesheuvel, Herbert Xu, LKML,
	linux-arch, Sebastian Andrzej Siewior, Valentin Schneider,
	Richard Henderson, Ivan Kokshaysky, Matt Turner, alpha,
	Jeff Dike, Richard Weinberger, Anton Ivanov, linux-um,
	Brian Cain, linux-hexagon, Geert Uytterhoeven, linux-m68k,
	Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
	Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman,
	Daniel Bristot de Oliveira, Will Deacon, Andrew Morton, Linux-MM,
	Ingo Molnar, Russell King, Linux ARM, Chris Zankel, Max Filippov,
	linux-xtensa, Jani Nikula, Joonas Lahtinen, Rodrigo Vivi,
	David Airlie, intel-gfx, dri-devel, Josh Triplett,
	Mathieu Desnoyers, Lai Jiangshan, Shuah Khan, rcu,
	open list:KERNEL SELFTEST FRAMEWORK

On Wed, Sep 16, 2020 at 11:32:00AM -0700, Linus Torvalds wrote:
> On Wed, Sep 16, 2020 at 8:29 AM Paul E. McKenney <paulmck@kernel.org> wrote:
> >
> > All fair, but some of us need to write code that must handle being
> > invoked from a wide variety of contexts.
> 
> Note that I think that core functionality is different from random drivers.
> 
> Of course core code can (and will) look at things like
> 
>         if (in_interrupt())
>             .. schedule work asynchronously ..
> 
> because core code ends up being called from odd places, and code like
> that is expected to have understanding of the rules it plays with.
> 
> But something like RCU is a very different beast from some "walk the
> scatter-gather list" code.
> 
> RCU does its work in the background, and works with lots of different
> things. And it's so core and used everywhere that it knows about these
> things. I mean, we literally have special code explicitly to let RCU
> know "we entered kernel context now".
> 
> But something like a driver list walking thing should not be doing
> different things behind peoples back depending on whether they hold
> spinlocks or not. It should either just work regardless, or there
> should be a flag (or special interface) for the "you're being called
> in a crtitical region".
> 
> Because dynamically changing behavior really is very confusing.

Whew!  I feel much better now.  ;-)

							Thanx, Paul

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-16 20:43                   ` Paul E. McKenney
  0 siblings, 0 replies; 282+ messages in thread
From: Paul E. McKenney @ 2020-09-16 20:43 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Daniel Vetter, Thomas Gleixner, Ard Biesheuvel, Herbert Xu, LKML,
	linux-arch, Sebastian Andrzej Siewior, Valentin Schneider,
	Richard Henderson, Ivan Kokshaysky, Matt Turner, alpha,
	Jeff Dike, Richard Weinberger, Anton Ivanov, linux-um,
	Brian Cain, linux-hexagon, Geert Uytterhoeven, linux-m68k,
	Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
	Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman,
	Daniel Bristot de Oliveira, Will Deacon, Andrew Morton, Linux-MM,
	Ingo Molnar, Russell King, Linux ARM, Chris Zankel, Max Filippov,
	linux-xtensa, Jani Nikula, Joonas Lahtinen, Rodrigo Vivi,
	David Airlie, intel-gfx, dri-devel, Josh Triplett,
	Mathieu Desnoyers, Lai Jiangshan, Shuah Khan, rcu,
	open list:KERNEL SELFTEST FRAMEWORK

On Wed, Sep 16, 2020 at 11:32:00AM -0700, Linus Torvalds wrote:
> On Wed, Sep 16, 2020 at 8:29 AM Paul E. McKenney <paulmck@kernel.org> wrote:
> >
> > All fair, but some of us need to write code that must handle being
> > invoked from a wide variety of contexts.
> 
> Note that I think that core functionality is different from random drivers.
> 
> Of course core code can (and will) look at things like
> 
>         if (in_interrupt())
>             .. schedule work asynchronously ..
> 
> because core code ends up being called from odd places, and code like
> that is expected to have understanding of the rules it plays with.
> 
> But something like RCU is a very different beast from some "walk the
> scatter-gather list" code.
> 
> RCU does its work in the background, and works with lots of different
> things. And it's so core and used everywhere that it knows about these
> things. I mean, we literally have special code explicitly to let RCU
> know "we entered kernel context now".
> 
> But something like a driver list walking thing should not be doing
> different things behind peoples back depending on whether they hold
> spinlocks or not. It should either just work regardless, or there
> should be a flag (or special interface) for the "you're being called
> in a crtitical region".
> 
> Because dynamically changing behavior really is very confusing.

Whew!  I feel much better now.  ;-)

							Thanx, Paul


^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-16 20:43                   ` Paul E. McKenney
  0 siblings, 0 replies; 282+ messages in thread
From: Paul E. McKenney @ 2020-09-16 20:43 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, Linux-MM,
	open list:KERNEL SELFTEST FRAMEWORK, linux-hexagon, Will Deacon,
	Ingo Molnar, Anton Ivanov, linux-arch, Vincent Guittot,
	Herbert Xu, Brian Cain, Richard Weinberger, Russell King,
	Ard Biesheuvel, David Airlie, Ingo Molnar, Geert Uytterhoeven,
	Mel Gorman, intel-gfx, Matt Turner, Valentin Schneider,
	linux-xtensa, Shuah Khan, Jeff Dike, linux-um, Josh Triplett,
	Steven Rostedt, rcu, linux-m68k, Ivan Kokshaysky, Rodrigo Vivi,
	Thomas Gleixner, Dietmar Eggemann, Linux ARM, Richard Henderson,
	Chris Zankel, Max Filippov, LKML, alpha, Mathieu Desnoyers,
	Andrew Morton, Daniel Bristot de Oliveira

On Wed, Sep 16, 2020 at 11:32:00AM -0700, Linus Torvalds wrote:
> On Wed, Sep 16, 2020 at 8:29 AM Paul E. McKenney <paulmck@kernel.org> wrote:
> >
> > All fair, but some of us need to write code that must handle being
> > invoked from a wide variety of contexts.
> 
> Note that I think that core functionality is different from random drivers.
> 
> Of course core code can (and will) look at things like
> 
>         if (in_interrupt())
>             .. schedule work asynchronously ..
> 
> because core code ends up being called from odd places, and code like
> that is expected to have understanding of the rules it plays with.
> 
> But something like RCU is a very different beast from some "walk the
> scatter-gather list" code.
> 
> RCU does its work in the background, and works with lots of different
> things. And it's so core and used everywhere that it knows about these
> things. I mean, we literally have special code explicitly to let RCU
> know "we entered kernel context now".
> 
> But something like a driver list walking thing should not be doing
> different things behind peoples back depending on whether they hold
> spinlocks or not. It should either just work regardless, or there
> should be a flag (or special interface) for the "you're being called
> in a crtitical region".
> 
> Because dynamically changing behavior really is very confusing.

Whew!  I feel much better now.  ;-)

							Thanx, Paul
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [Intel-gfx] [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-16 20:43                   ` Paul E. McKenney
  0 siblings, 0 replies; 282+ messages in thread
From: Paul E. McKenney @ 2020-09-16 20:43 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, Linux-MM,
	open list:KERNEL SELFTEST FRAMEWORK, linux-hexagon, Will Deacon,
	Ingo Molnar, Anton Ivanov, linux-arch, Herbert Xu, Brian Cain,
	Richard Weinberger, Russell King, Ard Biesheuvel, David Airlie,
	Ingo Molnar, Geert Uytterhoeven, Mel Gorman, intel-gfx,
	Matt Turner, Valentin Schneider, linux-xtensa, Shuah Khan,
	Jeff Dike, linux-um, Josh Triplett, Steven Rostedt, rcu,
	linux-m68k, Ivan Kokshaysky, Thomas Gleixner, Dietmar Eggemann,
	Linux ARM, Richard Henderson, Chris Zankel, Max Filippov, LKML,
	alpha, Mathieu Desnoyers, Andrew Morton,
	Daniel Bristot de Oliveira

On Wed, Sep 16, 2020 at 11:32:00AM -0700, Linus Torvalds wrote:
> On Wed, Sep 16, 2020 at 8:29 AM Paul E. McKenney <paulmck@kernel.org> wrote:
> >
> > All fair, but some of us need to write code that must handle being
> > invoked from a wide variety of contexts.
> 
> Note that I think that core functionality is different from random drivers.
> 
> Of course core code can (and will) look at things like
> 
>         if (in_interrupt())
>             .. schedule work asynchronously ..
> 
> because core code ends up being called from odd places, and code like
> that is expected to have understanding of the rules it plays with.
> 
> But something like RCU is a very different beast from some "walk the
> scatter-gather list" code.
> 
> RCU does its work in the background, and works with lots of different
> things. And it's so core and used everywhere that it knows about these
> things. I mean, we literally have special code explicitly to let RCU
> know "we entered kernel context now".
> 
> But something like a driver list walking thing should not be doing
> different things behind peoples back depending on whether they hold
> spinlocks or not. It should either just work regardless, or there
> should be a flag (or special interface) for the "you're being called
> in a crtitical region".
> 
> Because dynamically changing behavior really is very confusing.

Whew!  I feel much better now.  ;-)

							Thanx, Paul
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-16 20:43                   ` Paul E. McKenney
  0 siblings, 0 replies; 282+ messages in thread
From: Paul E. McKenney @ 2020-09-16 20:43 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Daniel Vetter, Thomas Gleixner, Ard Biesheuvel, Herbert Xu, LKML,
	linux-arch, Sebastian Andrzej Siewior, Valentin Schneider,
	Richard Henderson, Ivan Kokshaysky, Matt Turner, alpha,
	Jeff Dike, Richard Weinberger, Anton Ivanov, linux-um,
	Brian Cain, linux-hexagon, Geert Uytterhoeven, l

On Wed, Sep 16, 2020 at 11:32:00AM -0700, Linus Torvalds wrote:
> On Wed, Sep 16, 2020 at 8:29 AM Paul E. McKenney <paulmck@kernel.org> wrote:
> >
> > All fair, but some of us need to write code that must handle being
> > invoked from a wide variety of contexts.
> 
> Note that I think that core functionality is different from random drivers.
> 
> Of course core code can (and will) look at things like
> 
>         if (in_interrupt())
>             .. schedule work asynchronously ..
> 
> because core code ends up being called from odd places, and code like
> that is expected to have understanding of the rules it plays with.
> 
> But something like RCU is a very different beast from some "walk the
> scatter-gather list" code.
> 
> RCU does its work in the background, and works with lots of different
> things. And it's so core and used everywhere that it knows about these
> things. I mean, we literally have special code explicitly to let RCU
> know "we entered kernel context now".
> 
> But something like a driver list walking thing should not be doing
> different things behind peoples back depending on whether they hold
> spinlocks or not. It should either just work regardless, or there
> should be a flag (or special interface) for the "you're being called
> in a crtitical region".
> 
> Because dynamically changing behavior really is very confusing.

Whew!  I feel much better now.  ;-)

							Thanx, Paul

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
  2020-09-16 19:23       ` Matthew Wilcox
                           ` (2 preceding siblings ...)
  (?)
@ 2020-09-16 20:48         ` Paul E. McKenney
  -1 siblings, 0 replies; 282+ messages in thread
From: Paul E. McKenney @ 2020-09-16 20:48 UTC (permalink / raw)
  To: Matthew Wilcox
  Cc: Thomas Gleixner, Linus Torvalds, LKML, linux-arch,
	Sebastian Andrzej Siewior, Valentin Schneider, Richard Henderson,
	Ivan Kokshaysky, Matt Turner, alpha, Jeff Dike,
	Richard Weinberger, Anton Ivanov, linux-um, Brian Cain,
	linux-hexagon, Geert Uytterhoeven, linux-m68k, Ingo Molnar,
	Peter Zijlstra, Juri Lelli, Vincent Guittot, Dietmar Eggemann,
	Steven Rostedt, Ben Segall, Mel Gorman,
	Daniel Bristot de Oliveira, Will Deacon, Andrew Morton, Linux-MM,
	Ingo Molnar, Russell King, Linux ARM, Chris Zankel, Max Filippov,
	linux-xtensa, Jani Nikula, Joonas Lahtinen, Rodrigo Vivi,
	David Airlie, Daniel Vetter, intel-gfx, dri-devel, Josh Triplett,
	Mathieu Desnoyers, Lai Jiangshan, Shuah Khan, rcu,
	open list:KERNEL SELFTEST FRAMEWORK

On Wed, Sep 16, 2020 at 08:23:52PM +0100, Matthew Wilcox wrote:
> On Mon, Sep 14, 2020 at 11:55:24PM +0200, Thomas Gleixner wrote:
> > But just look at any check which uses preemptible(), especially those
> > which check !preemptible():
> 
> hmm.
> 
> +++ b/include/linux/preempt.h
> @@ -180,7 +180,9 @@ do { \
>  
>  #define preempt_enable_no_resched() sched_preempt_enable_no_resched()
>  
> +#ifndef MODULE
>  #define preemptible()  (preempt_count() == 0 && !irqs_disabled())
> +#endif
>  
>  #ifdef CONFIG_PREEMPTION
>  #define preempt_enable() \
> 
> 
> $ git grep -w preemptible drivers
> (slightly trimmed by hand to remove, eg, comments)
> drivers/firmware/arm_sdei.c:    WARN_ON_ONCE(preemptible());
> drivers/firmware/arm_sdei.c:    WARN_ON_ONCE(preemptible());
> drivers/firmware/arm_sdei.c:    WARN_ON_ONCE(preemptible());
> drivers/firmware/arm_sdei.c:    WARN_ON_ONCE(preemptible());
> drivers/firmware/arm_sdei.c:    WARN_ON(preemptible());
> drivers/firmware/efi/efi-pstore.c:                            preemptible(), record->size, record->psi->buf);
> drivers/irqchip/irq-gic-v4.c:   WARN_ON(preemptible());
> drivers/irqchip/irq-gic-v4.c:   WARN_ON(preemptible());
> drivers/scsi/hisi_sas/hisi_sas_main.c:          if (!preemptible())
> drivers/xen/time.c:     BUG_ON(preemptible());
> 
> That only looks like two drivers that need more than WARNectomies.

I could easily imagine someone thinking that these did something in
CONFIG_PREEMPT_NONE=y kernels.  In fact, I could easily imagine myself
making that mistake.  :-/

> Although maybe rcu_read_load_sched_held() or rcu_read_lock_any_held()
> might get called from a module ...

But yes, from the rcutorture module for certain and also from any other
RCU-using module that includes the usual RCU debug checks.

							Thanx, Paul

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-16 20:48         ` Paul E. McKenney
  0 siblings, 0 replies; 282+ messages in thread
From: Paul E. McKenney @ 2020-09-16 20:48 UTC (permalink / raw)
  To: Matthew Wilcox
  Cc: Thomas Gleixner, Linus Torvalds, LKML, linux-arch,
	Sebastian Andrzej Siewior, Valentin Schneider, Richard Henderson,
	Ivan Kokshaysky, Matt Turner, alpha, Jeff Dike,
	Richard Weinberger, Anton Ivanov, linux-um, Brian Cain,
	linux-hexagon, Geert Uytterhoeven, linux-m68k, Ingo Molnar,
	Peter Zijlstra, Juri Lelli, Vincent Guittot, Dietmar Eggemann,
	Steven Rostedt, Ben Segall, Mel Gorman,
	Daniel Bristot de Oliveira, Will Deacon, Andrew Morton, Linux-MM,
	Ingo Molnar, Russell King, Linux ARM, Chris Zankel, Max Filippov,
	linux-xtensa, Jani Nikula, Joonas Lahtinen, Rodrigo Vivi,
	David Airlie, Daniel Vetter, intel-gfx, dri-devel, Josh Triplett,
	Mathieu Desnoyers, Lai Jiangshan, Shuah Khan, rcu,
	open list:KERNEL SELFTEST FRAMEWORK

On Wed, Sep 16, 2020 at 08:23:52PM +0100, Matthew Wilcox wrote:
> On Mon, Sep 14, 2020 at 11:55:24PM +0200, Thomas Gleixner wrote:
> > But just look at any check which uses preemptible(), especially those
> > which check !preemptible():
> 
> hmm.
> 
> +++ b/include/linux/preempt.h
> @@ -180,7 +180,9 @@ do { \
>  
>  #define preempt_enable_no_resched() sched_preempt_enable_no_resched()
>  
> +#ifndef MODULE
>  #define preemptible()  (preempt_count() == 0 && !irqs_disabled())
> +#endif
>  
>  #ifdef CONFIG_PREEMPTION
>  #define preempt_enable() \
> 
> 
> $ git grep -w preemptible drivers
> (slightly trimmed by hand to remove, eg, comments)
> drivers/firmware/arm_sdei.c:    WARN_ON_ONCE(preemptible());
> drivers/firmware/arm_sdei.c:    WARN_ON_ONCE(preemptible());
> drivers/firmware/arm_sdei.c:    WARN_ON_ONCE(preemptible());
> drivers/firmware/arm_sdei.c:    WARN_ON_ONCE(preemptible());
> drivers/firmware/arm_sdei.c:    WARN_ON(preemptible());
> drivers/firmware/efi/efi-pstore.c:                            preemptible(), record->size, record->psi->buf);
> drivers/irqchip/irq-gic-v4.c:   WARN_ON(preemptible());
> drivers/irqchip/irq-gic-v4.c:   WARN_ON(preemptible());
> drivers/scsi/hisi_sas/hisi_sas_main.c:          if (!preemptible())
> drivers/xen/time.c:     BUG_ON(preemptible());
> 
> That only looks like two drivers that need more than WARNectomies.

I could easily imagine someone thinking that these did something in
CONFIG_PREEMPT_NONE=y kernels.  In fact, I could easily imagine myself
making that mistake.  :-/

> Although maybe rcu_read_load_sched_held() or rcu_read_lock_any_held()
> might get called from a module ...

But yes, from the rcutorture module for certain and also from any other
RCU-using module that includes the usual RCU debug checks.

							Thanx, Paul


^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-16 20:48         ` Paul E. McKenney
  0 siblings, 0 replies; 282+ messages in thread
From: Paul E. McKenney @ 2020-09-16 20:48 UTC (permalink / raw)
  To: Matthew Wilcox
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, Linux-MM,
	open list:KERNEL SELFTEST FRAMEWORK, linux-hexagon, Will Deacon,
	Ingo Molnar, Anton Ivanov, linux-arch, Vincent Guittot,
	Brian Cain, Richard Weinberger, Russell King, David Airlie,
	Ingo Molnar, Geert Uytterhoeven, Mel Gorman, intel-gfx,
	Matt Turner, Valentin Schneider, linux-xtensa, Shuah Khan,
	Jeff Dike, linux-um, Josh Triplett, Steven Rostedt, rcu,
	linux-m68k, Ivan Kokshaysky, Rodrigo Vivi, Thomas Gleixner,
	Dietmar Eggemann, Linux ARM, Richard Henderson, Chris Zankel,
	Max Filippov, Daniel Bristot de Oliveira, LKML, alpha,
	Mathieu Desnoyers, Andrew Morton, Linus Torvalds

On Wed, Sep 16, 2020 at 08:23:52PM +0100, Matthew Wilcox wrote:
> On Mon, Sep 14, 2020 at 11:55:24PM +0200, Thomas Gleixner wrote:
> > But just look at any check which uses preemptible(), especially those
> > which check !preemptible():
> 
> hmm.
> 
> +++ b/include/linux/preempt.h
> @@ -180,7 +180,9 @@ do { \
>  
>  #define preempt_enable_no_resched() sched_preempt_enable_no_resched()
>  
> +#ifndef MODULE
>  #define preemptible()  (preempt_count() == 0 && !irqs_disabled())
> +#endif
>  
>  #ifdef CONFIG_PREEMPTION
>  #define preempt_enable() \
> 
> 
> $ git grep -w preemptible drivers
> (slightly trimmed by hand to remove, eg, comments)
> drivers/firmware/arm_sdei.c:    WARN_ON_ONCE(preemptible());
> drivers/firmware/arm_sdei.c:    WARN_ON_ONCE(preemptible());
> drivers/firmware/arm_sdei.c:    WARN_ON_ONCE(preemptible());
> drivers/firmware/arm_sdei.c:    WARN_ON_ONCE(preemptible());
> drivers/firmware/arm_sdei.c:    WARN_ON(preemptible());
> drivers/firmware/efi/efi-pstore.c:                            preemptible(), record->size, record->psi->buf);
> drivers/irqchip/irq-gic-v4.c:   WARN_ON(preemptible());
> drivers/irqchip/irq-gic-v4.c:   WARN_ON(preemptible());
> drivers/scsi/hisi_sas/hisi_sas_main.c:          if (!preemptible())
> drivers/xen/time.c:     BUG_ON(preemptible());
> 
> That only looks like two drivers that need more than WARNectomies.

I could easily imagine someone thinking that these did something in
CONFIG_PREEMPT_NONE=y kernels.  In fact, I could easily imagine myself
making that mistake.  :-/

> Although maybe rcu_read_load_sched_held() or rcu_read_lock_any_held()
> might get called from a module ...

But yes, from the rcutorture module for certain and also from any other
RCU-using module that includes the usual RCU debug checks.

							Thanx, Paul
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [Intel-gfx] [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-16 20:48         ` Paul E. McKenney
  0 siblings, 0 replies; 282+ messages in thread
From: Paul E. McKenney @ 2020-09-16 20:48 UTC (permalink / raw)
  To: Matthew Wilcox
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, Linux-MM,
	open list:KERNEL SELFTEST FRAMEWORK, linux-hexagon, Will Deacon,
	Ingo Molnar, Anton Ivanov, linux-arch, Brian Cain,
	Richard Weinberger, Russell King, David Airlie, Ingo Molnar,
	Geert Uytterhoeven, Mel Gorman, intel-gfx, Matt Turner,
	Valentin Schneider, linux-xtensa, Shuah Khan, Jeff Dike,
	linux-um, Josh Triplett, Steven Rostedt, rcu, linux-m68k,
	Ivan Kokshaysky, Thomas Gleixner, Dietmar Eggemann, Linux ARM,
	Richard Henderson, Chris Zankel, Max Filippov,
	Daniel Bristot de Oliveira, LKML, alpha, Mathieu Desnoyers,
	Andrew Morton, Linus Torvalds

On Wed, Sep 16, 2020 at 08:23:52PM +0100, Matthew Wilcox wrote:
> On Mon, Sep 14, 2020 at 11:55:24PM +0200, Thomas Gleixner wrote:
> > But just look at any check which uses preemptible(), especially those
> > which check !preemptible():
> 
> hmm.
> 
> +++ b/include/linux/preempt.h
> @@ -180,7 +180,9 @@ do { \
>  
>  #define preempt_enable_no_resched() sched_preempt_enable_no_resched()
>  
> +#ifndef MODULE
>  #define preemptible()  (preempt_count() == 0 && !irqs_disabled())
> +#endif
>  
>  #ifdef CONFIG_PREEMPTION
>  #define preempt_enable() \
> 
> 
> $ git grep -w preemptible drivers
> (slightly trimmed by hand to remove, eg, comments)
> drivers/firmware/arm_sdei.c:    WARN_ON_ONCE(preemptible());
> drivers/firmware/arm_sdei.c:    WARN_ON_ONCE(preemptible());
> drivers/firmware/arm_sdei.c:    WARN_ON_ONCE(preemptible());
> drivers/firmware/arm_sdei.c:    WARN_ON_ONCE(preemptible());
> drivers/firmware/arm_sdei.c:    WARN_ON(preemptible());
> drivers/firmware/efi/efi-pstore.c:                            preemptible(), record->size, record->psi->buf);
> drivers/irqchip/irq-gic-v4.c:   WARN_ON(preemptible());
> drivers/irqchip/irq-gic-v4.c:   WARN_ON(preemptible());
> drivers/scsi/hisi_sas/hisi_sas_main.c:          if (!preemptible())
> drivers/xen/time.c:     BUG_ON(preemptible());
> 
> That only looks like two drivers that need more than WARNectomies.

I could easily imagine someone thinking that these did something in
CONFIG_PREEMPT_NONE=y kernels.  In fact, I could easily imagine myself
making that mistake.  :-/

> Although maybe rcu_read_load_sched_held() or rcu_read_lock_any_held()
> might get called from a module ...

But yes, from the rcutorture module for certain and also from any other
RCU-using module that includes the usual RCU debug checks.

							Thanx, Paul
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-16 20:48         ` Paul E. McKenney
  0 siblings, 0 replies; 282+ messages in thread
From: Paul E. McKenney @ 2020-09-16 20:48 UTC (permalink / raw)
  To: Matthew Wilcox
  Cc: Thomas Gleixner, Linus Torvalds, LKML, linux-arch,
	Sebastian Andrzej Siewior, Valentin Schneider, Richard Henderson,
	Ivan Kokshaysky, Matt Turner, alpha, Jeff Dike,
	Richard Weinberger, Anton Ivanov, linux-um, Brian Cain,
	linux-hexagon, Geert Uytterhoeven, linux-m68k, Ingo Molnar

On Wed, Sep 16, 2020 at 08:23:52PM +0100, Matthew Wilcox wrote:
> On Mon, Sep 14, 2020 at 11:55:24PM +0200, Thomas Gleixner wrote:
> > But just look at any check which uses preemptible(), especially those
> > which check !preemptible():
> 
> hmm.
> 
> +++ b/include/linux/preempt.h
> @@ -180,7 +180,9 @@ do { \
>  
>  #define preempt_enable_no_resched() sched_preempt_enable_no_resched()
>  
> +#ifndef MODULE
>  #define preemptible()  (preempt_count() == 0 && !irqs_disabled())
> +#endif
>  
>  #ifdef CONFIG_PREEMPTION
>  #define preempt_enable() \
> 
> 
> $ git grep -w preemptible drivers
> (slightly trimmed by hand to remove, eg, comments)
> drivers/firmware/arm_sdei.c:    WARN_ON_ONCE(preemptible());
> drivers/firmware/arm_sdei.c:    WARN_ON_ONCE(preemptible());
> drivers/firmware/arm_sdei.c:    WARN_ON_ONCE(preemptible());
> drivers/firmware/arm_sdei.c:    WARN_ON_ONCE(preemptible());
> drivers/firmware/arm_sdei.c:    WARN_ON(preemptible());
> drivers/firmware/efi/efi-pstore.c:                            preemptible(), record->size, record->psi->buf);
> drivers/irqchip/irq-gic-v4.c:   WARN_ON(preemptible());
> drivers/irqchip/irq-gic-v4.c:   WARN_ON(preemptible());
> drivers/scsi/hisi_sas/hisi_sas_main.c:          if (!preemptible())
> drivers/xen/time.c:     BUG_ON(preemptible());
> 
> That only looks like two drivers that need more than WARNectomies.

I could easily imagine someone thinking that these did something in
CONFIG_PREEMPT_NONE=y kernels.  In fact, I could easily imagine myself
making that mistake.  :-/

> Although maybe rcu_read_load_sched_held() or rcu_read_lock_any_held()
> might get called from a module ...

But yes, from the rcutorture module for certain and also from any other
RCU-using module that includes the usual RCU debug checks.

							Thanx, Paul

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
  2020-09-16 20:29                 ` Daniel Vetter
                                     ` (2 preceding siblings ...)
  (?)
@ 2020-09-16 20:58                   ` Paul E. McKenney
  -1 siblings, 0 replies; 282+ messages in thread
From: Paul E. McKenney @ 2020-09-16 20:58 UTC (permalink / raw)
  To: Daniel Vetter
  Cc: Linus Torvalds, Thomas Gleixner, Ard Biesheuvel, Herbert Xu,
	LKML, linux-arch, Sebastian Andrzej Siewior, Valentin Schneider,
	Richard Henderson, Ivan Kokshaysky, Matt Turner, alpha,
	Jeff Dike, Richard Weinberger, Anton Ivanov, linux-um,
	Brian Cain, linux-hexagon, Geert Uytterhoeven, linux-m68k,
	Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
	Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman,
	Daniel Bristot de Oliveira, Will Deacon, Andrew Morton, Linux-MM,
	Ingo Molnar, Russell King, Linux ARM, Chris Zankel, Max Filippov,
	linux-xtensa, Jani Nikula, Joonas Lahtinen, Rodrigo Vivi,
	David Airlie, intel-gfx, dri-devel, Josh Triplett,
	Mathieu Desnoyers, Lai Jiangshan, Shuah Khan, rcu,
	open list:KERNEL SELFTEST FRAMEWORK

On Wed, Sep 16, 2020 at 10:29:06PM +0200, Daniel Vetter wrote:
> On Wed, Sep 16, 2020 at 5:29 PM Paul E. McKenney <paulmck@kernel.org> wrote:
> >
> > On Wed, Sep 16, 2020 at 09:37:17AM +0200, Daniel Vetter wrote:
> > > On Tue, Sep 15, 2020 at 7:35 PM Linus Torvalds
> > > <torvalds@linux-foundation.org> wrote:
> > > >
> > > > On Tue, Sep 15, 2020 at 1:39 AM Thomas Gleixner <tglx@linutronix.de> wrote:
> > > > >
> > > > > OTOH, having a working 'preemptible()' or maybe better named
> > > > > 'can_schedule()' check makes tons of sense to make decisions about
> > > > > allocation modes or other things.
> > > >
> > > > No. I think that those kinds of decisions about actual behavior are
> > > > always simply fundamentally wrong.
> > > >
> > > > Note that this is very different from having warnings about invalid
> > > > use. THAT is correct. It may not warn in all configurations, but that
> > > > doesn't matter: what matters is that it warns in common enough
> > > > configurations that developers will catch it.
> > > >
> > > > So having a warning in "might_sleep()" that doesn't always trigger,
> > > > because you have a limited configuration that can't even detect the
> > > > situation, that's fine and dandy and intentional.
> > > >
> > > > But having code like
> > > >
> > > >        if (can_schedule())
> > > >            .. do something different ..
> > > >
> > > > is fundamentally complete and utter garbage.
> > > >
> > > > It's one thing if you test for "am I in hardware interrupt context".
> > > > Those tests aren't great either, but at least they make sense.
> > > >
> > > > But a driver - or some library routine - making a difference based on
> > > > some nebulous "can I schedule" is fundamentally and basically WRONG.
> > > >
> > > > If some code changes behavior, it needs to be explicit to the *caller*
> > > > of that code.
> > > >
> > > > So this is why GFP_ATOMIC is fine, but "if (!can_schedule())
> > > > do_something_atomic()" is pure shite.
> > > >
> > > > And I am not IN THE LEAST interested in trying to help people doing
> > > > pure shite. We need to fix them. Like the crypto code is getting
> > > > fixed.
> > >
> > > Just figured I'll throw my +1 in from reading too many (gpu) drivers.
> > > Code that tries to cleverly adjust its behaviour depending upon the
> > > context it's running in is harder to understand and blows up in more
> > > interesting ways. We still have drm_can_sleep() and it's mostly just
> > > used for debug code, and I've largely ended up just deleting
> > > everything that used it because when you're driver is blowing up the
> > > last thing you want is to realize your debug code and output can't be
> > > relied upon. Or worse, that the only Oops you have is the one in the
> > > debug code, because the real one scrolled away - the original idea
> > > behind drm_can_sleep was to make all the modeset code work
> > > automagically both in normal ioctl/kworker context and in the panic
> > > handlers or kgdb callbacks. Wishful thinking at best.
> > >
> > > Also at least for me that extends to everything, e.g. I much prefer
> > > explicit spin_lock and spin_lock_irq vs magic spin_lock_irqsave for
> > > locks shared with interrupt handlers, since the former two gives me
> > > clear information from which contexts such function can be called.
> > > Other end is the memalloc_no*_save/restore functions, where I recently
> > > made a real big fool of myself because I didn't realize how much that
> > > impacts everything that's run within - suddenly "GFP_KERNEL for small
> > > stuff never fails" is wrong everywhere.
> > >
> > > It's all great for debugging and sanity checks (and we run with all
> > > that stuff enabled in our CI), but really semantic changes depending
> > > upon magic context checks freak my out :-)
> >
> > All fair, but some of us need to write code that must handle being
> > invoked from a wide variety of contexts.  Now perhaps you like the idea of
> > call_rcu() for schedulable contexts, call_rcu_nosched() when preemption
> > is disabled, call_rcu_irqs_are_disabled() when interrupts are disabled,
> > call_rcu_raw_atomic() from contexts where (for example) raw spinlocks
> > are held, and so on.  However, from what I can see, most people instead
> > consistently prefer that the RCU API instead be consolidated.
> >
> > Some in-flight cache-efficiency work for kvfree_rcu() and call_rcu()
> > needs to be able to allocate memory occasionally.  It can do that when
> > invoked from some contexts, but not when invoked from others.  Right now,
> > in !PREEMPT kernels, it cannot tell, and must either do things to the
> > memory allocators that some of the MM hate or must unnecessarily invoke
> > workqueues.  Thomas's patches would allow the code to just allocate in
> > the common case when these primitives are invoked from contexts where
> > allocation is permitted.
> >
> > If we want to restrict access to the can_schedule() or whatever primitive,
> > fine and good.  We can add a check to checkpatch.pl, for example.  Maybe
> > we can go back to the old brlock approach of requiring certain people's
> > review for each addition to the kernel.
> >
> > But there really are use cases that it would greatly help.
> 
> We can deadlock in random fun places if random stuff we're calling
> suddenly starts allocating. Sometimes. Maybe once in a blue moon, to
> make it extra fun to reproduce. Maybe most driver subsystems are less
> brittle, but gpu drivers definitely need to know about the details for
> exactly this example. And yes gpu drivers use rcu for freeing
> dma_fence structures, and that tends to happen in code that we only
> recently figured out should really not allocate memory.
> 
> I think minimally you need to throw in an unconditional
> fs_reclaim_acquire();fs_reclaim_release(); so that everyone who runs
> with full debugging knows what might happen. It's kinda like
> might_sleep, but a lot more specific. might_sleep() alone is not
> enough, because in the specific code paths I'm thinking of (and
> created special lockdep annotations for just recently) sleeping is
> allowed, but any memory allocations with GFP_RECLAIM set are no-go.

Completely agreed!  Any allocation on any free path must be handled
-extremely- carefully.  To that end...

First, there is always a fallback in case the allocation fails.  Which
might have performance or corner-case robustness issues, but which will
at least allow forward progress.  Second, we consulted with a number of
MM experts to arrive at appropriate GFP_* flags (and their patience is
greatly appreciated).  Third, the paths that can allocate will do so about
one time of 500, so any issues should be spotted sooner rather than later.

So you are quite right to be concerned, but I believe we will be doing the
right things.  And based on his previous track record, I am also quite
certain that Mr. Murphy will be on hand to provide me any additional
education that I might require.

Finally, I have noted down your point about fs_reclaim_acquire() and
fs_reclaim_release().  Whether or not they prove to be needed, I do
appreciate your calling them to my attention.

							Thanx, Paul

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-16 20:58                   ` Paul E. McKenney
  0 siblings, 0 replies; 282+ messages in thread
From: Paul E. McKenney @ 2020-09-16 20:58 UTC (permalink / raw)
  To: Daniel Vetter
  Cc: Linus Torvalds, Thomas Gleixner, Ard Biesheuvel, Herbert Xu,
	LKML, linux-arch, Sebastian Andrzej Siewior, Valentin Schneider,
	Richard Henderson, Ivan Kokshaysky, Matt Turner, alpha,
	Jeff Dike, Richard Weinberger, Anton Ivanov, linux-um,
	Brian Cain, linux-hexagon, Geert Uytterhoeven, linux-m68k,
	Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
	Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman,
	Daniel Bristot de Oliveira, Will Deacon, Andrew Morton, Linux-MM,
	Ingo Molnar, Russell King, Linux ARM, Chris Zankel, Max Filippov,
	linux-xtensa, Jani Nikula, Joonas Lahtinen, Rodrigo Vivi,
	David Airlie, intel-gfx, dri-devel, Josh Triplett,
	Mathieu Desnoyers, Lai Jiangshan, Shuah Khan, rcu,
	open list:KERNEL SELFTEST FRAMEWORK

On Wed, Sep 16, 2020 at 10:29:06PM +0200, Daniel Vetter wrote:
> On Wed, Sep 16, 2020 at 5:29 PM Paul E. McKenney <paulmck@kernel.org> wrote:
> >
> > On Wed, Sep 16, 2020 at 09:37:17AM +0200, Daniel Vetter wrote:
> > > On Tue, Sep 15, 2020 at 7:35 PM Linus Torvalds
> > > <torvalds@linux-foundation.org> wrote:
> > > >
> > > > On Tue, Sep 15, 2020 at 1:39 AM Thomas Gleixner <tglx@linutronix.de> wrote:
> > > > >
> > > > > OTOH, having a working 'preemptible()' or maybe better named
> > > > > 'can_schedule()' check makes tons of sense to make decisions about
> > > > > allocation modes or other things.
> > > >
> > > > No. I think that those kinds of decisions about actual behavior are
> > > > always simply fundamentally wrong.
> > > >
> > > > Note that this is very different from having warnings about invalid
> > > > use. THAT is correct. It may not warn in all configurations, but that
> > > > doesn't matter: what matters is that it warns in common enough
> > > > configurations that developers will catch it.
> > > >
> > > > So having a warning in "might_sleep()" that doesn't always trigger,
> > > > because you have a limited configuration that can't even detect the
> > > > situation, that's fine and dandy and intentional.
> > > >
> > > > But having code like
> > > >
> > > >        if (can_schedule())
> > > >            .. do something different ..
> > > >
> > > > is fundamentally complete and utter garbage.
> > > >
> > > > It's one thing if you test for "am I in hardware interrupt context".
> > > > Those tests aren't great either, but at least they make sense.
> > > >
> > > > But a driver - or some library routine - making a difference based on
> > > > some nebulous "can I schedule" is fundamentally and basically WRONG.
> > > >
> > > > If some code changes behavior, it needs to be explicit to the *caller*
> > > > of that code.
> > > >
> > > > So this is why GFP_ATOMIC is fine, but "if (!can_schedule())
> > > > do_something_atomic()" is pure shite.
> > > >
> > > > And I am not IN THE LEAST interested in trying to help people doing
> > > > pure shite. We need to fix them. Like the crypto code is getting
> > > > fixed.
> > >
> > > Just figured I'll throw my +1 in from reading too many (gpu) drivers.
> > > Code that tries to cleverly adjust its behaviour depending upon the
> > > context it's running in is harder to understand and blows up in more
> > > interesting ways. We still have drm_can_sleep() and it's mostly just
> > > used for debug code, and I've largely ended up just deleting
> > > everything that used it because when you're driver is blowing up the
> > > last thing you want is to realize your debug code and output can't be
> > > relied upon. Or worse, that the only Oops you have is the one in the
> > > debug code, because the real one scrolled away - the original idea
> > > behind drm_can_sleep was to make all the modeset code work
> > > automagically both in normal ioctl/kworker context and in the panic
> > > handlers or kgdb callbacks. Wishful thinking at best.
> > >
> > > Also at least for me that extends to everything, e.g. I much prefer
> > > explicit spin_lock and spin_lock_irq vs magic spin_lock_irqsave for
> > > locks shared with interrupt handlers, since the former two gives me
> > > clear information from which contexts such function can be called.
> > > Other end is the memalloc_no*_save/restore functions, where I recently
> > > made a real big fool of myself because I didn't realize how much that
> > > impacts everything that's run within - suddenly "GFP_KERNEL for small
> > > stuff never fails" is wrong everywhere.
> > >
> > > It's all great for debugging and sanity checks (and we run with all
> > > that stuff enabled in our CI), but really semantic changes depending
> > > upon magic context checks freak my out :-)
> >
> > All fair, but some of us need to write code that must handle being
> > invoked from a wide variety of contexts.  Now perhaps you like the idea of
> > call_rcu() for schedulable contexts, call_rcu_nosched() when preemption
> > is disabled, call_rcu_irqs_are_disabled() when interrupts are disabled,
> > call_rcu_raw_atomic() from contexts where (for example) raw spinlocks
> > are held, and so on.  However, from what I can see, most people instead
> > consistently prefer that the RCU API instead be consolidated.
> >
> > Some in-flight cache-efficiency work for kvfree_rcu() and call_rcu()
> > needs to be able to allocate memory occasionally.  It can do that when
> > invoked from some contexts, but not when invoked from others.  Right now,
> > in !PREEMPT kernels, it cannot tell, and must either do things to the
> > memory allocators that some of the MM hate or must unnecessarily invoke
> > workqueues.  Thomas's patches would allow the code to just allocate in
> > the common case when these primitives are invoked from contexts where
> > allocation is permitted.
> >
> > If we want to restrict access to the can_schedule() or whatever primitive,
> > fine and good.  We can add a check to checkpatch.pl, for example.  Maybe
> > we can go back to the old brlock approach of requiring certain people's
> > review for each addition to the kernel.
> >
> > But there really are use cases that it would greatly help.
> 
> We can deadlock in random fun places if random stuff we're calling
> suddenly starts allocating. Sometimes. Maybe once in a blue moon, to
> make it extra fun to reproduce. Maybe most driver subsystems are less
> brittle, but gpu drivers definitely need to know about the details for
> exactly this example. And yes gpu drivers use rcu for freeing
> dma_fence structures, and that tends to happen in code that we only
> recently figured out should really not allocate memory.
> 
> I think minimally you need to throw in an unconditional
> fs_reclaim_acquire();fs_reclaim_release(); so that everyone who runs
> with full debugging knows what might happen. It's kinda like
> might_sleep, but a lot more specific. might_sleep() alone is not
> enough, because in the specific code paths I'm thinking of (and
> created special lockdep annotations for just recently) sleeping is
> allowed, but any memory allocations with GFP_RECLAIM set are no-go.

Completely agreed!  Any allocation on any free path must be handled
-extremely- carefully.  To that end...

First, there is always a fallback in case the allocation fails.  Which
might have performance or corner-case robustness issues, but which will
at least allow forward progress.  Second, we consulted with a number of
MM experts to arrive at appropriate GFP_* flags (and their patience is
greatly appreciated).  Third, the paths that can allocate will do so about
one time of 500, so any issues should be spotted sooner rather than later.

So you are quite right to be concerned, but I believe we will be doing the
right things.  And based on his previous track record, I am also quite
certain that Mr. Murphy will be on hand to provide me any additional
education that I might require.

Finally, I have noted down your point about fs_reclaim_acquire() and
fs_reclaim_release().  Whether or not they prove to be needed, I do
appreciate your calling them to my attention.

							Thanx, Paul


^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-16 20:58                   ` Paul E. McKenney
  0 siblings, 0 replies; 282+ messages in thread
From: Paul E. McKenney @ 2020-09-16 20:58 UTC (permalink / raw)
  To: Daniel Vetter
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, Linux-MM,
	open list:KERNEL SELFTEST FRAMEWORK, linux-hexagon, Will Deacon,
	Ingo Molnar, Anton Ivanov, linux-arch, Vincent Guittot,
	Herbert Xu, Brian Cain, Richard Weinberger, Russell King,
	Ard Biesheuvel, David Airlie, Ingo Molnar, Geert Uytterhoeven,
	Mel Gorman, intel-gfx, Matt Turner, Valentin Schneider,
	linux-xtensa, Shuah Khan, Jeff Dike, linux-um, Josh Triplett,
	Steven Rostedt, rcu, linux-m68k, Ivan Kokshaysky, Rodrigo Vivi,
	Thomas Gleixner, Dietmar Eggemann, Linux ARM, Richard Henderson,
	Chris Zankel, Max Filippov, Daniel Bristot de Oliveira, LKML,
	alpha, Mathieu Desnoyers, Andrew Morton, Linus Torvalds

On Wed, Sep 16, 2020 at 10:29:06PM +0200, Daniel Vetter wrote:
> On Wed, Sep 16, 2020 at 5:29 PM Paul E. McKenney <paulmck@kernel.org> wrote:
> >
> > On Wed, Sep 16, 2020 at 09:37:17AM +0200, Daniel Vetter wrote:
> > > On Tue, Sep 15, 2020 at 7:35 PM Linus Torvalds
> > > <torvalds@linux-foundation.org> wrote:
> > > >
> > > > On Tue, Sep 15, 2020 at 1:39 AM Thomas Gleixner <tglx@linutronix.de> wrote:
> > > > >
> > > > > OTOH, having a working 'preemptible()' or maybe better named
> > > > > 'can_schedule()' check makes tons of sense to make decisions about
> > > > > allocation modes or other things.
> > > >
> > > > No. I think that those kinds of decisions about actual behavior are
> > > > always simply fundamentally wrong.
> > > >
> > > > Note that this is very different from having warnings about invalid
> > > > use. THAT is correct. It may not warn in all configurations, but that
> > > > doesn't matter: what matters is that it warns in common enough
> > > > configurations that developers will catch it.
> > > >
> > > > So having a warning in "might_sleep()" that doesn't always trigger,
> > > > because you have a limited configuration that can't even detect the
> > > > situation, that's fine and dandy and intentional.
> > > >
> > > > But having code like
> > > >
> > > >        if (can_schedule())
> > > >            .. do something different ..
> > > >
> > > > is fundamentally complete and utter garbage.
> > > >
> > > > It's one thing if you test for "am I in hardware interrupt context".
> > > > Those tests aren't great either, but at least they make sense.
> > > >
> > > > But a driver - or some library routine - making a difference based on
> > > > some nebulous "can I schedule" is fundamentally and basically WRONG.
> > > >
> > > > If some code changes behavior, it needs to be explicit to the *caller*
> > > > of that code.
> > > >
> > > > So this is why GFP_ATOMIC is fine, but "if (!can_schedule())
> > > > do_something_atomic()" is pure shite.
> > > >
> > > > And I am not IN THE LEAST interested in trying to help people doing
> > > > pure shite. We need to fix them. Like the crypto code is getting
> > > > fixed.
> > >
> > > Just figured I'll throw my +1 in from reading too many (gpu) drivers.
> > > Code that tries to cleverly adjust its behaviour depending upon the
> > > context it's running in is harder to understand and blows up in more
> > > interesting ways. We still have drm_can_sleep() and it's mostly just
> > > used for debug code, and I've largely ended up just deleting
> > > everything that used it because when you're driver is blowing up the
> > > last thing you want is to realize your debug code and output can't be
> > > relied upon. Or worse, that the only Oops you have is the one in the
> > > debug code, because the real one scrolled away - the original idea
> > > behind drm_can_sleep was to make all the modeset code work
> > > automagically both in normal ioctl/kworker context and in the panic
> > > handlers or kgdb callbacks. Wishful thinking at best.
> > >
> > > Also at least for me that extends to everything, e.g. I much prefer
> > > explicit spin_lock and spin_lock_irq vs magic spin_lock_irqsave for
> > > locks shared with interrupt handlers, since the former two gives me
> > > clear information from which contexts such function can be called.
> > > Other end is the memalloc_no*_save/restore functions, where I recently
> > > made a real big fool of myself because I didn't realize how much that
> > > impacts everything that's run within - suddenly "GFP_KERNEL for small
> > > stuff never fails" is wrong everywhere.
> > >
> > > It's all great for debugging and sanity checks (and we run with all
> > > that stuff enabled in our CI), but really semantic changes depending
> > > upon magic context checks freak my out :-)
> >
> > All fair, but some of us need to write code that must handle being
> > invoked from a wide variety of contexts.  Now perhaps you like the idea of
> > call_rcu() for schedulable contexts, call_rcu_nosched() when preemption
> > is disabled, call_rcu_irqs_are_disabled() when interrupts are disabled,
> > call_rcu_raw_atomic() from contexts where (for example) raw spinlocks
> > are held, and so on.  However, from what I can see, most people instead
> > consistently prefer that the RCU API instead be consolidated.
> >
> > Some in-flight cache-efficiency work for kvfree_rcu() and call_rcu()
> > needs to be able to allocate memory occasionally.  It can do that when
> > invoked from some contexts, but not when invoked from others.  Right now,
> > in !PREEMPT kernels, it cannot tell, and must either do things to the
> > memory allocators that some of the MM hate or must unnecessarily invoke
> > workqueues.  Thomas's patches would allow the code to just allocate in
> > the common case when these primitives are invoked from contexts where
> > allocation is permitted.
> >
> > If we want to restrict access to the can_schedule() or whatever primitive,
> > fine and good.  We can add a check to checkpatch.pl, for example.  Maybe
> > we can go back to the old brlock approach of requiring certain people's
> > review for each addition to the kernel.
> >
> > But there really are use cases that it would greatly help.
> 
> We can deadlock in random fun places if random stuff we're calling
> suddenly starts allocating. Sometimes. Maybe once in a blue moon, to
> make it extra fun to reproduce. Maybe most driver subsystems are less
> brittle, but gpu drivers definitely need to know about the details for
> exactly this example. And yes gpu drivers use rcu for freeing
> dma_fence structures, and that tends to happen in code that we only
> recently figured out should really not allocate memory.
> 
> I think minimally you need to throw in an unconditional
> fs_reclaim_acquire();fs_reclaim_release(); so that everyone who runs
> with full debugging knows what might happen. It's kinda like
> might_sleep, but a lot more specific. might_sleep() alone is not
> enough, because in the specific code paths I'm thinking of (and
> created special lockdep annotations for just recently) sleeping is
> allowed, but any memory allocations with GFP_RECLAIM set are no-go.

Completely agreed!  Any allocation on any free path must be handled
-extremely- carefully.  To that end...

First, there is always a fallback in case the allocation fails.  Which
might have performance or corner-case robustness issues, but which will
at least allow forward progress.  Second, we consulted with a number of
MM experts to arrive at appropriate GFP_* flags (and their patience is
greatly appreciated).  Third, the paths that can allocate will do so about
one time of 500, so any issues should be spotted sooner rather than later.

So you are quite right to be concerned, but I believe we will be doing the
right things.  And based on his previous track record, I am also quite
certain that Mr. Murphy will be on hand to provide me any additional
education that I might require.

Finally, I have noted down your point about fs_reclaim_acquire() and
fs_reclaim_release().  Whether or not they prove to be needed, I do
appreciate your calling them to my attention.

							Thanx, Paul
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [Intel-gfx] [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-16 20:58                   ` Paul E. McKenney
  0 siblings, 0 replies; 282+ messages in thread
From: Paul E. McKenney @ 2020-09-16 20:58 UTC (permalink / raw)
  To: Daniel Vetter
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, Linux-MM,
	open list:KERNEL SELFTEST FRAMEWORK, linux-hexagon, Will Deacon,
	Ingo Molnar, Anton Ivanov, linux-arch, Herbert Xu, Brian Cain,
	Richard Weinberger, Russell King, Ard Biesheuvel, David Airlie,
	Ingo Molnar, Geert Uytterhoeven, Mel Gorman, intel-gfx,
	Matt Turner, Valentin Schneider, linux-xtensa, Shuah Khan,
	Jeff Dike, linux-um, Josh Triplett, Steven Rostedt, rcu,
	linux-m68k, Ivan Kokshaysky, Thomas Gleixner, Dietmar Eggemann,
	Linux ARM, Richard Henderson, Chris Zankel, Max Filippov,
	Daniel Bristot de Oliveira, LKML, alpha, Mathieu Desnoyers,
	Andrew Morton, Linus Torvalds

On Wed, Sep 16, 2020 at 10:29:06PM +0200, Daniel Vetter wrote:
> On Wed, Sep 16, 2020 at 5:29 PM Paul E. McKenney <paulmck@kernel.org> wrote:
> >
> > On Wed, Sep 16, 2020 at 09:37:17AM +0200, Daniel Vetter wrote:
> > > On Tue, Sep 15, 2020 at 7:35 PM Linus Torvalds
> > > <torvalds@linux-foundation.org> wrote:
> > > >
> > > > On Tue, Sep 15, 2020 at 1:39 AM Thomas Gleixner <tglx@linutronix.de> wrote:
> > > > >
> > > > > OTOH, having a working 'preemptible()' or maybe better named
> > > > > 'can_schedule()' check makes tons of sense to make decisions about
> > > > > allocation modes or other things.
> > > >
> > > > No. I think that those kinds of decisions about actual behavior are
> > > > always simply fundamentally wrong.
> > > >
> > > > Note that this is very different from having warnings about invalid
> > > > use. THAT is correct. It may not warn in all configurations, but that
> > > > doesn't matter: what matters is that it warns in common enough
> > > > configurations that developers will catch it.
> > > >
> > > > So having a warning in "might_sleep()" that doesn't always trigger,
> > > > because you have a limited configuration that can't even detect the
> > > > situation, that's fine and dandy and intentional.
> > > >
> > > > But having code like
> > > >
> > > >        if (can_schedule())
> > > >            .. do something different ..
> > > >
> > > > is fundamentally complete and utter garbage.
> > > >
> > > > It's one thing if you test for "am I in hardware interrupt context".
> > > > Those tests aren't great either, but at least they make sense.
> > > >
> > > > But a driver - or some library routine - making a difference based on
> > > > some nebulous "can I schedule" is fundamentally and basically WRONG.
> > > >
> > > > If some code changes behavior, it needs to be explicit to the *caller*
> > > > of that code.
> > > >
> > > > So this is why GFP_ATOMIC is fine, but "if (!can_schedule())
> > > > do_something_atomic()" is pure shite.
> > > >
> > > > And I am not IN THE LEAST interested in trying to help people doing
> > > > pure shite. We need to fix them. Like the crypto code is getting
> > > > fixed.
> > >
> > > Just figured I'll throw my +1 in from reading too many (gpu) drivers.
> > > Code that tries to cleverly adjust its behaviour depending upon the
> > > context it's running in is harder to understand and blows up in more
> > > interesting ways. We still have drm_can_sleep() and it's mostly just
> > > used for debug code, and I've largely ended up just deleting
> > > everything that used it because when you're driver is blowing up the
> > > last thing you want is to realize your debug code and output can't be
> > > relied upon. Or worse, that the only Oops you have is the one in the
> > > debug code, because the real one scrolled away - the original idea
> > > behind drm_can_sleep was to make all the modeset code work
> > > automagically both in normal ioctl/kworker context and in the panic
> > > handlers or kgdb callbacks. Wishful thinking at best.
> > >
> > > Also at least for me that extends to everything, e.g. I much prefer
> > > explicit spin_lock and spin_lock_irq vs magic spin_lock_irqsave for
> > > locks shared with interrupt handlers, since the former two gives me
> > > clear information from which contexts such function can be called.
> > > Other end is the memalloc_no*_save/restore functions, where I recently
> > > made a real big fool of myself because I didn't realize how much that
> > > impacts everything that's run within - suddenly "GFP_KERNEL for small
> > > stuff never fails" is wrong everywhere.
> > >
> > > It's all great for debugging and sanity checks (and we run with all
> > > that stuff enabled in our CI), but really semantic changes depending
> > > upon magic context checks freak my out :-)
> >
> > All fair, but some of us need to write code that must handle being
> > invoked from a wide variety of contexts.  Now perhaps you like the idea of
> > call_rcu() for schedulable contexts, call_rcu_nosched() when preemption
> > is disabled, call_rcu_irqs_are_disabled() when interrupts are disabled,
> > call_rcu_raw_atomic() from contexts where (for example) raw spinlocks
> > are held, and so on.  However, from what I can see, most people instead
> > consistently prefer that the RCU API instead be consolidated.
> >
> > Some in-flight cache-efficiency work for kvfree_rcu() and call_rcu()
> > needs to be able to allocate memory occasionally.  It can do that when
> > invoked from some contexts, but not when invoked from others.  Right now,
> > in !PREEMPT kernels, it cannot tell, and must either do things to the
> > memory allocators that some of the MM hate or must unnecessarily invoke
> > workqueues.  Thomas's patches would allow the code to just allocate in
> > the common case when these primitives are invoked from contexts where
> > allocation is permitted.
> >
> > If we want to restrict access to the can_schedule() or whatever primitive,
> > fine and good.  We can add a check to checkpatch.pl, for example.  Maybe
> > we can go back to the old brlock approach of requiring certain people's
> > review for each addition to the kernel.
> >
> > But there really are use cases that it would greatly help.
> 
> We can deadlock in random fun places if random stuff we're calling
> suddenly starts allocating. Sometimes. Maybe once in a blue moon, to
> make it extra fun to reproduce. Maybe most driver subsystems are less
> brittle, but gpu drivers definitely need to know about the details for
> exactly this example. And yes gpu drivers use rcu for freeing
> dma_fence structures, and that tends to happen in code that we only
> recently figured out should really not allocate memory.
> 
> I think minimally you need to throw in an unconditional
> fs_reclaim_acquire();fs_reclaim_release(); so that everyone who runs
> with full debugging knows what might happen. It's kinda like
> might_sleep, but a lot more specific. might_sleep() alone is not
> enough, because in the specific code paths I'm thinking of (and
> created special lockdep annotations for just recently) sleeping is
> allowed, but any memory allocations with GFP_RECLAIM set are no-go.

Completely agreed!  Any allocation on any free path must be handled
-extremely- carefully.  To that end...

First, there is always a fallback in case the allocation fails.  Which
might have performance or corner-case robustness issues, but which will
at least allow forward progress.  Second, we consulted with a number of
MM experts to arrive at appropriate GFP_* flags (and their patience is
greatly appreciated).  Third, the paths that can allocate will do so about
one time of 500, so any issues should be spotted sooner rather than later.

So you are quite right to be concerned, but I believe we will be doing the
right things.  And based on his previous track record, I am also quite
certain that Mr. Murphy will be on hand to provide me any additional
education that I might require.

Finally, I have noted down your point about fs_reclaim_acquire() and
fs_reclaim_release().  Whether or not they prove to be needed, I do
appreciate your calling them to my attention.

							Thanx, Paul
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-16 20:58                   ` Paul E. McKenney
  0 siblings, 0 replies; 282+ messages in thread
From: Paul E. McKenney @ 2020-09-16 20:58 UTC (permalink / raw)
  To: Daniel Vetter
  Cc: Linus Torvalds, Thomas Gleixner, Ard Biesheuvel, Herbert Xu,
	LKML, linux-arch, Sebastian Andrzej Siewior, Valentin Schneider,
	Richard Henderson, Ivan Kokshaysky, Matt Turner, alpha,
	Jeff Dike, Richard Weinberger, Anton Ivanov, linux-um,
	Brian Cain, linux-hexagon, Geert Uytterhoeven

On Wed, Sep 16, 2020 at 10:29:06PM +0200, Daniel Vetter wrote:
> On Wed, Sep 16, 2020 at 5:29 PM Paul E. McKenney <paulmck@kernel.org> wrote:
> >
> > On Wed, Sep 16, 2020 at 09:37:17AM +0200, Daniel Vetter wrote:
> > > On Tue, Sep 15, 2020 at 7:35 PM Linus Torvalds
> > > <torvalds@linux-foundation.org> wrote:
> > > >
> > > > On Tue, Sep 15, 2020 at 1:39 AM Thomas Gleixner <tglx@linutronix.de> wrote:
> > > > >
> > > > > OTOH, having a working 'preemptible()' or maybe better named
> > > > > 'can_schedule()' check makes tons of sense to make decisions about
> > > > > allocation modes or other things.
> > > >
> > > > No. I think that those kinds of decisions about actual behavior are
> > > > always simply fundamentally wrong.
> > > >
> > > > Note that this is very different from having warnings about invalid
> > > > use. THAT is correct. It may not warn in all configurations, but that
> > > > doesn't matter: what matters is that it warns in common enough
> > > > configurations that developers will catch it.
> > > >
> > > > So having a warning in "might_sleep()" that doesn't always trigger,
> > > > because you have a limited configuration that can't even detect the
> > > > situation, that's fine and dandy and intentional.
> > > >
> > > > But having code like
> > > >
> > > >        if (can_schedule())
> > > >            .. do something different ..
> > > >
> > > > is fundamentally complete and utter garbage.
> > > >
> > > > It's one thing if you test for "am I in hardware interrupt context".
> > > > Those tests aren't great either, but at least they make sense.
> > > >
> > > > But a driver - or some library routine - making a difference based on
> > > > some nebulous "can I schedule" is fundamentally and basically WRONG.
> > > >
> > > > If some code changes behavior, it needs to be explicit to the *caller*
> > > > of that code.
> > > >
> > > > So this is why GFP_ATOMIC is fine, but "if (!can_schedule())
> > > > do_something_atomic()" is pure shite.
> > > >
> > > > And I am not IN THE LEAST interested in trying to help people doing
> > > > pure shite. We need to fix them. Like the crypto code is getting
> > > > fixed.
> > >
> > > Just figured I'll throw my +1 in from reading too many (gpu) drivers.
> > > Code that tries to cleverly adjust its behaviour depending upon the
> > > context it's running in is harder to understand and blows up in more
> > > interesting ways. We still have drm_can_sleep() and it's mostly just
> > > used for debug code, and I've largely ended up just deleting
> > > everything that used it because when you're driver is blowing up the
> > > last thing you want is to realize your debug code and output can't be
> > > relied upon. Or worse, that the only Oops you have is the one in the
> > > debug code, because the real one scrolled away - the original idea
> > > behind drm_can_sleep was to make all the modeset code work
> > > automagically both in normal ioctl/kworker context and in the panic
> > > handlers or kgdb callbacks. Wishful thinking at best.
> > >
> > > Also at least for me that extends to everything, e.g. I much prefer
> > > explicit spin_lock and spin_lock_irq vs magic spin_lock_irqsave for
> > > locks shared with interrupt handlers, since the former two gives me
> > > clear information from which contexts such function can be called.
> > > Other end is the memalloc_no*_save/restore functions, where I recently
> > > made a real big fool of myself because I didn't realize how much that
> > > impacts everything that's run within - suddenly "GFP_KERNEL for small
> > > stuff never fails" is wrong everywhere.
> > >
> > > It's all great for debugging and sanity checks (and we run with all
> > > that stuff enabled in our CI), but really semantic changes depending
> > > upon magic context checks freak my out :-)
> >
> > All fair, but some of us need to write code that must handle being
> > invoked from a wide variety of contexts.  Now perhaps you like the idea of
> > call_rcu() for schedulable contexts, call_rcu_nosched() when preemption
> > is disabled, call_rcu_irqs_are_disabled() when interrupts are disabled,
> > call_rcu_raw_atomic() from contexts where (for example) raw spinlocks
> > are held, and so on.  However, from what I can see, most people instead
> > consistently prefer that the RCU API instead be consolidated.
> >
> > Some in-flight cache-efficiency work for kvfree_rcu() and call_rcu()
> > needs to be able to allocate memory occasionally.  It can do that when
> > invoked from some contexts, but not when invoked from others.  Right now,
> > in !PREEMPT kernels, it cannot tell, and must either do things to the
> > memory allocators that some of the MM hate or must unnecessarily invoke
> > workqueues.  Thomas's patches would allow the code to just allocate in
> > the common case when these primitives are invoked from contexts where
> > allocation is permitted.
> >
> > If we want to restrict access to the can_schedule() or whatever primitive,
> > fine and good.  We can add a check to checkpatch.pl, for example.  Maybe
> > we can go back to the old brlock approach of requiring certain people's
> > review for each addition to the kernel.
> >
> > But there really are use cases that it would greatly help.
> 
> We can deadlock in random fun places if random stuff we're calling
> suddenly starts allocating. Sometimes. Maybe once in a blue moon, to
> make it extra fun to reproduce. Maybe most driver subsystems are less
> brittle, but gpu drivers definitely need to know about the details for
> exactly this example. And yes gpu drivers use rcu for freeing
> dma_fence structures, and that tends to happen in code that we only
> recently figured out should really not allocate memory.
> 
> I think minimally you need to throw in an unconditional
> fs_reclaim_acquire();fs_reclaim_release(); so that everyone who runs
> with full debugging knows what might happen. It's kinda like
> might_sleep, but a lot more specific. might_sleep() alone is not
> enough, because in the specific code paths I'm thinking of (and
> created special lockdep annotations for just recently) sleeping is
> allowed, but any memory allocations with GFP_RECLAIM set are no-go.

Completely agreed!  Any allocation on any free path must be handled
-extremely- carefully.  To that end...

First, there is always a fallback in case the allocation fails.  Which
might have performance or corner-case robustness issues, but which will
at least allow forward progress.  Second, we consulted with a number of
MM experts to arrive at appropriate GFP_* flags (and their patience is
greatly appreciated).  Third, the paths that can allocate will do so about
one time of 500, so any issues should be spotted sooner rather than later.

So you are quite right to be concerned, but I believe we will be doing the
right things.  And based on his previous track record, I am also quite
certain that Mr. Murphy will be on hand to provide me any additional
education that I might require.

Finally, I have noted down your point about fs_reclaim_acquire() and
fs_reclaim_release().  Whether or not they prove to be needed, I do
appreciate your calling them to my attention.

							Thanx, Paul

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
  2020-09-16 20:58                   ` Paul E. McKenney
                                       ` (2 preceding siblings ...)
  (?)
@ 2020-09-16 21:43                     ` Daniel Vetter
  -1 siblings, 0 replies; 282+ messages in thread
From: Daniel Vetter @ 2020-09-16 21:43 UTC (permalink / raw)
  To: Paul E. McKenney
  Cc: Linus Torvalds, Thomas Gleixner, Ard Biesheuvel, Herbert Xu,
	LKML, linux-arch, Sebastian Andrzej Siewior, Valentin Schneider,
	Richard Henderson, Ivan Kokshaysky, Matt Turner, alpha,
	Jeff Dike, Richard Weinberger, Anton Ivanov, linux-um,
	Brian Cain, linux-hexagon, Geert Uytterhoeven, linux-m68k,
	Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
	Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman,
	Daniel Bristot de Oliveira, Will Deacon, Andrew Morton, Linux-MM,
	Ingo Molnar, Russell King, Linux ARM, Chris Zankel, Max Filippov,
	linux-xtensa, Jani Nikula, Joonas Lahtinen, Rodrigo Vivi,
	David Airlie, intel-gfx, dri-devel, Josh Triplett,
	Mathieu Desnoyers, Lai Jiangshan, Shuah Khan, rcu,
	open list:KERNEL SELFTEST FRAMEWORK

On Wed, Sep 16, 2020 at 10:58 PM Paul E. McKenney <paulmck@kernel.org> wrote:
>
> On Wed, Sep 16, 2020 at 10:29:06PM +0200, Daniel Vetter wrote:
> > On Wed, Sep 16, 2020 at 5:29 PM Paul E. McKenney <paulmck@kernel.org> wrote:
> > >
> > > On Wed, Sep 16, 2020 at 09:37:17AM +0200, Daniel Vetter wrote:
> > > > On Tue, Sep 15, 2020 at 7:35 PM Linus Torvalds
> > > > <torvalds@linux-foundation.org> wrote:
> > > > >
> > > > > On Tue, Sep 15, 2020 at 1:39 AM Thomas Gleixner <tglx@linutronix.de> wrote:
> > > > > >
> > > > > > OTOH, having a working 'preemptible()' or maybe better named
> > > > > > 'can_schedule()' check makes tons of sense to make decisions about
> > > > > > allocation modes or other things.
> > > > >
> > > > > No. I think that those kinds of decisions about actual behavior are
> > > > > always simply fundamentally wrong.
> > > > >
> > > > > Note that this is very different from having warnings about invalid
> > > > > use. THAT is correct. It may not warn in all configurations, but that
> > > > > doesn't matter: what matters is that it warns in common enough
> > > > > configurations that developers will catch it.
> > > > >
> > > > > So having a warning in "might_sleep()" that doesn't always trigger,
> > > > > because you have a limited configuration that can't even detect the
> > > > > situation, that's fine and dandy and intentional.
> > > > >
> > > > > But having code like
> > > > >
> > > > >        if (can_schedule())
> > > > >            .. do something different ..
> > > > >
> > > > > is fundamentally complete and utter garbage.
> > > > >
> > > > > It's one thing if you test for "am I in hardware interrupt context".
> > > > > Those tests aren't great either, but at least they make sense.
> > > > >
> > > > > But a driver - or some library routine - making a difference based on
> > > > > some nebulous "can I schedule" is fundamentally and basically WRONG.
> > > > >
> > > > > If some code changes behavior, it needs to be explicit to the *caller*
> > > > > of that code.
> > > > >
> > > > > So this is why GFP_ATOMIC is fine, but "if (!can_schedule())
> > > > > do_something_atomic()" is pure shite.
> > > > >
> > > > > And I am not IN THE LEAST interested in trying to help people doing
> > > > > pure shite. We need to fix them. Like the crypto code is getting
> > > > > fixed.
> > > >
> > > > Just figured I'll throw my +1 in from reading too many (gpu) drivers.
> > > > Code that tries to cleverly adjust its behaviour depending upon the
> > > > context it's running in is harder to understand and blows up in more
> > > > interesting ways. We still have drm_can_sleep() and it's mostly just
> > > > used for debug code, and I've largely ended up just deleting
> > > > everything that used it because when you're driver is blowing up the
> > > > last thing you want is to realize your debug code and output can't be
> > > > relied upon. Or worse, that the only Oops you have is the one in the
> > > > debug code, because the real one scrolled away - the original idea
> > > > behind drm_can_sleep was to make all the modeset code work
> > > > automagically both in normal ioctl/kworker context and in the panic
> > > > handlers or kgdb callbacks. Wishful thinking at best.
> > > >
> > > > Also at least for me that extends to everything, e.g. I much prefer
> > > > explicit spin_lock and spin_lock_irq vs magic spin_lock_irqsave for
> > > > locks shared with interrupt handlers, since the former two gives me
> > > > clear information from which contexts such function can be called.
> > > > Other end is the memalloc_no*_save/restore functions, where I recently
> > > > made a real big fool of myself because I didn't realize how much that
> > > > impacts everything that's run within - suddenly "GFP_KERNEL for small
> > > > stuff never fails" is wrong everywhere.
> > > >
> > > > It's all great for debugging and sanity checks (and we run with all
> > > > that stuff enabled in our CI), but really semantic changes depending
> > > > upon magic context checks freak my out :-)
> > >
> > > All fair, but some of us need to write code that must handle being
> > > invoked from a wide variety of contexts.  Now perhaps you like the idea of
> > > call_rcu() for schedulable contexts, call_rcu_nosched() when preemption
> > > is disabled, call_rcu_irqs_are_disabled() when interrupts are disabled,
> > > call_rcu_raw_atomic() from contexts where (for example) raw spinlocks
> > > are held, and so on.  However, from what I can see, most people instead
> > > consistently prefer that the RCU API instead be consolidated.
> > >
> > > Some in-flight cache-efficiency work for kvfree_rcu() and call_rcu()
> > > needs to be able to allocate memory occasionally.  It can do that when
> > > invoked from some contexts, but not when invoked from others.  Right now,
> > > in !PREEMPT kernels, it cannot tell, and must either do things to the
> > > memory allocators that some of the MM hate or must unnecessarily invoke
> > > workqueues.  Thomas's patches would allow the code to just allocate in
> > > the common case when these primitives are invoked from contexts where
> > > allocation is permitted.
> > >
> > > If we want to restrict access to the can_schedule() or whatever primitive,
> > > fine and good.  We can add a check to checkpatch.pl, for example.  Maybe
> > > we can go back to the old brlock approach of requiring certain people's
> > > review for each addition to the kernel.
> > >
> > > But there really are use cases that it would greatly help.
> >
> > We can deadlock in random fun places if random stuff we're calling
> > suddenly starts allocating. Sometimes. Maybe once in a blue moon, to
> > make it extra fun to reproduce. Maybe most driver subsystems are less
> > brittle, but gpu drivers definitely need to know about the details for
> > exactly this example. And yes gpu drivers use rcu for freeing
> > dma_fence structures, and that tends to happen in code that we only
> > recently figured out should really not allocate memory.
> >
> > I think minimally you need to throw in an unconditional
> > fs_reclaim_acquire();fs_reclaim_release(); so that everyone who runs
> > with full debugging knows what might happen. It's kinda like
> > might_sleep, but a lot more specific. might_sleep() alone is not
> > enough, because in the specific code paths I'm thinking of (and
> > created special lockdep annotations for just recently) sleeping is
> > allowed, but any memory allocations with GFP_RECLAIM set are no-go.
>
> Completely agreed!  Any allocation on any free path must be handled
> -extremely- carefully.  To that end...
>
> First, there is always a fallback in case the allocation fails.  Which
> might have performance or corner-case robustness issues, but which will
> at least allow forward progress.  Second, we consulted with a number of
> MM experts to arrive at appropriate GFP_* flags (and their patience is
> greatly appreciated).  Third, the paths that can allocate will do so about
> one time of 500, so any issues should be spotted sooner rather than later.
>
> So you are quite right to be concerned, but I believe we will be doing the
> right things.  And based on his previous track record, I am also quite
> certain that Mr. Murphy will be on hand to provide me any additional
> education that I might require.
>
> Finally, I have noted down your point about fs_reclaim_acquire() and
> fs_reclaim_release().  Whether or not they prove to be needed, I do
> appreciate your calling them to my attention.

I just realized that since these dma_fence structs are refcounted and
userspace can hold references (directly, it can pass them around
behind file descriptors) we might never hit such a path until slightly
unusual or evil userspace does something interesting. Do you have
links to those patches? Some googling didn't turn up anything. I can
then figure out whether it's better to risk not spotting issues with
call_rcu vs slapping a memalloc_noio_save/restore around all these
critical section which force-degrades any allocation to GFP_ATOMIC at
most, but has the risk that we run into code that assumes "GFP_KERNEL
never fails for small stuff" and has a decidedly less tested fallback
path than rcu code.
-Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-16 21:43                     ` Daniel Vetter
  0 siblings, 0 replies; 282+ messages in thread
From: Daniel Vetter @ 2020-09-16 21:43 UTC (permalink / raw)
  To: Paul E. McKenney
  Cc: Linus Torvalds, Thomas Gleixner, Ard Biesheuvel, Herbert Xu,
	LKML, linux-arch, Sebastian Andrzej Siewior, Valentin Schneider,
	Richard Henderson, Ivan Kokshaysky, Matt Turner, alpha,
	Jeff Dike, Richard Weinberger, Anton Ivanov, linux-um,
	Brian Cain, linux-hexagon, Geert Uytterhoeven, linux-m68k,
	Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
	Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman,
	Daniel Bristot de Oliveira, Will Deacon, Andrew Morton, Linux-MM,
	Ingo Molnar, Russell King, Linux ARM, Chris Zankel, Max Filippov,
	linux-xtensa, Jani Nikula, Joonas Lahtinen, Rodrigo Vivi,
	David Airlie, intel-gfx, dri-devel, Josh Triplett,
	Mathieu Desnoyers, Lai Jiangshan, Shuah Khan, rcu,
	open list:KERNEL SELFTEST FRAMEWORK

On Wed, Sep 16, 2020 at 10:58 PM Paul E. McKenney <paulmck@kernel.org> wrote:
>
> On Wed, Sep 16, 2020 at 10:29:06PM +0200, Daniel Vetter wrote:
> > On Wed, Sep 16, 2020 at 5:29 PM Paul E. McKenney <paulmck@kernel.org> wrote:
> > >
> > > On Wed, Sep 16, 2020 at 09:37:17AM +0200, Daniel Vetter wrote:
> > > > On Tue, Sep 15, 2020 at 7:35 PM Linus Torvalds
> > > > <torvalds@linux-foundation.org> wrote:
> > > > >
> > > > > On Tue, Sep 15, 2020 at 1:39 AM Thomas Gleixner <tglx@linutronix.de> wrote:
> > > > > >
> > > > > > OTOH, having a working 'preemptible()' or maybe better named
> > > > > > 'can_schedule()' check makes tons of sense to make decisions about
> > > > > > allocation modes or other things.
> > > > >
> > > > > No. I think that those kinds of decisions about actual behavior are
> > > > > always simply fundamentally wrong.
> > > > >
> > > > > Note that this is very different from having warnings about invalid
> > > > > use. THAT is correct. It may not warn in all configurations, but that
> > > > > doesn't matter: what matters is that it warns in common enough
> > > > > configurations that developers will catch it.
> > > > >
> > > > > So having a warning in "might_sleep()" that doesn't always trigger,
> > > > > because you have a limited configuration that can't even detect the
> > > > > situation, that's fine and dandy and intentional.
> > > > >
> > > > > But having code like
> > > > >
> > > > >        if (can_schedule())
> > > > >            .. do something different ..
> > > > >
> > > > > is fundamentally complete and utter garbage.
> > > > >
> > > > > It's one thing if you test for "am I in hardware interrupt context".
> > > > > Those tests aren't great either, but at least they make sense.
> > > > >
> > > > > But a driver - or some library routine - making a difference based on
> > > > > some nebulous "can I schedule" is fundamentally and basically WRONG.
> > > > >
> > > > > If some code changes behavior, it needs to be explicit to the *caller*
> > > > > of that code.
> > > > >
> > > > > So this is why GFP_ATOMIC is fine, but "if (!can_schedule())
> > > > > do_something_atomic()" is pure shite.
> > > > >
> > > > > And I am not IN THE LEAST interested in trying to help people doing
> > > > > pure shite. We need to fix them. Like the crypto code is getting
> > > > > fixed.
> > > >
> > > > Just figured I'll throw my +1 in from reading too many (gpu) drivers.
> > > > Code that tries to cleverly adjust its behaviour depending upon the
> > > > context it's running in is harder to understand and blows up in more
> > > > interesting ways. We still have drm_can_sleep() and it's mostly just
> > > > used for debug code, and I've largely ended up just deleting
> > > > everything that used it because when you're driver is blowing up the
> > > > last thing you want is to realize your debug code and output can't be
> > > > relied upon. Or worse, that the only Oops you have is the one in the
> > > > debug code, because the real one scrolled away - the original idea
> > > > behind drm_can_sleep was to make all the modeset code work
> > > > automagically both in normal ioctl/kworker context and in the panic
> > > > handlers or kgdb callbacks. Wishful thinking at best.
> > > >
> > > > Also at least for me that extends to everything, e.g. I much prefer
> > > > explicit spin_lock and spin_lock_irq vs magic spin_lock_irqsave for
> > > > locks shared with interrupt handlers, since the former two gives me
> > > > clear information from which contexts such function can be called.
> > > > Other end is the memalloc_no*_save/restore functions, where I recently
> > > > made a real big fool of myself because I didn't realize how much that
> > > > impacts everything that's run within - suddenly "GFP_KERNEL for small
> > > > stuff never fails" is wrong everywhere.
> > > >
> > > > It's all great for debugging and sanity checks (and we run with all
> > > > that stuff enabled in our CI), but really semantic changes depending
> > > > upon magic context checks freak my out :-)
> > >
> > > All fair, but some of us need to write code that must handle being
> > > invoked from a wide variety of contexts.  Now perhaps you like the idea of
> > > call_rcu() for schedulable contexts, call_rcu_nosched() when preemption
> > > is disabled, call_rcu_irqs_are_disabled() when interrupts are disabled,
> > > call_rcu_raw_atomic() from contexts where (for example) raw spinlocks
> > > are held, and so on.  However, from what I can see, most people instead
> > > consistently prefer that the RCU API instead be consolidated.
> > >
> > > Some in-flight cache-efficiency work for kvfree_rcu() and call_rcu()
> > > needs to be able to allocate memory occasionally.  It can do that when
> > > invoked from some contexts, but not when invoked from others.  Right now,
> > > in !PREEMPT kernels, it cannot tell, and must either do things to the
> > > memory allocators that some of the MM hate or must unnecessarily invoke
> > > workqueues.  Thomas's patches would allow the code to just allocate in
> > > the common case when these primitives are invoked from contexts where
> > > allocation is permitted.
> > >
> > > If we want to restrict access to the can_schedule() or whatever primitive,
> > > fine and good.  We can add a check to checkpatch.pl, for example.  Maybe
> > > we can go back to the old brlock approach of requiring certain people's
> > > review for each addition to the kernel.
> > >
> > > But there really are use cases that it would greatly help.
> >
> > We can deadlock in random fun places if random stuff we're calling
> > suddenly starts allocating. Sometimes. Maybe once in a blue moon, to
> > make it extra fun to reproduce. Maybe most driver subsystems are less
> > brittle, but gpu drivers definitely need to know about the details for
> > exactly this example. And yes gpu drivers use rcu for freeing
> > dma_fence structures, and that tends to happen in code that we only
> > recently figured out should really not allocate memory.
> >
> > I think minimally you need to throw in an unconditional
> > fs_reclaim_acquire();fs_reclaim_release(); so that everyone who runs
> > with full debugging knows what might happen. It's kinda like
> > might_sleep, but a lot more specific. might_sleep() alone is not
> > enough, because in the specific code paths I'm thinking of (and
> > created special lockdep annotations for just recently) sleeping is
> > allowed, but any memory allocations with GFP_RECLAIM set are no-go.
>
> Completely agreed!  Any allocation on any free path must be handled
> -extremely- carefully.  To that end...
>
> First, there is always a fallback in case the allocation fails.  Which
> might have performance or corner-case robustness issues, but which will
> at least allow forward progress.  Second, we consulted with a number of
> MM experts to arrive at appropriate GFP_* flags (and their patience is
> greatly appreciated).  Third, the paths that can allocate will do so about
> one time of 500, so any issues should be spotted sooner rather than later.
>
> So you are quite right to be concerned, but I believe we will be doing the
> right things.  And based on his previous track record, I am also quite
> certain that Mr. Murphy will be on hand to provide me any additional
> education that I might require.
>
> Finally, I have noted down your point about fs_reclaim_acquire() and
> fs_reclaim_release().  Whether or not they prove to be needed, I do
> appreciate your calling them to my attention.

I just realized that since these dma_fence structs are refcounted and
userspace can hold references (directly, it can pass them around
behind file descriptors) we might never hit such a path until slightly
unusual or evil userspace does something interesting. Do you have
links to those patches? Some googling didn't turn up anything. I can
then figure out whether it's better to risk not spotting issues with
call_rcu vs slapping a memalloc_noio_save/restore around all these
critical section which force-degrades any allocation to GFP_ATOMIC at
most, but has the risk that we run into code that assumes "GFP_KERNEL
never fails for small stuff" and has a decidedly less tested fallback
path than rcu code.
-Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch


^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-16 21:43                     ` Daniel Vetter
  0 siblings, 0 replies; 282+ messages in thread
From: Daniel Vetter @ 2020-09-16 21:43 UTC (permalink / raw)
  To: Paul E. McKenney
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, Linux-MM,
	open list:KERNEL SELFTEST FRAMEWORK, linux-hexagon, Will Deacon,
	Ingo Molnar, Anton Ivanov, linux-arch, Vincent Guittot,
	Herbert Xu, Brian Cain, Richard Weinberger, Russell King,
	Ard Biesheuvel, David Airlie, Ingo Molnar, Geert Uytterhoeven,
	Mel Gorman, intel-gfx, Matt Turner, Valentin Schneider,
	linux-xtensa, Shuah Khan, Jeff Dike, linux-um, Josh Triplett,
	Steven Rostedt, rcu, linux-m68k, Ivan Kokshaysky, Rodrigo Vivi,
	Thomas Gleixner, Dietmar Eggemann, Linux ARM, Richard Henderson,
	Chris Zankel, Max Filippov, Daniel Bristot de Oliveira, LKML,
	alpha, Mathieu Desnoyers, Andrew Morton, Linus Torvalds

On Wed, Sep 16, 2020 at 10:58 PM Paul E. McKenney <paulmck@kernel.org> wrote:
>
> On Wed, Sep 16, 2020 at 10:29:06PM +0200, Daniel Vetter wrote:
> > On Wed, Sep 16, 2020 at 5:29 PM Paul E. McKenney <paulmck@kernel.org> wrote:
> > >
> > > On Wed, Sep 16, 2020 at 09:37:17AM +0200, Daniel Vetter wrote:
> > > > On Tue, Sep 15, 2020 at 7:35 PM Linus Torvalds
> > > > <torvalds@linux-foundation.org> wrote:
> > > > >
> > > > > On Tue, Sep 15, 2020 at 1:39 AM Thomas Gleixner <tglx@linutronix.de> wrote:
> > > > > >
> > > > > > OTOH, having a working 'preemptible()' or maybe better named
> > > > > > 'can_schedule()' check makes tons of sense to make decisions about
> > > > > > allocation modes or other things.
> > > > >
> > > > > No. I think that those kinds of decisions about actual behavior are
> > > > > always simply fundamentally wrong.
> > > > >
> > > > > Note that this is very different from having warnings about invalid
> > > > > use. THAT is correct. It may not warn in all configurations, but that
> > > > > doesn't matter: what matters is that it warns in common enough
> > > > > configurations that developers will catch it.
> > > > >
> > > > > So having a warning in "might_sleep()" that doesn't always trigger,
> > > > > because you have a limited configuration that can't even detect the
> > > > > situation, that's fine and dandy and intentional.
> > > > >
> > > > > But having code like
> > > > >
> > > > >        if (can_schedule())
> > > > >            .. do something different ..
> > > > >
> > > > > is fundamentally complete and utter garbage.
> > > > >
> > > > > It's one thing if you test for "am I in hardware interrupt context".
> > > > > Those tests aren't great either, but at least they make sense.
> > > > >
> > > > > But a driver - or some library routine - making a difference based on
> > > > > some nebulous "can I schedule" is fundamentally and basically WRONG.
> > > > >
> > > > > If some code changes behavior, it needs to be explicit to the *caller*
> > > > > of that code.
> > > > >
> > > > > So this is why GFP_ATOMIC is fine, but "if (!can_schedule())
> > > > > do_something_atomic()" is pure shite.
> > > > >
> > > > > And I am not IN THE LEAST interested in trying to help people doing
> > > > > pure shite. We need to fix them. Like the crypto code is getting
> > > > > fixed.
> > > >
> > > > Just figured I'll throw my +1 in from reading too many (gpu) drivers.
> > > > Code that tries to cleverly adjust its behaviour depending upon the
> > > > context it's running in is harder to understand and blows up in more
> > > > interesting ways. We still have drm_can_sleep() and it's mostly just
> > > > used for debug code, and I've largely ended up just deleting
> > > > everything that used it because when you're driver is blowing up the
> > > > last thing you want is to realize your debug code and output can't be
> > > > relied upon. Or worse, that the only Oops you have is the one in the
> > > > debug code, because the real one scrolled away - the original idea
> > > > behind drm_can_sleep was to make all the modeset code work
> > > > automagically both in normal ioctl/kworker context and in the panic
> > > > handlers or kgdb callbacks. Wishful thinking at best.
> > > >
> > > > Also at least for me that extends to everything, e.g. I much prefer
> > > > explicit spin_lock and spin_lock_irq vs magic spin_lock_irqsave for
> > > > locks shared with interrupt handlers, since the former two gives me
> > > > clear information from which contexts such function can be called.
> > > > Other end is the memalloc_no*_save/restore functions, where I recently
> > > > made a real big fool of myself because I didn't realize how much that
> > > > impacts everything that's run within - suddenly "GFP_KERNEL for small
> > > > stuff never fails" is wrong everywhere.
> > > >
> > > > It's all great for debugging and sanity checks (and we run with all
> > > > that stuff enabled in our CI), but really semantic changes depending
> > > > upon magic context checks freak my out :-)
> > >
> > > All fair, but some of us need to write code that must handle being
> > > invoked from a wide variety of contexts.  Now perhaps you like the idea of
> > > call_rcu() for schedulable contexts, call_rcu_nosched() when preemption
> > > is disabled, call_rcu_irqs_are_disabled() when interrupts are disabled,
> > > call_rcu_raw_atomic() from contexts where (for example) raw spinlocks
> > > are held, and so on.  However, from what I can see, most people instead
> > > consistently prefer that the RCU API instead be consolidated.
> > >
> > > Some in-flight cache-efficiency work for kvfree_rcu() and call_rcu()
> > > needs to be able to allocate memory occasionally.  It can do that when
> > > invoked from some contexts, but not when invoked from others.  Right now,
> > > in !PREEMPT kernels, it cannot tell, and must either do things to the
> > > memory allocators that some of the MM hate or must unnecessarily invoke
> > > workqueues.  Thomas's patches would allow the code to just allocate in
> > > the common case when these primitives are invoked from contexts where
> > > allocation is permitted.
> > >
> > > If we want to restrict access to the can_schedule() or whatever primitive,
> > > fine and good.  We can add a check to checkpatch.pl, for example.  Maybe
> > > we can go back to the old brlock approach of requiring certain people's
> > > review for each addition to the kernel.
> > >
> > > But there really are use cases that it would greatly help.
> >
> > We can deadlock in random fun places if random stuff we're calling
> > suddenly starts allocating. Sometimes. Maybe once in a blue moon, to
> > make it extra fun to reproduce. Maybe most driver subsystems are less
> > brittle, but gpu drivers definitely need to know about the details for
> > exactly this example. And yes gpu drivers use rcu for freeing
> > dma_fence structures, and that tends to happen in code that we only
> > recently figured out should really not allocate memory.
> >
> > I think minimally you need to throw in an unconditional
> > fs_reclaim_acquire();fs_reclaim_release(); so that everyone who runs
> > with full debugging knows what might happen. It's kinda like
> > might_sleep, but a lot more specific. might_sleep() alone is not
> > enough, because in the specific code paths I'm thinking of (and
> > created special lockdep annotations for just recently) sleeping is
> > allowed, but any memory allocations with GFP_RECLAIM set are no-go.
>
> Completely agreed!  Any allocation on any free path must be handled
> -extremely- carefully.  To that end...
>
> First, there is always a fallback in case the allocation fails.  Which
> might have performance or corner-case robustness issues, but which will
> at least allow forward progress.  Second, we consulted with a number of
> MM experts to arrive at appropriate GFP_* flags (and their patience is
> greatly appreciated).  Third, the paths that can allocate will do so about
> one time of 500, so any issues should be spotted sooner rather than later.
>
> So you are quite right to be concerned, but I believe we will be doing the
> right things.  And based on his previous track record, I am also quite
> certain that Mr. Murphy will be on hand to provide me any additional
> education that I might require.
>
> Finally, I have noted down your point about fs_reclaim_acquire() and
> fs_reclaim_release().  Whether or not they prove to be needed, I do
> appreciate your calling them to my attention.

I just realized that since these dma_fence structs are refcounted and
userspace can hold references (directly, it can pass them around
behind file descriptors) we might never hit such a path until slightly
unusual or evil userspace does something interesting. Do you have
links to those patches? Some googling didn't turn up anything. I can
then figure out whether it's better to risk not spotting issues with
call_rcu vs slapping a memalloc_noio_save/restore around all these
critical section which force-degrades any allocation to GFP_ATOMIC at
most, but has the risk that we run into code that assumes "GFP_KERNEL
never fails for small stuff" and has a decidedly less tested fallback
path than rcu code.
-Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [Intel-gfx] [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-16 21:43                     ` Daniel Vetter
  0 siblings, 0 replies; 282+ messages in thread
From: Daniel Vetter @ 2020-09-16 21:43 UTC (permalink / raw)
  To: Paul E. McKenney
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, Linux-MM,
	open list:KERNEL SELFTEST FRAMEWORK, linux-hexagon, Will Deacon,
	Ingo Molnar, Anton Ivanov, linux-arch, Herbert Xu, Brian Cain,
	Richard Weinberger, Russell King, Ard Biesheuvel, David Airlie,
	Ingo Molnar, Geert Uytterhoeven, Mel Gorman, intel-gfx,
	Matt Turner, Valentin Schneider, linux-xtensa, Shuah Khan,
	Jeff Dike, linux-um, Josh Triplett, Steven Rostedt, rcu,
	linux-m68k, Ivan Kokshaysky, Thomas Gleixner, Dietmar Eggemann,
	Linux ARM, Richard Henderson, Chris Zankel, Max Filippov,
	Daniel Bristot de Oliveira, LKML, alpha, Mathieu Desnoyers,
	Andrew Morton, Linus Torvalds

On Wed, Sep 16, 2020 at 10:58 PM Paul E. McKenney <paulmck@kernel.org> wrote:
>
> On Wed, Sep 16, 2020 at 10:29:06PM +0200, Daniel Vetter wrote:
> > On Wed, Sep 16, 2020 at 5:29 PM Paul E. McKenney <paulmck@kernel.org> wrote:
> > >
> > > On Wed, Sep 16, 2020 at 09:37:17AM +0200, Daniel Vetter wrote:
> > > > On Tue, Sep 15, 2020 at 7:35 PM Linus Torvalds
> > > > <torvalds@linux-foundation.org> wrote:
> > > > >
> > > > > On Tue, Sep 15, 2020 at 1:39 AM Thomas Gleixner <tglx@linutronix.de> wrote:
> > > > > >
> > > > > > OTOH, having a working 'preemptible()' or maybe better named
> > > > > > 'can_schedule()' check makes tons of sense to make decisions about
> > > > > > allocation modes or other things.
> > > > >
> > > > > No. I think that those kinds of decisions about actual behavior are
> > > > > always simply fundamentally wrong.
> > > > >
> > > > > Note that this is very different from having warnings about invalid
> > > > > use. THAT is correct. It may not warn in all configurations, but that
> > > > > doesn't matter: what matters is that it warns in common enough
> > > > > configurations that developers will catch it.
> > > > >
> > > > > So having a warning in "might_sleep()" that doesn't always trigger,
> > > > > because you have a limited configuration that can't even detect the
> > > > > situation, that's fine and dandy and intentional.
> > > > >
> > > > > But having code like
> > > > >
> > > > >        if (can_schedule())
> > > > >            .. do something different ..
> > > > >
> > > > > is fundamentally complete and utter garbage.
> > > > >
> > > > > It's one thing if you test for "am I in hardware interrupt context".
> > > > > Those tests aren't great either, but at least they make sense.
> > > > >
> > > > > But a driver - or some library routine - making a difference based on
> > > > > some nebulous "can I schedule" is fundamentally and basically WRONG.
> > > > >
> > > > > If some code changes behavior, it needs to be explicit to the *caller*
> > > > > of that code.
> > > > >
> > > > > So this is why GFP_ATOMIC is fine, but "if (!can_schedule())
> > > > > do_something_atomic()" is pure shite.
> > > > >
> > > > > And I am not IN THE LEAST interested in trying to help people doing
> > > > > pure shite. We need to fix them. Like the crypto code is getting
> > > > > fixed.
> > > >
> > > > Just figured I'll throw my +1 in from reading too many (gpu) drivers.
> > > > Code that tries to cleverly adjust its behaviour depending upon the
> > > > context it's running in is harder to understand and blows up in more
> > > > interesting ways. We still have drm_can_sleep() and it's mostly just
> > > > used for debug code, and I've largely ended up just deleting
> > > > everything that used it because when you're driver is blowing up the
> > > > last thing you want is to realize your debug code and output can't be
> > > > relied upon. Or worse, that the only Oops you have is the one in the
> > > > debug code, because the real one scrolled away - the original idea
> > > > behind drm_can_sleep was to make all the modeset code work
> > > > automagically both in normal ioctl/kworker context and in the panic
> > > > handlers or kgdb callbacks. Wishful thinking at best.
> > > >
> > > > Also at least for me that extends to everything, e.g. I much prefer
> > > > explicit spin_lock and spin_lock_irq vs magic spin_lock_irqsave for
> > > > locks shared with interrupt handlers, since the former two gives me
> > > > clear information from which contexts such function can be called.
> > > > Other end is the memalloc_no*_save/restore functions, where I recently
> > > > made a real big fool of myself because I didn't realize how much that
> > > > impacts everything that's run within - suddenly "GFP_KERNEL for small
> > > > stuff never fails" is wrong everywhere.
> > > >
> > > > It's all great for debugging and sanity checks (and we run with all
> > > > that stuff enabled in our CI), but really semantic changes depending
> > > > upon magic context checks freak my out :-)
> > >
> > > All fair, but some of us need to write code that must handle being
> > > invoked from a wide variety of contexts.  Now perhaps you like the idea of
> > > call_rcu() for schedulable contexts, call_rcu_nosched() when preemption
> > > is disabled, call_rcu_irqs_are_disabled() when interrupts are disabled,
> > > call_rcu_raw_atomic() from contexts where (for example) raw spinlocks
> > > are held, and so on.  However, from what I can see, most people instead
> > > consistently prefer that the RCU API instead be consolidated.
> > >
> > > Some in-flight cache-efficiency work for kvfree_rcu() and call_rcu()
> > > needs to be able to allocate memory occasionally.  It can do that when
> > > invoked from some contexts, but not when invoked from others.  Right now,
> > > in !PREEMPT kernels, it cannot tell, and must either do things to the
> > > memory allocators that some of the MM hate or must unnecessarily invoke
> > > workqueues.  Thomas's patches would allow the code to just allocate in
> > > the common case when these primitives are invoked from contexts where
> > > allocation is permitted.
> > >
> > > If we want to restrict access to the can_schedule() or whatever primitive,
> > > fine and good.  We can add a check to checkpatch.pl, for example.  Maybe
> > > we can go back to the old brlock approach of requiring certain people's
> > > review for each addition to the kernel.
> > >
> > > But there really are use cases that it would greatly help.
> >
> > We can deadlock in random fun places if random stuff we're calling
> > suddenly starts allocating. Sometimes. Maybe once in a blue moon, to
> > make it extra fun to reproduce. Maybe most driver subsystems are less
> > brittle, but gpu drivers definitely need to know about the details for
> > exactly this example. And yes gpu drivers use rcu for freeing
> > dma_fence structures, and that tends to happen in code that we only
> > recently figured out should really not allocate memory.
> >
> > I think minimally you need to throw in an unconditional
> > fs_reclaim_acquire();fs_reclaim_release(); so that everyone who runs
> > with full debugging knows what might happen. It's kinda like
> > might_sleep, but a lot more specific. might_sleep() alone is not
> > enough, because in the specific code paths I'm thinking of (and
> > created special lockdep annotations for just recently) sleeping is
> > allowed, but any memory allocations with GFP_RECLAIM set are no-go.
>
> Completely agreed!  Any allocation on any free path must be handled
> -extremely- carefully.  To that end...
>
> First, there is always a fallback in case the allocation fails.  Which
> might have performance or corner-case robustness issues, but which will
> at least allow forward progress.  Second, we consulted with a number of
> MM experts to arrive at appropriate GFP_* flags (and their patience is
> greatly appreciated).  Third, the paths that can allocate will do so about
> one time of 500, so any issues should be spotted sooner rather than later.
>
> So you are quite right to be concerned, but I believe we will be doing the
> right things.  And based on his previous track record, I am also quite
> certain that Mr. Murphy will be on hand to provide me any additional
> education that I might require.
>
> Finally, I have noted down your point about fs_reclaim_acquire() and
> fs_reclaim_release().  Whether or not they prove to be needed, I do
> appreciate your calling them to my attention.

I just realized that since these dma_fence structs are refcounted and
userspace can hold references (directly, it can pass them around
behind file descriptors) we might never hit such a path until slightly
unusual or evil userspace does something interesting. Do you have
links to those patches? Some googling didn't turn up anything. I can
then figure out whether it's better to risk not spotting issues with
call_rcu vs slapping a memalloc_noio_save/restore around all these
critical section which force-degrades any allocation to GFP_ATOMIC at
most, but has the risk that we run into code that assumes "GFP_KERNEL
never fails for small stuff" and has a decidedly less tested fallback
path than rcu code.
-Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-16 21:43                     ` Daniel Vetter
  0 siblings, 0 replies; 282+ messages in thread
From: Daniel Vetter @ 2020-09-16 21:43 UTC (permalink / raw)
  To: Paul E. McKenney
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, Linux-MM,
	open list:KERNEL SELFTEST FRAMEWORK, linux-hexagon, Will Deacon,
	Ingo Molnar, Anton Ivanov, linux-arch, Herbert Xu, Brian Cain,
	Richard Weinberger, Russell King, Ard Biesheuvel, David Airlie,
	Ingo Molnar, Geert Uytterhoeven, Mel Gorman, intel-gfx

On Wed, Sep 16, 2020 at 10:58 PM Paul E. McKenney <paulmck@kernel.org> wrote:
>
> On Wed, Sep 16, 2020 at 10:29:06PM +0200, Daniel Vetter wrote:
> > On Wed, Sep 16, 2020 at 5:29 PM Paul E. McKenney <paulmck@kernel.org> wrote:
> > >
> > > On Wed, Sep 16, 2020 at 09:37:17AM +0200, Daniel Vetter wrote:
> > > > On Tue, Sep 15, 2020 at 7:35 PM Linus Torvalds
> > > > <torvalds@linux-foundation.org> wrote:
> > > > >
> > > > > On Tue, Sep 15, 2020 at 1:39 AM Thomas Gleixner <tglx@linutronix.de> wrote:
> > > > > >
> > > > > > OTOH, having a working 'preemptible()' or maybe better named
> > > > > > 'can_schedule()' check makes tons of sense to make decisions about
> > > > > > allocation modes or other things.
> > > > >
> > > > > No. I think that those kinds of decisions about actual behavior are
> > > > > always simply fundamentally wrong.
> > > > >
> > > > > Note that this is very different from having warnings about invalid
> > > > > use. THAT is correct. It may not warn in all configurations, but that
> > > > > doesn't matter: what matters is that it warns in common enough
> > > > > configurations that developers will catch it.
> > > > >
> > > > > So having a warning in "might_sleep()" that doesn't always trigger,
> > > > > because you have a limited configuration that can't even detect the
> > > > > situation, that's fine and dandy and intentional.
> > > > >
> > > > > But having code like
> > > > >
> > > > >        if (can_schedule())
> > > > >            .. do something different ..
> > > > >
> > > > > is fundamentally complete and utter garbage.
> > > > >
> > > > > It's one thing if you test for "am I in hardware interrupt context".
> > > > > Those tests aren't great either, but at least they make sense.
> > > > >
> > > > > But a driver - or some library routine - making a difference based on
> > > > > some nebulous "can I schedule" is fundamentally and basically WRONG.
> > > > >
> > > > > If some code changes behavior, it needs to be explicit to the *caller*
> > > > > of that code.
> > > > >
> > > > > So this is why GFP_ATOMIC is fine, but "if (!can_schedule())
> > > > > do_something_atomic()" is pure shite.
> > > > >
> > > > > And I am not IN THE LEAST interested in trying to help people doing
> > > > > pure shite. We need to fix them. Like the crypto code is getting
> > > > > fixed.
> > > >
> > > > Just figured I'll throw my +1 in from reading too many (gpu) drivers.
> > > > Code that tries to cleverly adjust its behaviour depending upon the
> > > > context it's running in is harder to understand and blows up in more
> > > > interesting ways. We still have drm_can_sleep() and it's mostly just
> > > > used for debug code, and I've largely ended up just deleting
> > > > everything that used it because when you're driver is blowing up the
> > > > last thing you want is to realize your debug code and output can't be
> > > > relied upon. Or worse, that the only Oops you have is the one in the
> > > > debug code, because the real one scrolled away - the original idea
> > > > behind drm_can_sleep was to make all the modeset code work
> > > > automagically both in normal ioctl/kworker context and in the panic
> > > > handlers or kgdb callbacks. Wishful thinking at best.
> > > >
> > > > Also at least for me that extends to everything, e.g. I much prefer
> > > > explicit spin_lock and spin_lock_irq vs magic spin_lock_irqsave for
> > > > locks shared with interrupt handlers, since the former two gives me
> > > > clear information from which contexts such function can be called.
> > > > Other end is the memalloc_no*_save/restore functions, where I recently
> > > > made a real big fool of myself because I didn't realize how much that
> > > > impacts everything that's run within - suddenly "GFP_KERNEL for small
> > > > stuff never fails" is wrong everywhere.
> > > >
> > > > It's all great for debugging and sanity checks (and we run with all
> > > > that stuff enabled in our CI), but really semantic changes depending
> > > > upon magic context checks freak my out :-)
> > >
> > > All fair, but some of us need to write code that must handle being
> > > invoked from a wide variety of contexts.  Now perhaps you like the idea of
> > > call_rcu() for schedulable contexts, call_rcu_nosched() when preemption
> > > is disabled, call_rcu_irqs_are_disabled() when interrupts are disabled,
> > > call_rcu_raw_atomic() from contexts where (for example) raw spinlocks
> > > are held, and so on.  However, from what I can see, most people instead
> > > consistently prefer that the RCU API instead be consolidated.
> > >
> > > Some in-flight cache-efficiency work for kvfree_rcu() and call_rcu()
> > > needs to be able to allocate memory occasionally.  It can do that when
> > > invoked from some contexts, but not when invoked from others.  Right now,
> > > in !PREEMPT kernels, it cannot tell, and must either do things to the
> > > memory allocators that some of the MM hate or must unnecessarily invoke
> > > workqueues.  Thomas's patches would allow the code to just allocate in
> > > the common case when these primitives are invoked from contexts where
> > > allocation is permitted.
> > >
> > > If we want to restrict access to the can_schedule() or whatever primitive,
> > > fine and good.  We can add a check to checkpatch.pl, for example.  Maybe
> > > we can go back to the old brlock approach of requiring certain people's
> > > review for each addition to the kernel.
> > >
> > > But there really are use cases that it would greatly help.
> >
> > We can deadlock in random fun places if random stuff we're calling
> > suddenly starts allocating. Sometimes. Maybe once in a blue moon, to
> > make it extra fun to reproduce. Maybe most driver subsystems are less
> > brittle, but gpu drivers definitely need to know about the details for
> > exactly this example. And yes gpu drivers use rcu for freeing
> > dma_fence structures, and that tends to happen in code that we only
> > recently figured out should really not allocate memory.
> >
> > I think minimally you need to throw in an unconditional
> > fs_reclaim_acquire();fs_reclaim_release(); so that everyone who runs
> > with full debugging knows what might happen. It's kinda like
> > might_sleep, but a lot more specific. might_sleep() alone is not
> > enough, because in the specific code paths I'm thinking of (and
> > created special lockdep annotations for just recently) sleeping is
> > allowed, but any memory allocations with GFP_RECLAIM set are no-go.
>
> Completely agreed!  Any allocation on any free path must be handled
> -extremely- carefully.  To that end...
>
> First, there is always a fallback in case the allocation fails.  Which
> might have performance or corner-case robustness issues, but which will
> at least allow forward progress.  Second, we consulted with a number of
> MM experts to arrive at appropriate GFP_* flags (and their patience is
> greatly appreciated).  Third, the paths that can allocate will do so about
> one time of 500, so any issues should be spotted sooner rather than later.
>
> So you are quite right to be concerned, but I believe we will be doing the
> right things.  And based on his previous track record, I am also quite
> certain that Mr. Murphy will be on hand to provide me any additional
> education that I might require.
>
> Finally, I have noted down your point about fs_reclaim_acquire() and
> fs_reclaim_release().  Whether or not they prove to be needed, I do
> appreciate your calling them to my attention.

I just realized that since these dma_fence structs are refcounted and
userspace can hold references (directly, it can pass them around
behind file descriptors) we might never hit such a path until slightly
unusual or evil userspace does something interesting. Do you have
links to those patches? Some googling didn't turn up anything. I can
then figure out whether it's better to risk not spotting issues with
call_rcu vs slapping a memalloc_noio_save/restore around all these
critical section which force-degrades any allocation to GFP_ATOMIC at
most, but has the risk that we run into code that assumes "GFP_KERNEL
never fails for small stuff" and has a decidedly less tested fallback
path than rcu code.
-Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
  2020-09-16 21:43                     ` Daniel Vetter
                                         ` (2 preceding siblings ...)
  (?)
@ 2020-09-16 22:39                       ` Paul E. McKenney
  -1 siblings, 0 replies; 282+ messages in thread
From: Paul E. McKenney @ 2020-09-16 22:39 UTC (permalink / raw)
  To: Daniel Vetter
  Cc: Linus Torvalds, Thomas Gleixner, Ard Biesheuvel, Herbert Xu,
	LKML, linux-arch, Sebastian Andrzej Siewior, Valentin Schneider,
	Richard Henderson, Ivan Kokshaysky, Matt Turner, alpha,
	Jeff Dike, Richard Weinberger, Anton Ivanov, linux-um,
	Brian Cain, linux-hexagon, Geert Uytterhoeven, linux-m68k,
	Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
	Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman,
	Daniel Bristot de Oliveira, Will Deacon, Andrew Morton, Linux-MM,
	Ingo Molnar, Russell King, Linux ARM, Chris Zankel, Max Filippov,
	linux-xtensa, Jani Nikula, Joonas Lahtinen, Rodrigo Vivi,
	David Airlie, intel-gfx, dri-devel, Josh Triplett,
	Mathieu Desnoyers, Lai Jiangshan, Shuah Khan, rcu,
	open list:KERNEL SELFTEST FRAMEWORK

On Wed, Sep 16, 2020 at 11:43:02PM +0200, Daniel Vetter wrote:
> On Wed, Sep 16, 2020 at 10:58 PM Paul E. McKenney <paulmck@kernel.org> wrote:
> >
> > On Wed, Sep 16, 2020 at 10:29:06PM +0200, Daniel Vetter wrote:
> > > On Wed, Sep 16, 2020 at 5:29 PM Paul E. McKenney <paulmck@kernel.org> wrote:
> > > >
> > > > On Wed, Sep 16, 2020 at 09:37:17AM +0200, Daniel Vetter wrote:
> > > > > On Tue, Sep 15, 2020 at 7:35 PM Linus Torvalds
> > > > > <torvalds@linux-foundation.org> wrote:
> > > > > >
> > > > > > On Tue, Sep 15, 2020 at 1:39 AM Thomas Gleixner <tglx@linutronix.de> wrote:
> > > > > > >
> > > > > > > OTOH, having a working 'preemptible()' or maybe better named
> > > > > > > 'can_schedule()' check makes tons of sense to make decisions about
> > > > > > > allocation modes or other things.
> > > > > >
> > > > > > No. I think that those kinds of decisions about actual behavior are
> > > > > > always simply fundamentally wrong.
> > > > > >
> > > > > > Note that this is very different from having warnings about invalid
> > > > > > use. THAT is correct. It may not warn in all configurations, but that
> > > > > > doesn't matter: what matters is that it warns in common enough
> > > > > > configurations that developers will catch it.
> > > > > >
> > > > > > So having a warning in "might_sleep()" that doesn't always trigger,
> > > > > > because you have a limited configuration that can't even detect the
> > > > > > situation, that's fine and dandy and intentional.
> > > > > >
> > > > > > But having code like
> > > > > >
> > > > > >        if (can_schedule())
> > > > > >            .. do something different ..
> > > > > >
> > > > > > is fundamentally complete and utter garbage.
> > > > > >
> > > > > > It's one thing if you test for "am I in hardware interrupt context".
> > > > > > Those tests aren't great either, but at least they make sense.
> > > > > >
> > > > > > But a driver - or some library routine - making a difference based on
> > > > > > some nebulous "can I schedule" is fundamentally and basically WRONG.
> > > > > >
> > > > > > If some code changes behavior, it needs to be explicit to the *caller*
> > > > > > of that code.
> > > > > >
> > > > > > So this is why GFP_ATOMIC is fine, but "if (!can_schedule())
> > > > > > do_something_atomic()" is pure shite.
> > > > > >
> > > > > > And I am not IN THE LEAST interested in trying to help people doing
> > > > > > pure shite. We need to fix them. Like the crypto code is getting
> > > > > > fixed.
> > > > >
> > > > > Just figured I'll throw my +1 in from reading too many (gpu) drivers.
> > > > > Code that tries to cleverly adjust its behaviour depending upon the
> > > > > context it's running in is harder to understand and blows up in more
> > > > > interesting ways. We still have drm_can_sleep() and it's mostly just
> > > > > used for debug code, and I've largely ended up just deleting
> > > > > everything that used it because when you're driver is blowing up the
> > > > > last thing you want is to realize your debug code and output can't be
> > > > > relied upon. Or worse, that the only Oops you have is the one in the
> > > > > debug code, because the real one scrolled away - the original idea
> > > > > behind drm_can_sleep was to make all the modeset code work
> > > > > automagically both in normal ioctl/kworker context and in the panic
> > > > > handlers or kgdb callbacks. Wishful thinking at best.
> > > > >
> > > > > Also at least for me that extends to everything, e.g. I much prefer
> > > > > explicit spin_lock and spin_lock_irq vs magic spin_lock_irqsave for
> > > > > locks shared with interrupt handlers, since the former two gives me
> > > > > clear information from which contexts such function can be called.
> > > > > Other end is the memalloc_no*_save/restore functions, where I recently
> > > > > made a real big fool of myself because I didn't realize how much that
> > > > > impacts everything that's run within - suddenly "GFP_KERNEL for small
> > > > > stuff never fails" is wrong everywhere.
> > > > >
> > > > > It's all great for debugging and sanity checks (and we run with all
> > > > > that stuff enabled in our CI), but really semantic changes depending
> > > > > upon magic context checks freak my out :-)
> > > >
> > > > All fair, but some of us need to write code that must handle being
> > > > invoked from a wide variety of contexts.  Now perhaps you like the idea of
> > > > call_rcu() for schedulable contexts, call_rcu_nosched() when preemption
> > > > is disabled, call_rcu_irqs_are_disabled() when interrupts are disabled,
> > > > call_rcu_raw_atomic() from contexts where (for example) raw spinlocks
> > > > are held, and so on.  However, from what I can see, most people instead
> > > > consistently prefer that the RCU API instead be consolidated.
> > > >
> > > > Some in-flight cache-efficiency work for kvfree_rcu() and call_rcu()
> > > > needs to be able to allocate memory occasionally.  It can do that when
> > > > invoked from some contexts, but not when invoked from others.  Right now,
> > > > in !PREEMPT kernels, it cannot tell, and must either do things to the
> > > > memory allocators that some of the MM hate or must unnecessarily invoke
> > > > workqueues.  Thomas's patches would allow the code to just allocate in
> > > > the common case when these primitives are invoked from contexts where
> > > > allocation is permitted.
> > > >
> > > > If we want to restrict access to the can_schedule() or whatever primitive,
> > > > fine and good.  We can add a check to checkpatch.pl, for example.  Maybe
> > > > we can go back to the old brlock approach of requiring certain people's
> > > > review for each addition to the kernel.
> > > >
> > > > But there really are use cases that it would greatly help.
> > >
> > > We can deadlock in random fun places if random stuff we're calling
> > > suddenly starts allocating. Sometimes. Maybe once in a blue moon, to
> > > make it extra fun to reproduce. Maybe most driver subsystems are less
> > > brittle, but gpu drivers definitely need to know about the details for
> > > exactly this example. And yes gpu drivers use rcu for freeing
> > > dma_fence structures, and that tends to happen in code that we only
> > > recently figured out should really not allocate memory.
> > >
> > > I think minimally you need to throw in an unconditional
> > > fs_reclaim_acquire();fs_reclaim_release(); so that everyone who runs
> > > with full debugging knows what might happen. It's kinda like
> > > might_sleep, but a lot more specific. might_sleep() alone is not
> > > enough, because in the specific code paths I'm thinking of (and
> > > created special lockdep annotations for just recently) sleeping is
> > > allowed, but any memory allocations with GFP_RECLAIM set are no-go.
> >
> > Completely agreed!  Any allocation on any free path must be handled
> > -extremely- carefully.  To that end...
> >
> > First, there is always a fallback in case the allocation fails.  Which
> > might have performance or corner-case robustness issues, but which will
> > at least allow forward progress.  Second, we consulted with a number of
> > MM experts to arrive at appropriate GFP_* flags (and their patience is
> > greatly appreciated).  Third, the paths that can allocate will do so about
> > one time of 500, so any issues should be spotted sooner rather than later.
> >
> > So you are quite right to be concerned, but I believe we will be doing the
> > right things.  And based on his previous track record, I am also quite
> > certain that Mr. Murphy will be on hand to provide me any additional
> > education that I might require.
> >
> > Finally, I have noted down your point about fs_reclaim_acquire() and
> > fs_reclaim_release().  Whether or not they prove to be needed, I do
> > appreciate your calling them to my attention.
> 
> I just realized that since these dma_fence structs are refcounted and
> userspace can hold references (directly, it can pass them around
> behind file descriptors) we might never hit such a path until slightly
> unusual or evil userspace does something interesting. Do you have
> links to those patches? Some googling didn't turn up anything. I can
> then figure out whether it's better to risk not spotting issues with
> call_rcu vs slapping a memalloc_noio_save/restore around all these
> critical section which force-degrades any allocation to GFP_ATOMIC at
> most, but has the risk that we run into code that assumes "GFP_KERNEL
> never fails for small stuff" and has a decidedly less tested fallback
> path than rcu code.

Here is the previous early draft version, which will change considerably
for the next version:

	lore.kernel.org/lkml/20200809204354.20137-1-urezki@gmail.com

This does kvfree_rcu(), but we expect to handle call_rcu() similarly.

The version in preparation will use workqueues to do the allocation in a
known-safe environment and also use lockless access to certain portions
of the allocator caches (as noted earlier, this last is not much loved
by some of the MM guys).  Given Thomas's patch, we could with high
probability allocate directly, perhaps even not needing memory-allocator
modifications.

Either way, kvfree_rcu(), and later call_rcu(), will avoid asking the
allocator to do anything that the calling context prohibits.  So what
types of bugs are you looking for?  Where reclaim calls back into the
driver or some such?

							Thanx, Paul

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-16 22:39                       ` Paul E. McKenney
  0 siblings, 0 replies; 282+ messages in thread
From: Paul E. McKenney @ 2020-09-16 22:39 UTC (permalink / raw)
  To: Daniel Vetter
  Cc: Linus Torvalds, Thomas Gleixner, Ard Biesheuvel, Herbert Xu,
	LKML, linux-arch, Sebastian Andrzej Siewior, Valentin Schneider,
	Richard Henderson, Ivan Kokshaysky, Matt Turner, alpha,
	Jeff Dike, Richard Weinberger, Anton Ivanov, linux-um,
	Brian Cain, linux-hexagon, Geert Uytterhoeven, linux-m68k,
	Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
	Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman,
	Daniel Bristot de Oliveira, Will Deacon, Andrew Morton, Linux-MM,
	Ingo Molnar, Russell King, Linux ARM, Chris Zankel, Max Filippov,
	linux-xtensa, Jani Nikula, Joonas Lahtinen, Rodrigo Vivi,
	David Airlie, intel-gfx, dri-devel, Josh Triplett,
	Mathieu Desnoyers, Lai Jiangshan, Shuah Khan, rcu,
	open list:KERNEL SELFTEST FRAMEWORK

On Wed, Sep 16, 2020 at 11:43:02PM +0200, Daniel Vetter wrote:
> On Wed, Sep 16, 2020 at 10:58 PM Paul E. McKenney <paulmck@kernel.org> wrote:
> >
> > On Wed, Sep 16, 2020 at 10:29:06PM +0200, Daniel Vetter wrote:
> > > On Wed, Sep 16, 2020 at 5:29 PM Paul E. McKenney <paulmck@kernel.org> wrote:
> > > >
> > > > On Wed, Sep 16, 2020 at 09:37:17AM +0200, Daniel Vetter wrote:
> > > > > On Tue, Sep 15, 2020 at 7:35 PM Linus Torvalds
> > > > > <torvalds@linux-foundation.org> wrote:
> > > > > >
> > > > > > On Tue, Sep 15, 2020 at 1:39 AM Thomas Gleixner <tglx@linutronix.de> wrote:
> > > > > > >
> > > > > > > OTOH, having a working 'preemptible()' or maybe better named
> > > > > > > 'can_schedule()' check makes tons of sense to make decisions about
> > > > > > > allocation modes or other things.
> > > > > >
> > > > > > No. I think that those kinds of decisions about actual behavior are
> > > > > > always simply fundamentally wrong.
> > > > > >
> > > > > > Note that this is very different from having warnings about invalid
> > > > > > use. THAT is correct. It may not warn in all configurations, but that
> > > > > > doesn't matter: what matters is that it warns in common enough
> > > > > > configurations that developers will catch it.
> > > > > >
> > > > > > So having a warning in "might_sleep()" that doesn't always trigger,
> > > > > > because you have a limited configuration that can't even detect the
> > > > > > situation, that's fine and dandy and intentional.
> > > > > >
> > > > > > But having code like
> > > > > >
> > > > > >        if (can_schedule())
> > > > > >            .. do something different ..
> > > > > >
> > > > > > is fundamentally complete and utter garbage.
> > > > > >
> > > > > > It's one thing if you test for "am I in hardware interrupt context".
> > > > > > Those tests aren't great either, but at least they make sense.
> > > > > >
> > > > > > But a driver - or some library routine - making a difference based on
> > > > > > some nebulous "can I schedule" is fundamentally and basically WRONG.
> > > > > >
> > > > > > If some code changes behavior, it needs to be explicit to the *caller*
> > > > > > of that code.
> > > > > >
> > > > > > So this is why GFP_ATOMIC is fine, but "if (!can_schedule())
> > > > > > do_something_atomic()" is pure shite.
> > > > > >
> > > > > > And I am not IN THE LEAST interested in trying to help people doing
> > > > > > pure shite. We need to fix them. Like the crypto code is getting
> > > > > > fixed.
> > > > >
> > > > > Just figured I'll throw my +1 in from reading too many (gpu) drivers.
> > > > > Code that tries to cleverly adjust its behaviour depending upon the
> > > > > context it's running in is harder to understand and blows up in more
> > > > > interesting ways. We still have drm_can_sleep() and it's mostly just
> > > > > used for debug code, and I've largely ended up just deleting
> > > > > everything that used it because when you're driver is blowing up the
> > > > > last thing you want is to realize your debug code and output can't be
> > > > > relied upon. Or worse, that the only Oops you have is the one in the
> > > > > debug code, because the real one scrolled away - the original idea
> > > > > behind drm_can_sleep was to make all the modeset code work
> > > > > automagically both in normal ioctl/kworker context and in the panic
> > > > > handlers or kgdb callbacks. Wishful thinking at best.
> > > > >
> > > > > Also at least for me that extends to everything, e.g. I much prefer
> > > > > explicit spin_lock and spin_lock_irq vs magic spin_lock_irqsave for
> > > > > locks shared with interrupt handlers, since the former two gives me
> > > > > clear information from which contexts such function can be called.
> > > > > Other end is the memalloc_no*_save/restore functions, where I recently
> > > > > made a real big fool of myself because I didn't realize how much that
> > > > > impacts everything that's run within - suddenly "GFP_KERNEL for small
> > > > > stuff never fails" is wrong everywhere.
> > > > >
> > > > > It's all great for debugging and sanity checks (and we run with all
> > > > > that stuff enabled in our CI), but really semantic changes depending
> > > > > upon magic context checks freak my out :-)
> > > >
> > > > All fair, but some of us need to write code that must handle being
> > > > invoked from a wide variety of contexts.  Now perhaps you like the idea of
> > > > call_rcu() for schedulable contexts, call_rcu_nosched() when preemption
> > > > is disabled, call_rcu_irqs_are_disabled() when interrupts are disabled,
> > > > call_rcu_raw_atomic() from contexts where (for example) raw spinlocks
> > > > are held, and so on.  However, from what I can see, most people instead
> > > > consistently prefer that the RCU API instead be consolidated.
> > > >
> > > > Some in-flight cache-efficiency work for kvfree_rcu() and call_rcu()
> > > > needs to be able to allocate memory occasionally.  It can do that when
> > > > invoked from some contexts, but not when invoked from others.  Right now,
> > > > in !PREEMPT kernels, it cannot tell, and must either do things to the
> > > > memory allocators that some of the MM hate or must unnecessarily invoke
> > > > workqueues.  Thomas's patches would allow the code to just allocate in
> > > > the common case when these primitives are invoked from contexts where
> > > > allocation is permitted.
> > > >
> > > > If we want to restrict access to the can_schedule() or whatever primitive,
> > > > fine and good.  We can add a check to checkpatch.pl, for example.  Maybe
> > > > we can go back to the old brlock approach of requiring certain people's
> > > > review for each addition to the kernel.
> > > >
> > > > But there really are use cases that it would greatly help.
> > >
> > > We can deadlock in random fun places if random stuff we're calling
> > > suddenly starts allocating. Sometimes. Maybe once in a blue moon, to
> > > make it extra fun to reproduce. Maybe most driver subsystems are less
> > > brittle, but gpu drivers definitely need to know about the details for
> > > exactly this example. And yes gpu drivers use rcu for freeing
> > > dma_fence structures, and that tends to happen in code that we only
> > > recently figured out should really not allocate memory.
> > >
> > > I think minimally you need to throw in an unconditional
> > > fs_reclaim_acquire();fs_reclaim_release(); so that everyone who runs
> > > with full debugging knows what might happen. It's kinda like
> > > might_sleep, but a lot more specific. might_sleep() alone is not
> > > enough, because in the specific code paths I'm thinking of (and
> > > created special lockdep annotations for just recently) sleeping is
> > > allowed, but any memory allocations with GFP_RECLAIM set are no-go.
> >
> > Completely agreed!  Any allocation on any free path must be handled
> > -extremely- carefully.  To that end...
> >
> > First, there is always a fallback in case the allocation fails.  Which
> > might have performance or corner-case robustness issues, but which will
> > at least allow forward progress.  Second, we consulted with a number of
> > MM experts to arrive at appropriate GFP_* flags (and their patience is
> > greatly appreciated).  Third, the paths that can allocate will do so about
> > one time of 500, so any issues should be spotted sooner rather than later.
> >
> > So you are quite right to be concerned, but I believe we will be doing the
> > right things.  And based on his previous track record, I am also quite
> > certain that Mr. Murphy will be on hand to provide me any additional
> > education that I might require.
> >
> > Finally, I have noted down your point about fs_reclaim_acquire() and
> > fs_reclaim_release().  Whether or not they prove to be needed, I do
> > appreciate your calling them to my attention.
> 
> I just realized that since these dma_fence structs are refcounted and
> userspace can hold references (directly, it can pass them around
> behind file descriptors) we might never hit such a path until slightly
> unusual or evil userspace does something interesting. Do you have
> links to those patches? Some googling didn't turn up anything. I can
> then figure out whether it's better to risk not spotting issues with
> call_rcu vs slapping a memalloc_noio_save/restore around all these
> critical section which force-degrades any allocation to GFP_ATOMIC at
> most, but has the risk that we run into code that assumes "GFP_KERNEL
> never fails for small stuff" and has a decidedly less tested fallback
> path than rcu code.

Here is the previous early draft version, which will change considerably
for the next version:

	lore.kernel.org/lkml/20200809204354.20137-1-urezki@gmail.com

This does kvfree_rcu(), but we expect to handle call_rcu() similarly.

The version in preparation will use workqueues to do the allocation in a
known-safe environment and also use lockless access to certain portions
of the allocator caches (as noted earlier, this last is not much loved
by some of the MM guys).  Given Thomas's patch, we could with high
probability allocate directly, perhaps even not needing memory-allocator
modifications.

Either way, kvfree_rcu(), and later call_rcu(), will avoid asking the
allocator to do anything that the calling context prohibits.  So what
types of bugs are you looking for?  Where reclaim calls back into the
driver or some such?

							Thanx, Paul


^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-16 22:39                       ` Paul E. McKenney
  0 siblings, 0 replies; 282+ messages in thread
From: Paul E. McKenney @ 2020-09-16 22:39 UTC (permalink / raw)
  To: Daniel Vetter
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, Linux-MM,
	open list:KERNEL SELFTEST FRAMEWORK, linux-hexagon, Will Deacon,
	Ingo Molnar, Anton Ivanov, linux-arch, Vincent Guittot,
	Herbert Xu, Brian Cain, Richard Weinberger, Russell King,
	Ard Biesheuvel, David Airlie, Ingo Molnar, Geert Uytterhoeven,
	Mel Gorman, intel-gfx, Matt Turner, Valentin Schneider,
	linux-xtensa, Shuah Khan, Jeff Dike, linux-um, Josh Triplett,
	Steven Rostedt, rcu, linux-m68k, Ivan Kokshaysky, Rodrigo Vivi,
	Thomas Gleixner, Dietmar Eggemann, Linux ARM, Richard Henderson,
	Chris Zankel, Max Filippov, Daniel Bristot de Oliveira, LKML,
	alpha, Mathieu Desnoyers, Andrew Morton, Linus Torvalds

On Wed, Sep 16, 2020 at 11:43:02PM +0200, Daniel Vetter wrote:
> On Wed, Sep 16, 2020 at 10:58 PM Paul E. McKenney <paulmck@kernel.org> wrote:
> >
> > On Wed, Sep 16, 2020 at 10:29:06PM +0200, Daniel Vetter wrote:
> > > On Wed, Sep 16, 2020 at 5:29 PM Paul E. McKenney <paulmck@kernel.org> wrote:
> > > >
> > > > On Wed, Sep 16, 2020 at 09:37:17AM +0200, Daniel Vetter wrote:
> > > > > On Tue, Sep 15, 2020 at 7:35 PM Linus Torvalds
> > > > > <torvalds@linux-foundation.org> wrote:
> > > > > >
> > > > > > On Tue, Sep 15, 2020 at 1:39 AM Thomas Gleixner <tglx@linutronix.de> wrote:
> > > > > > >
> > > > > > > OTOH, having a working 'preemptible()' or maybe better named
> > > > > > > 'can_schedule()' check makes tons of sense to make decisions about
> > > > > > > allocation modes or other things.
> > > > > >
> > > > > > No. I think that those kinds of decisions about actual behavior are
> > > > > > always simply fundamentally wrong.
> > > > > >
> > > > > > Note that this is very different from having warnings about invalid
> > > > > > use. THAT is correct. It may not warn in all configurations, but that
> > > > > > doesn't matter: what matters is that it warns in common enough
> > > > > > configurations that developers will catch it.
> > > > > >
> > > > > > So having a warning in "might_sleep()" that doesn't always trigger,
> > > > > > because you have a limited configuration that can't even detect the
> > > > > > situation, that's fine and dandy and intentional.
> > > > > >
> > > > > > But having code like
> > > > > >
> > > > > >        if (can_schedule())
> > > > > >            .. do something different ..
> > > > > >
> > > > > > is fundamentally complete and utter garbage.
> > > > > >
> > > > > > It's one thing if you test for "am I in hardware interrupt context".
> > > > > > Those tests aren't great either, but at least they make sense.
> > > > > >
> > > > > > But a driver - or some library routine - making a difference based on
> > > > > > some nebulous "can I schedule" is fundamentally and basically WRONG.
> > > > > >
> > > > > > If some code changes behavior, it needs to be explicit to the *caller*
> > > > > > of that code.
> > > > > >
> > > > > > So this is why GFP_ATOMIC is fine, but "if (!can_schedule())
> > > > > > do_something_atomic()" is pure shite.
> > > > > >
> > > > > > And I am not IN THE LEAST interested in trying to help people doing
> > > > > > pure shite. We need to fix them. Like the crypto code is getting
> > > > > > fixed.
> > > > >
> > > > > Just figured I'll throw my +1 in from reading too many (gpu) drivers.
> > > > > Code that tries to cleverly adjust its behaviour depending upon the
> > > > > context it's running in is harder to understand and blows up in more
> > > > > interesting ways. We still have drm_can_sleep() and it's mostly just
> > > > > used for debug code, and I've largely ended up just deleting
> > > > > everything that used it because when you're driver is blowing up the
> > > > > last thing you want is to realize your debug code and output can't be
> > > > > relied upon. Or worse, that the only Oops you have is the one in the
> > > > > debug code, because the real one scrolled away - the original idea
> > > > > behind drm_can_sleep was to make all the modeset code work
> > > > > automagically both in normal ioctl/kworker context and in the panic
> > > > > handlers or kgdb callbacks. Wishful thinking at best.
> > > > >
> > > > > Also at least for me that extends to everything, e.g. I much prefer
> > > > > explicit spin_lock and spin_lock_irq vs magic spin_lock_irqsave for
> > > > > locks shared with interrupt handlers, since the former two gives me
> > > > > clear information from which contexts such function can be called.
> > > > > Other end is the memalloc_no*_save/restore functions, where I recently
> > > > > made a real big fool of myself because I didn't realize how much that
> > > > > impacts everything that's run within - suddenly "GFP_KERNEL for small
> > > > > stuff never fails" is wrong everywhere.
> > > > >
> > > > > It's all great for debugging and sanity checks (and we run with all
> > > > > that stuff enabled in our CI), but really semantic changes depending
> > > > > upon magic context checks freak my out :-)
> > > >
> > > > All fair, but some of us need to write code that must handle being
> > > > invoked from a wide variety of contexts.  Now perhaps you like the idea of
> > > > call_rcu() for schedulable contexts, call_rcu_nosched() when preemption
> > > > is disabled, call_rcu_irqs_are_disabled() when interrupts are disabled,
> > > > call_rcu_raw_atomic() from contexts where (for example) raw spinlocks
> > > > are held, and so on.  However, from what I can see, most people instead
> > > > consistently prefer that the RCU API instead be consolidated.
> > > >
> > > > Some in-flight cache-efficiency work for kvfree_rcu() and call_rcu()
> > > > needs to be able to allocate memory occasionally.  It can do that when
> > > > invoked from some contexts, but not when invoked from others.  Right now,
> > > > in !PREEMPT kernels, it cannot tell, and must either do things to the
> > > > memory allocators that some of the MM hate or must unnecessarily invoke
> > > > workqueues.  Thomas's patches would allow the code to just allocate in
> > > > the common case when these primitives are invoked from contexts where
> > > > allocation is permitted.
> > > >
> > > > If we want to restrict access to the can_schedule() or whatever primitive,
> > > > fine and good.  We can add a check to checkpatch.pl, for example.  Maybe
> > > > we can go back to the old brlock approach of requiring certain people's
> > > > review for each addition to the kernel.
> > > >
> > > > But there really are use cases that it would greatly help.
> > >
> > > We can deadlock in random fun places if random stuff we're calling
> > > suddenly starts allocating. Sometimes. Maybe once in a blue moon, to
> > > make it extra fun to reproduce. Maybe most driver subsystems are less
> > > brittle, but gpu drivers definitely need to know about the details for
> > > exactly this example. And yes gpu drivers use rcu for freeing
> > > dma_fence structures, and that tends to happen in code that we only
> > > recently figured out should really not allocate memory.
> > >
> > > I think minimally you need to throw in an unconditional
> > > fs_reclaim_acquire();fs_reclaim_release(); so that everyone who runs
> > > with full debugging knows what might happen. It's kinda like
> > > might_sleep, but a lot more specific. might_sleep() alone is not
> > > enough, because in the specific code paths I'm thinking of (and
> > > created special lockdep annotations for just recently) sleeping is
> > > allowed, but any memory allocations with GFP_RECLAIM set are no-go.
> >
> > Completely agreed!  Any allocation on any free path must be handled
> > -extremely- carefully.  To that end...
> >
> > First, there is always a fallback in case the allocation fails.  Which
> > might have performance or corner-case robustness issues, but which will
> > at least allow forward progress.  Second, we consulted with a number of
> > MM experts to arrive at appropriate GFP_* flags (and their patience is
> > greatly appreciated).  Third, the paths that can allocate will do so about
> > one time of 500, so any issues should be spotted sooner rather than later.
> >
> > So you are quite right to be concerned, but I believe we will be doing the
> > right things.  And based on his previous track record, I am also quite
> > certain that Mr. Murphy will be on hand to provide me any additional
> > education that I might require.
> >
> > Finally, I have noted down your point about fs_reclaim_acquire() and
> > fs_reclaim_release().  Whether or not they prove to be needed, I do
> > appreciate your calling them to my attention.
> 
> I just realized that since these dma_fence structs are refcounted and
> userspace can hold references (directly, it can pass them around
> behind file descriptors) we might never hit such a path until slightly
> unusual or evil userspace does something interesting. Do you have
> links to those patches? Some googling didn't turn up anything. I can
> then figure out whether it's better to risk not spotting issues with
> call_rcu vs slapping a memalloc_noio_save/restore around all these
> critical section which force-degrades any allocation to GFP_ATOMIC at
> most, but has the risk that we run into code that assumes "GFP_KERNEL
> never fails for small stuff" and has a decidedly less tested fallback
> path than rcu code.

Here is the previous early draft version, which will change considerably
for the next version:

	lore.kernel.org/lkml/20200809204354.20137-1-urezki@gmail.com

This does kvfree_rcu(), but we expect to handle call_rcu() similarly.

The version in preparation will use workqueues to do the allocation in a
known-safe environment and also use lockless access to certain portions
of the allocator caches (as noted earlier, this last is not much loved
by some of the MM guys).  Given Thomas's patch, we could with high
probability allocate directly, perhaps even not needing memory-allocator
modifications.

Either way, kvfree_rcu(), and later call_rcu(), will avoid asking the
allocator to do anything that the calling context prohibits.  So what
types of bugs are you looking for?  Where reclaim calls back into the
driver or some such?

							Thanx, Paul
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [Intel-gfx] [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-16 22:39                       ` Paul E. McKenney
  0 siblings, 0 replies; 282+ messages in thread
From: Paul E. McKenney @ 2020-09-16 22:39 UTC (permalink / raw)
  To: Daniel Vetter
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, Linux-MM,
	open list:KERNEL SELFTEST FRAMEWORK, linux-hexagon, Will Deacon,
	Ingo Molnar, Anton Ivanov, linux-arch, Herbert Xu, Brian Cain,
	Richard Weinberger, Russell King, Ard Biesheuvel, David Airlie,
	Ingo Molnar, Geert Uytterhoeven, Mel Gorman, intel-gfx,
	Matt Turner, Valentin Schneider, linux-xtensa, Shuah Khan,
	Jeff Dike, linux-um, Josh Triplett, Steven Rostedt, rcu,
	linux-m68k, Ivan Kokshaysky, Thomas Gleixner, Dietmar Eggemann,
	Linux ARM, Richard Henderson, Chris Zankel, Max Filippov,
	Daniel Bristot de Oliveira, LKML, alpha, Mathieu Desnoyers,
	Andrew Morton, Linus Torvalds

On Wed, Sep 16, 2020 at 11:43:02PM +0200, Daniel Vetter wrote:
> On Wed, Sep 16, 2020 at 10:58 PM Paul E. McKenney <paulmck@kernel.org> wrote:
> >
> > On Wed, Sep 16, 2020 at 10:29:06PM +0200, Daniel Vetter wrote:
> > > On Wed, Sep 16, 2020 at 5:29 PM Paul E. McKenney <paulmck@kernel.org> wrote:
> > > >
> > > > On Wed, Sep 16, 2020 at 09:37:17AM +0200, Daniel Vetter wrote:
> > > > > On Tue, Sep 15, 2020 at 7:35 PM Linus Torvalds
> > > > > <torvalds@linux-foundation.org> wrote:
> > > > > >
> > > > > > On Tue, Sep 15, 2020 at 1:39 AM Thomas Gleixner <tglx@linutronix.de> wrote:
> > > > > > >
> > > > > > > OTOH, having a working 'preemptible()' or maybe better named
> > > > > > > 'can_schedule()' check makes tons of sense to make decisions about
> > > > > > > allocation modes or other things.
> > > > > >
> > > > > > No. I think that those kinds of decisions about actual behavior are
> > > > > > always simply fundamentally wrong.
> > > > > >
> > > > > > Note that this is very different from having warnings about invalid
> > > > > > use. THAT is correct. It may not warn in all configurations, but that
> > > > > > doesn't matter: what matters is that it warns in common enough
> > > > > > configurations that developers will catch it.
> > > > > >
> > > > > > So having a warning in "might_sleep()" that doesn't always trigger,
> > > > > > because you have a limited configuration that can't even detect the
> > > > > > situation, that's fine and dandy and intentional.
> > > > > >
> > > > > > But having code like
> > > > > >
> > > > > >        if (can_schedule())
> > > > > >            .. do something different ..
> > > > > >
> > > > > > is fundamentally complete and utter garbage.
> > > > > >
> > > > > > It's one thing if you test for "am I in hardware interrupt context".
> > > > > > Those tests aren't great either, but at least they make sense.
> > > > > >
> > > > > > But a driver - or some library routine - making a difference based on
> > > > > > some nebulous "can I schedule" is fundamentally and basically WRONG.
> > > > > >
> > > > > > If some code changes behavior, it needs to be explicit to the *caller*
> > > > > > of that code.
> > > > > >
> > > > > > So this is why GFP_ATOMIC is fine, but "if (!can_schedule())
> > > > > > do_something_atomic()" is pure shite.
> > > > > >
> > > > > > And I am not IN THE LEAST interested in trying to help people doing
> > > > > > pure shite. We need to fix them. Like the crypto code is getting
> > > > > > fixed.
> > > > >
> > > > > Just figured I'll throw my +1 in from reading too many (gpu) drivers.
> > > > > Code that tries to cleverly adjust its behaviour depending upon the
> > > > > context it's running in is harder to understand and blows up in more
> > > > > interesting ways. We still have drm_can_sleep() and it's mostly just
> > > > > used for debug code, and I've largely ended up just deleting
> > > > > everything that used it because when you're driver is blowing up the
> > > > > last thing you want is to realize your debug code and output can't be
> > > > > relied upon. Or worse, that the only Oops you have is the one in the
> > > > > debug code, because the real one scrolled away - the original idea
> > > > > behind drm_can_sleep was to make all the modeset code work
> > > > > automagically both in normal ioctl/kworker context and in the panic
> > > > > handlers or kgdb callbacks. Wishful thinking at best.
> > > > >
> > > > > Also at least for me that extends to everything, e.g. I much prefer
> > > > > explicit spin_lock and spin_lock_irq vs magic spin_lock_irqsave for
> > > > > locks shared with interrupt handlers, since the former two gives me
> > > > > clear information from which contexts such function can be called.
> > > > > Other end is the memalloc_no*_save/restore functions, where I recently
> > > > > made a real big fool of myself because I didn't realize how much that
> > > > > impacts everything that's run within - suddenly "GFP_KERNEL for small
> > > > > stuff never fails" is wrong everywhere.
> > > > >
> > > > > It's all great for debugging and sanity checks (and we run with all
> > > > > that stuff enabled in our CI), but really semantic changes depending
> > > > > upon magic context checks freak my out :-)
> > > >
> > > > All fair, but some of us need to write code that must handle being
> > > > invoked from a wide variety of contexts.  Now perhaps you like the idea of
> > > > call_rcu() for schedulable contexts, call_rcu_nosched() when preemption
> > > > is disabled, call_rcu_irqs_are_disabled() when interrupts are disabled,
> > > > call_rcu_raw_atomic() from contexts where (for example) raw spinlocks
> > > > are held, and so on.  However, from what I can see, most people instead
> > > > consistently prefer that the RCU API instead be consolidated.
> > > >
> > > > Some in-flight cache-efficiency work for kvfree_rcu() and call_rcu()
> > > > needs to be able to allocate memory occasionally.  It can do that when
> > > > invoked from some contexts, but not when invoked from others.  Right now,
> > > > in !PREEMPT kernels, it cannot tell, and must either do things to the
> > > > memory allocators that some of the MM hate or must unnecessarily invoke
> > > > workqueues.  Thomas's patches would allow the code to just allocate in
> > > > the common case when these primitives are invoked from contexts where
> > > > allocation is permitted.
> > > >
> > > > If we want to restrict access to the can_schedule() or whatever primitive,
> > > > fine and good.  We can add a check to checkpatch.pl, for example.  Maybe
> > > > we can go back to the old brlock approach of requiring certain people's
> > > > review for each addition to the kernel.
> > > >
> > > > But there really are use cases that it would greatly help.
> > >
> > > We can deadlock in random fun places if random stuff we're calling
> > > suddenly starts allocating. Sometimes. Maybe once in a blue moon, to
> > > make it extra fun to reproduce. Maybe most driver subsystems are less
> > > brittle, but gpu drivers definitely need to know about the details for
> > > exactly this example. And yes gpu drivers use rcu for freeing
> > > dma_fence structures, and that tends to happen in code that we only
> > > recently figured out should really not allocate memory.
> > >
> > > I think minimally you need to throw in an unconditional
> > > fs_reclaim_acquire();fs_reclaim_release(); so that everyone who runs
> > > with full debugging knows what might happen. It's kinda like
> > > might_sleep, but a lot more specific. might_sleep() alone is not
> > > enough, because in the specific code paths I'm thinking of (and
> > > created special lockdep annotations for just recently) sleeping is
> > > allowed, but any memory allocations with GFP_RECLAIM set are no-go.
> >
> > Completely agreed!  Any allocation on any free path must be handled
> > -extremely- carefully.  To that end...
> >
> > First, there is always a fallback in case the allocation fails.  Which
> > might have performance or corner-case robustness issues, but which will
> > at least allow forward progress.  Second, we consulted with a number of
> > MM experts to arrive at appropriate GFP_* flags (and their patience is
> > greatly appreciated).  Third, the paths that can allocate will do so about
> > one time of 500, so any issues should be spotted sooner rather than later.
> >
> > So you are quite right to be concerned, but I believe we will be doing the
> > right things.  And based on his previous track record, I am also quite
> > certain that Mr. Murphy will be on hand to provide me any additional
> > education that I might require.
> >
> > Finally, I have noted down your point about fs_reclaim_acquire() and
> > fs_reclaim_release().  Whether or not they prove to be needed, I do
> > appreciate your calling them to my attention.
> 
> I just realized that since these dma_fence structs are refcounted and
> userspace can hold references (directly, it can pass them around
> behind file descriptors) we might never hit such a path until slightly
> unusual or evil userspace does something interesting. Do you have
> links to those patches? Some googling didn't turn up anything. I can
> then figure out whether it's better to risk not spotting issues with
> call_rcu vs slapping a memalloc_noio_save/restore around all these
> critical section which force-degrades any allocation to GFP_ATOMIC at
> most, but has the risk that we run into code that assumes "GFP_KERNEL
> never fails for small stuff" and has a decidedly less tested fallback
> path than rcu code.

Here is the previous early draft version, which will change considerably
for the next version:

	lore.kernel.org/lkml/20200809204354.20137-1-urezki@gmail.com

This does kvfree_rcu(), but we expect to handle call_rcu() similarly.

The version in preparation will use workqueues to do the allocation in a
known-safe environment and also use lockless access to certain portions
of the allocator caches (as noted earlier, this last is not much loved
by some of the MM guys).  Given Thomas's patch, we could with high
probability allocate directly, perhaps even not needing memory-allocator
modifications.

Either way, kvfree_rcu(), and later call_rcu(), will avoid asking the
allocator to do anything that the calling context prohibits.  So what
types of bugs are you looking for?  Where reclaim calls back into the
driver or some such?

							Thanx, Paul
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-16 22:39                       ` Paul E. McKenney
  0 siblings, 0 replies; 282+ messages in thread
From: Paul E. McKenney @ 2020-09-16 22:39 UTC (permalink / raw)
  To: Daniel Vetter
  Cc: Linus Torvalds, Thomas Gleixner, Ard Biesheuvel, Herbert Xu,
	LKML, linux-arch, Sebastian Andrzej Siewior, Valentin Schneider,
	Richard Henderson, Ivan Kokshaysky, Matt Turner, alpha,
	Jeff Dike, Richard Weinberger, Anton Ivanov, linux-um,
	Brian Cain, linux-hexagon, Geert Uytterhoeven

On Wed, Sep 16, 2020 at 11:43:02PM +0200, Daniel Vetter wrote:
> On Wed, Sep 16, 2020 at 10:58 PM Paul E. McKenney <paulmck@kernel.org> wrote:
> >
> > On Wed, Sep 16, 2020 at 10:29:06PM +0200, Daniel Vetter wrote:
> > > On Wed, Sep 16, 2020 at 5:29 PM Paul E. McKenney <paulmck@kernel.org> wrote:
> > > >
> > > > On Wed, Sep 16, 2020 at 09:37:17AM +0200, Daniel Vetter wrote:
> > > > > On Tue, Sep 15, 2020 at 7:35 PM Linus Torvalds
> > > > > <torvalds@linux-foundation.org> wrote:
> > > > > >
> > > > > > On Tue, Sep 15, 2020 at 1:39 AM Thomas Gleixner <tglx@linutronix.de> wrote:
> > > > > > >
> > > > > > > OTOH, having a working 'preemptible()' or maybe better named
> > > > > > > 'can_schedule()' check makes tons of sense to make decisions about
> > > > > > > allocation modes or other things.
> > > > > >
> > > > > > No. I think that those kinds of decisions about actual behavior are
> > > > > > always simply fundamentally wrong.
> > > > > >
> > > > > > Note that this is very different from having warnings about invalid
> > > > > > use. THAT is correct. It may not warn in all configurations, but that
> > > > > > doesn't matter: what matters is that it warns in common enough
> > > > > > configurations that developers will catch it.
> > > > > >
> > > > > > So having a warning in "might_sleep()" that doesn't always trigger,
> > > > > > because you have a limited configuration that can't even detect the
> > > > > > situation, that's fine and dandy and intentional.
> > > > > >
> > > > > > But having code like
> > > > > >
> > > > > >        if (can_schedule())
> > > > > >            .. do something different ..
> > > > > >
> > > > > > is fundamentally complete and utter garbage.
> > > > > >
> > > > > > It's one thing if you test for "am I in hardware interrupt context".
> > > > > > Those tests aren't great either, but at least they make sense.
> > > > > >
> > > > > > But a driver - or some library routine - making a difference based on
> > > > > > some nebulous "can I schedule" is fundamentally and basically WRONG.
> > > > > >
> > > > > > If some code changes behavior, it needs to be explicit to the *caller*
> > > > > > of that code.
> > > > > >
> > > > > > So this is why GFP_ATOMIC is fine, but "if (!can_schedule())
> > > > > > do_something_atomic()" is pure shite.
> > > > > >
> > > > > > And I am not IN THE LEAST interested in trying to help people doing
> > > > > > pure shite. We need to fix them. Like the crypto code is getting
> > > > > > fixed.
> > > > >
> > > > > Just figured I'll throw my +1 in from reading too many (gpu) drivers.
> > > > > Code that tries to cleverly adjust its behaviour depending upon the
> > > > > context it's running in is harder to understand and blows up in more
> > > > > interesting ways. We still have drm_can_sleep() and it's mostly just
> > > > > used for debug code, and I've largely ended up just deleting
> > > > > everything that used it because when you're driver is blowing up the
> > > > > last thing you want is to realize your debug code and output can't be
> > > > > relied upon. Or worse, that the only Oops you have is the one in the
> > > > > debug code, because the real one scrolled away - the original idea
> > > > > behind drm_can_sleep was to make all the modeset code work
> > > > > automagically both in normal ioctl/kworker context and in the panic
> > > > > handlers or kgdb callbacks. Wishful thinking at best.
> > > > >
> > > > > Also at least for me that extends to everything, e.g. I much prefer
> > > > > explicit spin_lock and spin_lock_irq vs magic spin_lock_irqsave for
> > > > > locks shared with interrupt handlers, since the former two gives me
> > > > > clear information from which contexts such function can be called.
> > > > > Other end is the memalloc_no*_save/restore functions, where I recently
> > > > > made a real big fool of myself because I didn't realize how much that
> > > > > impacts everything that's run within - suddenly "GFP_KERNEL for small
> > > > > stuff never fails" is wrong everywhere.
> > > > >
> > > > > It's all great for debugging and sanity checks (and we run with all
> > > > > that stuff enabled in our CI), but really semantic changes depending
> > > > > upon magic context checks freak my out :-)
> > > >
> > > > All fair, but some of us need to write code that must handle being
> > > > invoked from a wide variety of contexts.  Now perhaps you like the idea of
> > > > call_rcu() for schedulable contexts, call_rcu_nosched() when preemption
> > > > is disabled, call_rcu_irqs_are_disabled() when interrupts are disabled,
> > > > call_rcu_raw_atomic() from contexts where (for example) raw spinlocks
> > > > are held, and so on.  However, from what I can see, most people instead
> > > > consistently prefer that the RCU API instead be consolidated.
> > > >
> > > > Some in-flight cache-efficiency work for kvfree_rcu() and call_rcu()
> > > > needs to be able to allocate memory occasionally.  It can do that when
> > > > invoked from some contexts, but not when invoked from others.  Right now,
> > > > in !PREEMPT kernels, it cannot tell, and must either do things to the
> > > > memory allocators that some of the MM hate or must unnecessarily invoke
> > > > workqueues.  Thomas's patches would allow the code to just allocate in
> > > > the common case when these primitives are invoked from contexts where
> > > > allocation is permitted.
> > > >
> > > > If we want to restrict access to the can_schedule() or whatever primitive,
> > > > fine and good.  We can add a check to checkpatch.pl, for example.  Maybe
> > > > we can go back to the old brlock approach of requiring certain people's
> > > > review for each addition to the kernel.
> > > >
> > > > But there really are use cases that it would greatly help.
> > >
> > > We can deadlock in random fun places if random stuff we're calling
> > > suddenly starts allocating. Sometimes. Maybe once in a blue moon, to
> > > make it extra fun to reproduce. Maybe most driver subsystems are less
> > > brittle, but gpu drivers definitely need to know about the details for
> > > exactly this example. And yes gpu drivers use rcu for freeing
> > > dma_fence structures, and that tends to happen in code that we only
> > > recently figured out should really not allocate memory.
> > >
> > > I think minimally you need to throw in an unconditional
> > > fs_reclaim_acquire();fs_reclaim_release(); so that everyone who runs
> > > with full debugging knows what might happen. It's kinda like
> > > might_sleep, but a lot more specific. might_sleep() alone is not
> > > enough, because in the specific code paths I'm thinking of (and
> > > created special lockdep annotations for just recently) sleeping is
> > > allowed, but any memory allocations with GFP_RECLAIM set are no-go.
> >
> > Completely agreed!  Any allocation on any free path must be handled
> > -extremely- carefully.  To that end...
> >
> > First, there is always a fallback in case the allocation fails.  Which
> > might have performance or corner-case robustness issues, but which will
> > at least allow forward progress.  Second, we consulted with a number of
> > MM experts to arrive at appropriate GFP_* flags (and their patience is
> > greatly appreciated).  Third, the paths that can allocate will do so about
> > one time of 500, so any issues should be spotted sooner rather than later.
> >
> > So you are quite right to be concerned, but I believe we will be doing the
> > right things.  And based on his previous track record, I am also quite
> > certain that Mr. Murphy will be on hand to provide me any additional
> > education that I might require.
> >
> > Finally, I have noted down your point about fs_reclaim_acquire() and
> > fs_reclaim_release().  Whether or not they prove to be needed, I do
> > appreciate your calling them to my attention.
> 
> I just realized that since these dma_fence structs are refcounted and
> userspace can hold references (directly, it can pass them around
> behind file descriptors) we might never hit such a path until slightly
> unusual or evil userspace does something interesting. Do you have
> links to those patches? Some googling didn't turn up anything. I can
> then figure out whether it's better to risk not spotting issues with
> call_rcu vs slapping a memalloc_noio_save/restore around all these
> critical section which force-degrades any allocation to GFP_ATOMIC at
> most, but has the risk that we run into code that assumes "GFP_KERNEL
> never fails for small stuff" and has a decidedly less tested fallback
> path than rcu code.

Here is the previous early draft version, which will change considerably
for the next version:

	lore.kernel.org/lkml/20200809204354.20137-1-urezki@gmail.com

This does kvfree_rcu(), but we expect to handle call_rcu() similarly.

The version in preparation will use workqueues to do the allocation in a
known-safe environment and also use lockless access to certain portions
of the allocator caches (as noted earlier, this last is not much loved
by some of the MM guys).  Given Thomas's patch, we could with high
probability allocate directly, perhaps even not needing memory-allocator
modifications.

Either way, kvfree_rcu(), and later call_rcu(), will avoid asking the
allocator to do anything that the calling context prohibits.  So what
types of bugs are you looking for?  Where reclaim calls back into the
driver or some such?

							Thanx, Paul

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
  2020-09-16 18:32                 ` Linus Torvalds
                                     ` (2 preceding siblings ...)
  (?)
@ 2020-09-17  6:38                   ` Ard Biesheuvel
  -1 siblings, 0 replies; 282+ messages in thread
From: Ard Biesheuvel @ 2020-09-17  6:38 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Paul E. McKenney, Daniel Vetter, Thomas Gleixner, Herbert Xu,
	LKML, linux-arch, Sebastian Andrzej Siewior, Valentin Schneider,
	Richard Henderson, Ivan Kokshaysky, Matt Turner, alpha,
	Jeff Dike, Richard Weinberger, Anton Ivanov, linux-um,
	Brian Cain, linux-hexagon, Geert Uytterhoeven, linux-m68k,
	Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
	Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman,
	Daniel Bristot de Oliveira, Will Deacon, Andrew Morton, Linux-MM,
	Ingo Molnar, Russell King, Linux ARM, Chris Zankel, Max Filippov,
	linux-xtensa, Jani Nikula, Joonas Lahtinen, Rodrigo Vivi,
	David Airlie, intel-gfx, dri-devel, Josh Triplett,
	Mathieu Desnoyers, Lai Jiangshan, Shuah Khan, rcu,
	open list:KERNEL SELFTEST FRAMEWORK

On Wed, 16 Sep 2020 at 21:32, Linus Torvalds
<torvalds@linux-foundation.org> wrote:
>
> But something like a driver list walking thing should not be doing
> different things behind peoples back depending on whether they hold
> spinlocks or not. It should either just work regardless, or there
> should be a flag (or special interface) for the "you're being called
> in a crtitical region".
>
> Because dynamically changing behavior really is very confusing.
>

By the same reasoning, I don't think a generic crypto library should
be playing tricks with preemption en/disabling under the hood when
iterating over some data that is all directly accessible via the
linear map on the platforms that most people care about. And using
kmap_atomic() unconditionally achieves exactly that.

As I argued before, the fact that kmap_atomic() can be called from an
atomic context, and the fact that its implementation on HIGHMEM
platforms requires preemption to be disabled until the next kunmap()
are two different things, and I don't agree with your assertion that
the name kmap_atomic() implies the latter semantics. If we can avoid
disabling preemption on HIGHMEM, as Thomas suggests, we surely don't
need it on !HIGHMEM either, and given that kmap_atomic() is preferred
today anyway, we can just merge the two implementations. Are there any
existing debug features that could help us spot [ab]use of things like
raw per-CPU data within kmap_atomic regions?

Re your point about deprecating HIGHMEM: some work is underway on ARM
to implement a 3.75/3.75 GB kernel/user split on recent LPAE capable
hardware (which shouldn't suffer from the performance issues that
plagued the 4/4 split on i686), and so hopefully, there is a path
forward for ARM that does not rely on HIGHMEM as it does today.

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-17  6:38                   ` Ard Biesheuvel
  0 siblings, 0 replies; 282+ messages in thread
From: Ard Biesheuvel @ 2020-09-17  6:38 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Paul E. McKenney, Daniel Vetter, Thomas Gleixner, Herbert Xu,
	LKML, linux-arch, Sebastian Andrzej Siewior, Valentin Schneider,
	Richard Henderson, Ivan Kokshaysky, Matt Turner, alpha,
	Jeff Dike, Richard Weinberger, Anton Ivanov, linux-um,
	Brian Cain, linux-hexagon, Geert Uytterhoeven, linux-m68k,
	Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
	Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman,
	Daniel Bristot de Oliveira, Will Deacon, Andrew Morton, Linux-MM,
	Ingo Molnar, Russell King, Linux ARM, Chris Zankel, Max Filippov,
	linux-xtensa, Jani Nikula, Joonas Lahtinen, Rodrigo Vivi,
	David Airlie, intel-gfx, dri-devel, Josh Triplett,
	Mathieu Desnoyers, Lai Jiangshan, Shuah Khan, rcu,
	open list:KERNEL SELFTEST FRAMEWORK

On Wed, 16 Sep 2020 at 21:32, Linus Torvalds
<torvalds@linux-foundation.org> wrote:
>
> But something like a driver list walking thing should not be doing
> different things behind peoples back depending on whether they hold
> spinlocks or not. It should either just work regardless, or there
> should be a flag (or special interface) for the "you're being called
> in a crtitical region".
>
> Because dynamically changing behavior really is very confusing.
>

By the same reasoning, I don't think a generic crypto library should
be playing tricks with preemption en/disabling under the hood when
iterating over some data that is all directly accessible via the
linear map on the platforms that most people care about. And using
kmap_atomic() unconditionally achieves exactly that.

As I argued before, the fact that kmap_atomic() can be called from an
atomic context, and the fact that its implementation on HIGHMEM
platforms requires preemption to be disabled until the next kunmap()
are two different things, and I don't agree with your assertion that
the name kmap_atomic() implies the latter semantics. If we can avoid
disabling preemption on HIGHMEM, as Thomas suggests, we surely don't
need it on !HIGHMEM either, and given that kmap_atomic() is preferred
today anyway, we can just merge the two implementations. Are there any
existing debug features that could help us spot [ab]use of things like
raw per-CPU data within kmap_atomic regions?

Re your point about deprecating HIGHMEM: some work is underway on ARM
to implement a 3.75/3.75 GB kernel/user split on recent LPAE capable
hardware (which shouldn't suffer from the performance issues that
plagued the 4/4 split on i686), and so hopefully, there is a path
forward for ARM that does not rely on HIGHMEM as it does today.


^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-17  6:38                   ` Ard Biesheuvel
  0 siblings, 0 replies; 282+ messages in thread
From: Ard Biesheuvel @ 2020-09-17  6:38 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, Linux-MM,
	open list:KERNEL SELFTEST FRAMEWORK, linux-hexagon, Will Deacon,
	Ingo Molnar, Anton Ivanov, linux-arch, Vincent Guittot,
	Herbert Xu, Brian Cain, Richard Weinberger, Russell King,
	David Airlie, Ingo Molnar, Geert Uytterhoeven, Mel Gorman,
	intel-gfx, Matt Turner, Valentin Schneider, linux-xtensa,
	Shuah Khan, Paul E. McKenney, Jeff Dike, linux-um, Josh Triplett,
	Steven Rostedt, rcu, linux-m68k, Ivan Kokshaysky, Rodrigo Vivi,
	Thomas Gleixner, Dietmar Eggemann, Linux ARM, Richard Henderson,
	Chris Zankel, Max Filippov, LKML, alpha, Mathieu Desnoyers,
	Andrew Morton, Daniel Bristot de Oliveira

On Wed, 16 Sep 2020 at 21:32, Linus Torvalds
<torvalds@linux-foundation.org> wrote:
>
> But something like a driver list walking thing should not be doing
> different things behind peoples back depending on whether they hold
> spinlocks or not. It should either just work regardless, or there
> should be a flag (or special interface) for the "you're being called
> in a crtitical region".
>
> Because dynamically changing behavior really is very confusing.
>

By the same reasoning, I don't think a generic crypto library should
be playing tricks with preemption en/disabling under the hood when
iterating over some data that is all directly accessible via the
linear map on the platforms that most people care about. And using
kmap_atomic() unconditionally achieves exactly that.

As I argued before, the fact that kmap_atomic() can be called from an
atomic context, and the fact that its implementation on HIGHMEM
platforms requires preemption to be disabled until the next kunmap()
are two different things, and I don't agree with your assertion that
the name kmap_atomic() implies the latter semantics. If we can avoid
disabling preemption on HIGHMEM, as Thomas suggests, we surely don't
need it on !HIGHMEM either, and given that kmap_atomic() is preferred
today anyway, we can just merge the two implementations. Are there any
existing debug features that could help us spot [ab]use of things like
raw per-CPU data within kmap_atomic regions?

Re your point about deprecating HIGHMEM: some work is underway on ARM
to implement a 3.75/3.75 GB kernel/user split on recent LPAE capable
hardware (which shouldn't suffer from the performance issues that
plagued the 4/4 split on i686), and so hopefully, there is a path
forward for ARM that does not rely on HIGHMEM as it does today.
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [Intel-gfx] [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-17  6:38                   ` Ard Biesheuvel
  0 siblings, 0 replies; 282+ messages in thread
From: Ard Biesheuvel @ 2020-09-17  6:38 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, Linux-MM,
	open list:KERNEL SELFTEST FRAMEWORK, linux-hexagon, Will Deacon,
	Ingo Molnar, Anton Ivanov, linux-arch, Herbert Xu, Brian Cain,
	Richard Weinberger, Russell King, David Airlie, Ingo Molnar,
	Geert Uytterhoeven, Mel Gorman, intel-gfx, Matt Turner,
	Valentin Schneider, linux-xtensa, Shuah Khan, Paul E. McKenney,
	Jeff Dike, linux-um, Josh Triplett, Steven Rostedt, rcu,
	linux-m68k, Ivan Kokshaysky, Thomas Gleixner, Dietmar Eggemann,
	Linux ARM, Richard Henderson, Chris Zankel, Max Filippov, LKML,
	alpha, Mathieu Desnoyers, Andrew Morton,
	Daniel Bristot de Oliveira

On Wed, 16 Sep 2020 at 21:32, Linus Torvalds
<torvalds@linux-foundation.org> wrote:
>
> But something like a driver list walking thing should not be doing
> different things behind peoples back depending on whether they hold
> spinlocks or not. It should either just work regardless, or there
> should be a flag (or special interface) for the "you're being called
> in a crtitical region".
>
> Because dynamically changing behavior really is very confusing.
>

By the same reasoning, I don't think a generic crypto library should
be playing tricks with preemption en/disabling under the hood when
iterating over some data that is all directly accessible via the
linear map on the platforms that most people care about. And using
kmap_atomic() unconditionally achieves exactly that.

As I argued before, the fact that kmap_atomic() can be called from an
atomic context, and the fact that its implementation on HIGHMEM
platforms requires preemption to be disabled until the next kunmap()
are two different things, and I don't agree with your assertion that
the name kmap_atomic() implies the latter semantics. If we can avoid
disabling preemption on HIGHMEM, as Thomas suggests, we surely don't
need it on !HIGHMEM either, and given that kmap_atomic() is preferred
today anyway, we can just merge the two implementations. Are there any
existing debug features that could help us spot [ab]use of things like
raw per-CPU data within kmap_atomic regions?

Re your point about deprecating HIGHMEM: some work is underway on ARM
to implement a 3.75/3.75 GB kernel/user split on recent LPAE capable
hardware (which shouldn't suffer from the performance issues that
plagued the 4/4 split on i686), and so hopefully, there is a path
forward for ARM that does not rely on HIGHMEM as it does today.
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-17  6:38                   ` Ard Biesheuvel
  0 siblings, 0 replies; 282+ messages in thread
From: Ard Biesheuvel @ 2020-09-17  6:38 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Paul E. McKenney, Daniel Vetter, Thomas Gleixner, Herbert Xu,
	LKML, linux-arch, Sebastian Andrzej Siewior, Valentin Schneider,
	Richard Henderson, Ivan Kokshaysky, Matt Turner, alpha,
	Jeff Dike, Richard Weinberger, Anton Ivanov, linux-um,
	Brian Cain, linux-hexagon, Geert Uytterhoeven

On Wed, 16 Sep 2020 at 21:32, Linus Torvalds
<torvalds@linux-foundation.org> wrote:
>
> But something like a driver list walking thing should not be doing
> different things behind peoples back depending on whether they hold
> spinlocks or not. It should either just work regardless, or there
> should be a flag (or special interface) for the "you're being called
> in a crtitical region".
>
> Because dynamically changing behavior really is very confusing.
>

By the same reasoning, I don't think a generic crypto library should
be playing tricks with preemption en/disabling under the hood when
iterating over some data that is all directly accessible via the
linear map on the platforms that most people care about. And using
kmap_atomic() unconditionally achieves exactly that.

As I argued before, the fact that kmap_atomic() can be called from an
atomic context, and the fact that its implementation on HIGHMEM
platforms requires preemption to be disabled until the next kunmap()
are two different things, and I don't agree with your assertion that
the name kmap_atomic() implies the latter semantics. If we can avoid
disabling preemption on HIGHMEM, as Thomas suggests, we surely don't
need it on !HIGHMEM either, and given that kmap_atomic() is preferred
today anyway, we can just merge the two implementations. Are there any
existing debug features that could help us spot [ab]use of things like
raw per-CPU data within kmap_atomic regions?

Re your point about deprecating HIGHMEM: some work is underway on ARM
to implement a 3.75/3.75 GB kernel/user split on recent LPAE capable
hardware (which shouldn't suffer from the performance issues that
plagued the 4/4 split on i686), and so hopefully, there is a path
forward for ARM that does not rely on HIGHMEM as it does today.

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
  2020-09-16 22:39                       ` Paul E. McKenney
                                           ` (2 preceding siblings ...)
  (?)
@ 2020-09-17  7:52                         ` Daniel Vetter
  -1 siblings, 0 replies; 282+ messages in thread
From: Daniel Vetter @ 2020-09-17  7:52 UTC (permalink / raw)
  To: Paul E. McKenney
  Cc: Linus Torvalds, Thomas Gleixner, Ard Biesheuvel, Herbert Xu,
	LKML, linux-arch, Sebastian Andrzej Siewior, Valentin Schneider,
	Richard Henderson, Ivan Kokshaysky, Matt Turner, alpha,
	Jeff Dike, Richard Weinberger, Anton Ivanov, linux-um,
	Brian Cain, linux-hexagon, Geert Uytterhoeven, linux-m68k,
	Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
	Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman,
	Daniel Bristot de Oliveira, Will Deacon, Andrew Morton, Linux-MM,
	Ingo Molnar, Russell King, Linux ARM, Chris Zankel, Max Filippov,
	linux-xtensa, Jani Nikula, Joonas Lahtinen, Rodrigo Vivi,
	David Airlie, intel-gfx, dri-devel, Josh Triplett,
	Mathieu Desnoyers, Lai Jiangshan, Shuah Khan, rcu,
	open list:KERNEL SELFTEST FRAMEWORK

On Thu, Sep 17, 2020 at 12:39 AM Paul E. McKenney <paulmck@kernel.org> wrote:
>
> On Wed, Sep 16, 2020 at 11:43:02PM +0200, Daniel Vetter wrote:
> > On Wed, Sep 16, 2020 at 10:58 PM Paul E. McKenney <paulmck@kernel.org> wrote:
> > >
> > > On Wed, Sep 16, 2020 at 10:29:06PM +0200, Daniel Vetter wrote:
> > > > On Wed, Sep 16, 2020 at 5:29 PM Paul E. McKenney <paulmck@kernel.org> wrote:
> > > > >
> > > > > On Wed, Sep 16, 2020 at 09:37:17AM +0200, Daniel Vetter wrote:
> > > > > > On Tue, Sep 15, 2020 at 7:35 PM Linus Torvalds
> > > > > > <torvalds@linux-foundation.org> wrote:
> > > > > > >
> > > > > > > On Tue, Sep 15, 2020 at 1:39 AM Thomas Gleixner <tglx@linutronix.de> wrote:
> > > > > > > >
> > > > > > > > OTOH, having a working 'preemptible()' or maybe better named
> > > > > > > > 'can_schedule()' check makes tons of sense to make decisions about
> > > > > > > > allocation modes or other things.
> > > > > > >
> > > > > > > No. I think that those kinds of decisions about actual behavior are
> > > > > > > always simply fundamentally wrong.
> > > > > > >
> > > > > > > Note that this is very different from having warnings about invalid
> > > > > > > use. THAT is correct. It may not warn in all configurations, but that
> > > > > > > doesn't matter: what matters is that it warns in common enough
> > > > > > > configurations that developers will catch it.
> > > > > > >
> > > > > > > So having a warning in "might_sleep()" that doesn't always trigger,
> > > > > > > because you have a limited configuration that can't even detect the
> > > > > > > situation, that's fine and dandy and intentional.
> > > > > > >
> > > > > > > But having code like
> > > > > > >
> > > > > > >        if (can_schedule())
> > > > > > >            .. do something different ..
> > > > > > >
> > > > > > > is fundamentally complete and utter garbage.
> > > > > > >
> > > > > > > It's one thing if you test for "am I in hardware interrupt context".
> > > > > > > Those tests aren't great either, but at least they make sense.
> > > > > > >
> > > > > > > But a driver - or some library routine - making a difference based on
> > > > > > > some nebulous "can I schedule" is fundamentally and basically WRONG.
> > > > > > >
> > > > > > > If some code changes behavior, it needs to be explicit to the *caller*
> > > > > > > of that code.
> > > > > > >
> > > > > > > So this is why GFP_ATOMIC is fine, but "if (!can_schedule())
> > > > > > > do_something_atomic()" is pure shite.
> > > > > > >
> > > > > > > And I am not IN THE LEAST interested in trying to help people doing
> > > > > > > pure shite. We need to fix them. Like the crypto code is getting
> > > > > > > fixed.
> > > > > >
> > > > > > Just figured I'll throw my +1 in from reading too many (gpu) drivers.
> > > > > > Code that tries to cleverly adjust its behaviour depending upon the
> > > > > > context it's running in is harder to understand and blows up in more
> > > > > > interesting ways. We still have drm_can_sleep() and it's mostly just
> > > > > > used for debug code, and I've largely ended up just deleting
> > > > > > everything that used it because when you're driver is blowing up the
> > > > > > last thing you want is to realize your debug code and output can't be
> > > > > > relied upon. Or worse, that the only Oops you have is the one in the
> > > > > > debug code, because the real one scrolled away - the original idea
> > > > > > behind drm_can_sleep was to make all the modeset code work
> > > > > > automagically both in normal ioctl/kworker context and in the panic
> > > > > > handlers or kgdb callbacks. Wishful thinking at best.
> > > > > >
> > > > > > Also at least for me that extends to everything, e.g. I much prefer
> > > > > > explicit spin_lock and spin_lock_irq vs magic spin_lock_irqsave for
> > > > > > locks shared with interrupt handlers, since the former two gives me
> > > > > > clear information from which contexts such function can be called.
> > > > > > Other end is the memalloc_no*_save/restore functions, where I recently
> > > > > > made a real big fool of myself because I didn't realize how much that
> > > > > > impacts everything that's run within - suddenly "GFP_KERNEL for small
> > > > > > stuff never fails" is wrong everywhere.
> > > > > >
> > > > > > It's all great for debugging and sanity checks (and we run with all
> > > > > > that stuff enabled in our CI), but really semantic changes depending
> > > > > > upon magic context checks freak my out :-)
> > > > >
> > > > > All fair, but some of us need to write code that must handle being
> > > > > invoked from a wide variety of contexts.  Now perhaps you like the idea of
> > > > > call_rcu() for schedulable contexts, call_rcu_nosched() when preemption
> > > > > is disabled, call_rcu_irqs_are_disabled() when interrupts are disabled,
> > > > > call_rcu_raw_atomic() from contexts where (for example) raw spinlocks
> > > > > are held, and so on.  However, from what I can see, most people instead
> > > > > consistently prefer that the RCU API instead be consolidated.
> > > > >
> > > > > Some in-flight cache-efficiency work for kvfree_rcu() and call_rcu()
> > > > > needs to be able to allocate memory occasionally.  It can do that when
> > > > > invoked from some contexts, but not when invoked from others.  Right now,
> > > > > in !PREEMPT kernels, it cannot tell, and must either do things to the
> > > > > memory allocators that some of the MM hate or must unnecessarily invoke
> > > > > workqueues.  Thomas's patches would allow the code to just allocate in
> > > > > the common case when these primitives are invoked from contexts where
> > > > > allocation is permitted.
> > > > >
> > > > > If we want to restrict access to the can_schedule() or whatever primitive,
> > > > > fine and good.  We can add a check to checkpatch.pl, for example.  Maybe
> > > > > we can go back to the old brlock approach of requiring certain people's
> > > > > review for each addition to the kernel.
> > > > >
> > > > > But there really are use cases that it would greatly help.
> > > >
> > > > We can deadlock in random fun places if random stuff we're calling
> > > > suddenly starts allocating. Sometimes. Maybe once in a blue moon, to
> > > > make it extra fun to reproduce. Maybe most driver subsystems are less
> > > > brittle, but gpu drivers definitely need to know about the details for
> > > > exactly this example. And yes gpu drivers use rcu for freeing
> > > > dma_fence structures, and that tends to happen in code that we only
> > > > recently figured out should really not allocate memory.
> > > >
> > > > I think minimally you need to throw in an unconditional
> > > > fs_reclaim_acquire();fs_reclaim_release(); so that everyone who runs
> > > > with full debugging knows what might happen. It's kinda like
> > > > might_sleep, but a lot more specific. might_sleep() alone is not
> > > > enough, because in the specific code paths I'm thinking of (and
> > > > created special lockdep annotations for just recently) sleeping is
> > > > allowed, but any memory allocations with GFP_RECLAIM set are no-go.
> > >
> > > Completely agreed!  Any allocation on any free path must be handled
> > > -extremely- carefully.  To that end...
> > >
> > > First, there is always a fallback in case the allocation fails.  Which
> > > might have performance or corner-case robustness issues, but which will
> > > at least allow forward progress.  Second, we consulted with a number of
> > > MM experts to arrive at appropriate GFP_* flags (and their patience is
> > > greatly appreciated).  Third, the paths that can allocate will do so about
> > > one time of 500, so any issues should be spotted sooner rather than later.
> > >
> > > So you are quite right to be concerned, but I believe we will be doing the
> > > right things.  And based on his previous track record, I am also quite
> > > certain that Mr. Murphy will be on hand to provide me any additional
> > > education that I might require.
> > >
> > > Finally, I have noted down your point about fs_reclaim_acquire() and
> > > fs_reclaim_release().  Whether or not they prove to be needed, I do
> > > appreciate your calling them to my attention.
> >
> > I just realized that since these dma_fence structs are refcounted and
> > userspace can hold references (directly, it can pass them around
> > behind file descriptors) we might never hit such a path until slightly
> > unusual or evil userspace does something interesting. Do you have
> > links to those patches? Some googling didn't turn up anything. I can
> > then figure out whether it's better to risk not spotting issues with
> > call_rcu vs slapping a memalloc_noio_save/restore around all these
> > critical section which force-degrades any allocation to GFP_ATOMIC at
> > most, but has the risk that we run into code that assumes "GFP_KERNEL
> > never fails for small stuff" and has a decidedly less tested fallback
> > path than rcu code.
>
> Here is the previous early draft version, which will change considerably
> for the next version:
>
>         lore.kernel.org/lkml/20200809204354.20137-1-urezki@gmail.com
>
> This does kvfree_rcu(), but we expect to handle call_rcu() similarly.
>
> The version in preparation will use workqueues to do the allocation in a
> known-safe environment and also use lockless access to certain portions
> of the allocator caches (as noted earlier, this last is not much loved
> by some of the MM guys).  Given Thomas's patch, we could with high
> probability allocate directly, perhaps even not needing memory-allocator
> modifications.
>
> Either way, kvfree_rcu(), and later call_rcu(), will avoid asking the
> allocator to do anything that the calling context prohibits.  So what
> types of bugs are you looking for?  Where reclaim calls back into the
> driver or some such?

Yeah pretty much. It's a problem for gpu, fs, block drivers and really
anything else that's remotely involved in memory reclaim somehow.
Generally this is all handled explicitly by passing gfp_t flags down
any call chain, but in some cases it's instead solved with the
memalloc_no* functions. E.g. sunrpc uses that to make sure the network
stack (which generally just assumes it can allocate memory) doesn't,
to avoid recursions back into nfs/sunrpc. To my knowledge there's no
way to check at runtime with which gfp flags you're allowed to
allocate memory, a preemptible check is definitely not enough.
Disabled preemption implies only GFP_ATOMIC is allowed (ignoring nmi
and stuff like that), but the inverse is not true.

So if you want the automagic in call_rcu I think either
- we need to replace all explicit gfp flags with the context marking
memalloc_no* across the entire kernel, or at least anywhere rcu might
be used.
- audit all callchains and make sure a call_rcu_noalloc is used
anywhere there might be a problem. probably better to have a
call_rcu_gfp with explicit gfp flags parameter, since generally that
needs to be passed down.

But at least to me the lockless magic in mm sounds a lot safer, since
it contains the complexity and doesn't leak it out to callers of
call_rcu.
-Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-17  7:52                         ` Daniel Vetter
  0 siblings, 0 replies; 282+ messages in thread
From: Daniel Vetter @ 2020-09-17  7:52 UTC (permalink / raw)
  To: Paul E. McKenney
  Cc: Linus Torvalds, Thomas Gleixner, Ard Biesheuvel, Herbert Xu,
	LKML, linux-arch, Sebastian Andrzej Siewior, Valentin Schneider,
	Richard Henderson, Ivan Kokshaysky, Matt Turner, alpha,
	Jeff Dike, Richard Weinberger, Anton Ivanov, linux-um,
	Brian Cain, linux-hexagon, Geert Uytterhoeven, linux-m68k,
	Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
	Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman,
	Daniel Bristot de Oliveira, Will Deacon, Andrew Morton, Linux-MM,
	Ingo Molnar, Russell King, Linux ARM, Chris Zankel, Max Filippov,
	linux-xtensa, Jani Nikula, Joonas Lahtinen, Rodrigo Vivi,
	David Airlie, intel-gfx, dri-devel, Josh Triplett,
	Mathieu Desnoyers, Lai Jiangshan, Shuah Khan, rcu,
	open list:KERNEL SELFTEST FRAMEWORK

On Thu, Sep 17, 2020 at 12:39 AM Paul E. McKenney <paulmck@kernel.org> wrote:
>
> On Wed, Sep 16, 2020 at 11:43:02PM +0200, Daniel Vetter wrote:
> > On Wed, Sep 16, 2020 at 10:58 PM Paul E. McKenney <paulmck@kernel.org> wrote:
> > >
> > > On Wed, Sep 16, 2020 at 10:29:06PM +0200, Daniel Vetter wrote:
> > > > On Wed, Sep 16, 2020 at 5:29 PM Paul E. McKenney <paulmck@kernel.org> wrote:
> > > > >
> > > > > On Wed, Sep 16, 2020 at 09:37:17AM +0200, Daniel Vetter wrote:
> > > > > > On Tue, Sep 15, 2020 at 7:35 PM Linus Torvalds
> > > > > > <torvalds@linux-foundation.org> wrote:
> > > > > > >
> > > > > > > On Tue, Sep 15, 2020 at 1:39 AM Thomas Gleixner <tglx@linutronix.de> wrote:
> > > > > > > >
> > > > > > > > OTOH, having a working 'preemptible()' or maybe better named
> > > > > > > > 'can_schedule()' check makes tons of sense to make decisions about
> > > > > > > > allocation modes or other things.
> > > > > > >
> > > > > > > No. I think that those kinds of decisions about actual behavior are
> > > > > > > always simply fundamentally wrong.
> > > > > > >
> > > > > > > Note that this is very different from having warnings about invalid
> > > > > > > use. THAT is correct. It may not warn in all configurations, but that
> > > > > > > doesn't matter: what matters is that it warns in common enough
> > > > > > > configurations that developers will catch it.
> > > > > > >
> > > > > > > So having a warning in "might_sleep()" that doesn't always trigger,
> > > > > > > because you have a limited configuration that can't even detect the
> > > > > > > situation, that's fine and dandy and intentional.
> > > > > > >
> > > > > > > But having code like
> > > > > > >
> > > > > > >        if (can_schedule())
> > > > > > >            .. do something different ..
> > > > > > >
> > > > > > > is fundamentally complete and utter garbage.
> > > > > > >
> > > > > > > It's one thing if you test for "am I in hardware interrupt context".
> > > > > > > Those tests aren't great either, but at least they make sense.
> > > > > > >
> > > > > > > But a driver - or some library routine - making a difference based on
> > > > > > > some nebulous "can I schedule" is fundamentally and basically WRONG.
> > > > > > >
> > > > > > > If some code changes behavior, it needs to be explicit to the *caller*
> > > > > > > of that code.
> > > > > > >
> > > > > > > So this is why GFP_ATOMIC is fine, but "if (!can_schedule())
> > > > > > > do_something_atomic()" is pure shite.
> > > > > > >
> > > > > > > And I am not IN THE LEAST interested in trying to help people doing
> > > > > > > pure shite. We need to fix them. Like the crypto code is getting
> > > > > > > fixed.
> > > > > >
> > > > > > Just figured I'll throw my +1 in from reading too many (gpu) drivers.
> > > > > > Code that tries to cleverly adjust its behaviour depending upon the
> > > > > > context it's running in is harder to understand and blows up in more
> > > > > > interesting ways. We still have drm_can_sleep() and it's mostly just
> > > > > > used for debug code, and I've largely ended up just deleting
> > > > > > everything that used it because when you're driver is blowing up the
> > > > > > last thing you want is to realize your debug code and output can't be
> > > > > > relied upon. Or worse, that the only Oops you have is the one in the
> > > > > > debug code, because the real one scrolled away - the original idea
> > > > > > behind drm_can_sleep was to make all the modeset code work
> > > > > > automagically both in normal ioctl/kworker context and in the panic
> > > > > > handlers or kgdb callbacks. Wishful thinking at best.
> > > > > >
> > > > > > Also at least for me that extends to everything, e.g. I much prefer
> > > > > > explicit spin_lock and spin_lock_irq vs magic spin_lock_irqsave for
> > > > > > locks shared with interrupt handlers, since the former two gives me
> > > > > > clear information from which contexts such function can be called.
> > > > > > Other end is the memalloc_no*_save/restore functions, where I recently
> > > > > > made a real big fool of myself because I didn't realize how much that
> > > > > > impacts everything that's run within - suddenly "GFP_KERNEL for small
> > > > > > stuff never fails" is wrong everywhere.
> > > > > >
> > > > > > It's all great for debugging and sanity checks (and we run with all
> > > > > > that stuff enabled in our CI), but really semantic changes depending
> > > > > > upon magic context checks freak my out :-)
> > > > >
> > > > > All fair, but some of us need to write code that must handle being
> > > > > invoked from a wide variety of contexts.  Now perhaps you like the idea of
> > > > > call_rcu() for schedulable contexts, call_rcu_nosched() when preemption
> > > > > is disabled, call_rcu_irqs_are_disabled() when interrupts are disabled,
> > > > > call_rcu_raw_atomic() from contexts where (for example) raw spinlocks
> > > > > are held, and so on.  However, from what I can see, most people instead
> > > > > consistently prefer that the RCU API instead be consolidated.
> > > > >
> > > > > Some in-flight cache-efficiency work for kvfree_rcu() and call_rcu()
> > > > > needs to be able to allocate memory occasionally.  It can do that when
> > > > > invoked from some contexts, but not when invoked from others.  Right now,
> > > > > in !PREEMPT kernels, it cannot tell, and must either do things to the
> > > > > memory allocators that some of the MM hate or must unnecessarily invoke
> > > > > workqueues.  Thomas's patches would allow the code to just allocate in
> > > > > the common case when these primitives are invoked from contexts where
> > > > > allocation is permitted.
> > > > >
> > > > > If we want to restrict access to the can_schedule() or whatever primitive,
> > > > > fine and good.  We can add a check to checkpatch.pl, for example.  Maybe
> > > > > we can go back to the old brlock approach of requiring certain people's
> > > > > review for each addition to the kernel.
> > > > >
> > > > > But there really are use cases that it would greatly help.
> > > >
> > > > We can deadlock in random fun places if random stuff we're calling
> > > > suddenly starts allocating. Sometimes. Maybe once in a blue moon, to
> > > > make it extra fun to reproduce. Maybe most driver subsystems are less
> > > > brittle, but gpu drivers definitely need to know about the details for
> > > > exactly this example. And yes gpu drivers use rcu for freeing
> > > > dma_fence structures, and that tends to happen in code that we only
> > > > recently figured out should really not allocate memory.
> > > >
> > > > I think minimally you need to throw in an unconditional
> > > > fs_reclaim_acquire();fs_reclaim_release(); so that everyone who runs
> > > > with full debugging knows what might happen. It's kinda like
> > > > might_sleep, but a lot more specific. might_sleep() alone is not
> > > > enough, because in the specific code paths I'm thinking of (and
> > > > created special lockdep annotations for just recently) sleeping is
> > > > allowed, but any memory allocations with GFP_RECLAIM set are no-go.
> > >
> > > Completely agreed!  Any allocation on any free path must be handled
> > > -extremely- carefully.  To that end...
> > >
> > > First, there is always a fallback in case the allocation fails.  Which
> > > might have performance or corner-case robustness issues, but which will
> > > at least allow forward progress.  Second, we consulted with a number of
> > > MM experts to arrive at appropriate GFP_* flags (and their patience is
> > > greatly appreciated).  Third, the paths that can allocate will do so about
> > > one time of 500, so any issues should be spotted sooner rather than later.
> > >
> > > So you are quite right to be concerned, but I believe we will be doing the
> > > right things.  And based on his previous track record, I am also quite
> > > certain that Mr. Murphy will be on hand to provide me any additional
> > > education that I might require.
> > >
> > > Finally, I have noted down your point about fs_reclaim_acquire() and
> > > fs_reclaim_release().  Whether or not they prove to be needed, I do
> > > appreciate your calling them to my attention.
> >
> > I just realized that since these dma_fence structs are refcounted and
> > userspace can hold references (directly, it can pass them around
> > behind file descriptors) we might never hit such a path until slightly
> > unusual or evil userspace does something interesting. Do you have
> > links to those patches? Some googling didn't turn up anything. I can
> > then figure out whether it's better to risk not spotting issues with
> > call_rcu vs slapping a memalloc_noio_save/restore around all these
> > critical section which force-degrades any allocation to GFP_ATOMIC at
> > most, but has the risk that we run into code that assumes "GFP_KERNEL
> > never fails for small stuff" and has a decidedly less tested fallback
> > path than rcu code.
>
> Here is the previous early draft version, which will change considerably
> for the next version:
>
>         lore.kernel.org/lkml/20200809204354.20137-1-urezki@gmail.com
>
> This does kvfree_rcu(), but we expect to handle call_rcu() similarly.
>
> The version in preparation will use workqueues to do the allocation in a
> known-safe environment and also use lockless access to certain portions
> of the allocator caches (as noted earlier, this last is not much loved
> by some of the MM guys).  Given Thomas's patch, we could with high
> probability allocate directly, perhaps even not needing memory-allocator
> modifications.
>
> Either way, kvfree_rcu(), and later call_rcu(), will avoid asking the
> allocator to do anything that the calling context prohibits.  So what
> types of bugs are you looking for?  Where reclaim calls back into the
> driver or some such?

Yeah pretty much. It's a problem for gpu, fs, block drivers and really
anything else that's remotely involved in memory reclaim somehow.
Generally this is all handled explicitly by passing gfp_t flags down
any call chain, but in some cases it's instead solved with the
memalloc_no* functions. E.g. sunrpc uses that to make sure the network
stack (which generally just assumes it can allocate memory) doesn't,
to avoid recursions back into nfs/sunrpc. To my knowledge there's no
way to check at runtime with which gfp flags you're allowed to
allocate memory, a preemptible check is definitely not enough.
Disabled preemption implies only GFP_ATOMIC is allowed (ignoring nmi
and stuff like that), but the inverse is not true.

So if you want the automagic in call_rcu I think either
- we need to replace all explicit gfp flags with the context marking
memalloc_no* across the entire kernel, or at least anywhere rcu might
be used.
- audit all callchains and make sure a call_rcu_noalloc is used
anywhere there might be a problem. probably better to have a
call_rcu_gfp with explicit gfp flags parameter, since generally that
needs to be passed down.

But at least to me the lockless magic in mm sounds a lot safer, since
it contains the complexity and doesn't leak it out to callers of
call_rcu.
-Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch


^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-17  7:52                         ` Daniel Vetter
  0 siblings, 0 replies; 282+ messages in thread
From: Daniel Vetter @ 2020-09-17  7:52 UTC (permalink / raw)
  To: Paul E. McKenney
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, Linux-MM,
	open list:KERNEL SELFTEST FRAMEWORK, linux-hexagon, Will Deacon,
	Ingo Molnar, Anton Ivanov, linux-arch, Vincent Guittot,
	Herbert Xu, Brian Cain, Richard Weinberger, Russell King,
	Ard Biesheuvel, David Airlie, Ingo Molnar, Geert Uytterhoeven,
	Mel Gorman, intel-gfx, Matt Turner, Valentin Schneider,
	linux-xtensa, Shuah Khan, Jeff Dike, linux-um, Josh Triplett,
	Steven Rostedt, rcu, linux-m68k, Ivan Kokshaysky, Rodrigo Vivi,
	Thomas Gleixner, Dietmar Eggemann, Linux ARM, Richard Henderson,
	Chris Zankel, Max Filippov, Daniel Bristot de Oliveira, LKML,
	alpha, Mathieu Desnoyers, Andrew Morton, Linus Torvalds

On Thu, Sep 17, 2020 at 12:39 AM Paul E. McKenney <paulmck@kernel.org> wrote:
>
> On Wed, Sep 16, 2020 at 11:43:02PM +0200, Daniel Vetter wrote:
> > On Wed, Sep 16, 2020 at 10:58 PM Paul E. McKenney <paulmck@kernel.org> wrote:
> > >
> > > On Wed, Sep 16, 2020 at 10:29:06PM +0200, Daniel Vetter wrote:
> > > > On Wed, Sep 16, 2020 at 5:29 PM Paul E. McKenney <paulmck@kernel.org> wrote:
> > > > >
> > > > > On Wed, Sep 16, 2020 at 09:37:17AM +0200, Daniel Vetter wrote:
> > > > > > On Tue, Sep 15, 2020 at 7:35 PM Linus Torvalds
> > > > > > <torvalds@linux-foundation.org> wrote:
> > > > > > >
> > > > > > > On Tue, Sep 15, 2020 at 1:39 AM Thomas Gleixner <tglx@linutronix.de> wrote:
> > > > > > > >
> > > > > > > > OTOH, having a working 'preemptible()' or maybe better named
> > > > > > > > 'can_schedule()' check makes tons of sense to make decisions about
> > > > > > > > allocation modes or other things.
> > > > > > >
> > > > > > > No. I think that those kinds of decisions about actual behavior are
> > > > > > > always simply fundamentally wrong.
> > > > > > >
> > > > > > > Note that this is very different from having warnings about invalid
> > > > > > > use. THAT is correct. It may not warn in all configurations, but that
> > > > > > > doesn't matter: what matters is that it warns in common enough
> > > > > > > configurations that developers will catch it.
> > > > > > >
> > > > > > > So having a warning in "might_sleep()" that doesn't always trigger,
> > > > > > > because you have a limited configuration that can't even detect the
> > > > > > > situation, that's fine and dandy and intentional.
> > > > > > >
> > > > > > > But having code like
> > > > > > >
> > > > > > >        if (can_schedule())
> > > > > > >            .. do something different ..
> > > > > > >
> > > > > > > is fundamentally complete and utter garbage.
> > > > > > >
> > > > > > > It's one thing if you test for "am I in hardware interrupt context".
> > > > > > > Those tests aren't great either, but at least they make sense.
> > > > > > >
> > > > > > > But a driver - or some library routine - making a difference based on
> > > > > > > some nebulous "can I schedule" is fundamentally and basically WRONG.
> > > > > > >
> > > > > > > If some code changes behavior, it needs to be explicit to the *caller*
> > > > > > > of that code.
> > > > > > >
> > > > > > > So this is why GFP_ATOMIC is fine, but "if (!can_schedule())
> > > > > > > do_something_atomic()" is pure shite.
> > > > > > >
> > > > > > > And I am not IN THE LEAST interested in trying to help people doing
> > > > > > > pure shite. We need to fix them. Like the crypto code is getting
> > > > > > > fixed.
> > > > > >
> > > > > > Just figured I'll throw my +1 in from reading too many (gpu) drivers.
> > > > > > Code that tries to cleverly adjust its behaviour depending upon the
> > > > > > context it's running in is harder to understand and blows up in more
> > > > > > interesting ways. We still have drm_can_sleep() and it's mostly just
> > > > > > used for debug code, and I've largely ended up just deleting
> > > > > > everything that used it because when you're driver is blowing up the
> > > > > > last thing you want is to realize your debug code and output can't be
> > > > > > relied upon. Or worse, that the only Oops you have is the one in the
> > > > > > debug code, because the real one scrolled away - the original idea
> > > > > > behind drm_can_sleep was to make all the modeset code work
> > > > > > automagically both in normal ioctl/kworker context and in the panic
> > > > > > handlers or kgdb callbacks. Wishful thinking at best.
> > > > > >
> > > > > > Also at least for me that extends to everything, e.g. I much prefer
> > > > > > explicit spin_lock and spin_lock_irq vs magic spin_lock_irqsave for
> > > > > > locks shared with interrupt handlers, since the former two gives me
> > > > > > clear information from which contexts such function can be called.
> > > > > > Other end is the memalloc_no*_save/restore functions, where I recently
> > > > > > made a real big fool of myself because I didn't realize how much that
> > > > > > impacts everything that's run within - suddenly "GFP_KERNEL for small
> > > > > > stuff never fails" is wrong everywhere.
> > > > > >
> > > > > > It's all great for debugging and sanity checks (and we run with all
> > > > > > that stuff enabled in our CI), but really semantic changes depending
> > > > > > upon magic context checks freak my out :-)
> > > > >
> > > > > All fair, but some of us need to write code that must handle being
> > > > > invoked from a wide variety of contexts.  Now perhaps you like the idea of
> > > > > call_rcu() for schedulable contexts, call_rcu_nosched() when preemption
> > > > > is disabled, call_rcu_irqs_are_disabled() when interrupts are disabled,
> > > > > call_rcu_raw_atomic() from contexts where (for example) raw spinlocks
> > > > > are held, and so on.  However, from what I can see, most people instead
> > > > > consistently prefer that the RCU API instead be consolidated.
> > > > >
> > > > > Some in-flight cache-efficiency work for kvfree_rcu() and call_rcu()
> > > > > needs to be able to allocate memory occasionally.  It can do that when
> > > > > invoked from some contexts, but not when invoked from others.  Right now,
> > > > > in !PREEMPT kernels, it cannot tell, and must either do things to the
> > > > > memory allocators that some of the MM hate or must unnecessarily invoke
> > > > > workqueues.  Thomas's patches would allow the code to just allocate in
> > > > > the common case when these primitives are invoked from contexts where
> > > > > allocation is permitted.
> > > > >
> > > > > If we want to restrict access to the can_schedule() or whatever primitive,
> > > > > fine and good.  We can add a check to checkpatch.pl, for example.  Maybe
> > > > > we can go back to the old brlock approach of requiring certain people's
> > > > > review for each addition to the kernel.
> > > > >
> > > > > But there really are use cases that it would greatly help.
> > > >
> > > > We can deadlock in random fun places if random stuff we're calling
> > > > suddenly starts allocating. Sometimes. Maybe once in a blue moon, to
> > > > make it extra fun to reproduce. Maybe most driver subsystems are less
> > > > brittle, but gpu drivers definitely need to know about the details for
> > > > exactly this example. And yes gpu drivers use rcu for freeing
> > > > dma_fence structures, and that tends to happen in code that we only
> > > > recently figured out should really not allocate memory.
> > > >
> > > > I think minimally you need to throw in an unconditional
> > > > fs_reclaim_acquire();fs_reclaim_release(); so that everyone who runs
> > > > with full debugging knows what might happen. It's kinda like
> > > > might_sleep, but a lot more specific. might_sleep() alone is not
> > > > enough, because in the specific code paths I'm thinking of (and
> > > > created special lockdep annotations for just recently) sleeping is
> > > > allowed, but any memory allocations with GFP_RECLAIM set are no-go.
> > >
> > > Completely agreed!  Any allocation on any free path must be handled
> > > -extremely- carefully.  To that end...
> > >
> > > First, there is always a fallback in case the allocation fails.  Which
> > > might have performance or corner-case robustness issues, but which will
> > > at least allow forward progress.  Second, we consulted with a number of
> > > MM experts to arrive at appropriate GFP_* flags (and their patience is
> > > greatly appreciated).  Third, the paths that can allocate will do so about
> > > one time of 500, so any issues should be spotted sooner rather than later.
> > >
> > > So you are quite right to be concerned, but I believe we will be doing the
> > > right things.  And based on his previous track record, I am also quite
> > > certain that Mr. Murphy will be on hand to provide me any additional
> > > education that I might require.
> > >
> > > Finally, I have noted down your point about fs_reclaim_acquire() and
> > > fs_reclaim_release().  Whether or not they prove to be needed, I do
> > > appreciate your calling them to my attention.
> >
> > I just realized that since these dma_fence structs are refcounted and
> > userspace can hold references (directly, it can pass them around
> > behind file descriptors) we might never hit such a path until slightly
> > unusual or evil userspace does something interesting. Do you have
> > links to those patches? Some googling didn't turn up anything. I can
> > then figure out whether it's better to risk not spotting issues with
> > call_rcu vs slapping a memalloc_noio_save/restore around all these
> > critical section which force-degrades any allocation to GFP_ATOMIC at
> > most, but has the risk that we run into code that assumes "GFP_KERNEL
> > never fails for small stuff" and has a decidedly less tested fallback
> > path than rcu code.
>
> Here is the previous early draft version, which will change considerably
> for the next version:
>
>         lore.kernel.org/lkml/20200809204354.20137-1-urezki@gmail.com
>
> This does kvfree_rcu(), but we expect to handle call_rcu() similarly.
>
> The version in preparation will use workqueues to do the allocation in a
> known-safe environment and also use lockless access to certain portions
> of the allocator caches (as noted earlier, this last is not much loved
> by some of the MM guys).  Given Thomas's patch, we could with high
> probability allocate directly, perhaps even not needing memory-allocator
> modifications.
>
> Either way, kvfree_rcu(), and later call_rcu(), will avoid asking the
> allocator to do anything that the calling context prohibits.  So what
> types of bugs are you looking for?  Where reclaim calls back into the
> driver or some such?

Yeah pretty much. It's a problem for gpu, fs, block drivers and really
anything else that's remotely involved in memory reclaim somehow.
Generally this is all handled explicitly by passing gfp_t flags down
any call chain, but in some cases it's instead solved with the
memalloc_no* functions. E.g. sunrpc uses that to make sure the network
stack (which generally just assumes it can allocate memory) doesn't,
to avoid recursions back into nfs/sunrpc. To my knowledge there's no
way to check at runtime with which gfp flags you're allowed to
allocate memory, a preemptible check is definitely not enough.
Disabled preemption implies only GFP_ATOMIC is allowed (ignoring nmi
and stuff like that), but the inverse is not true.

So if you want the automagic in call_rcu I think either
- we need to replace all explicit gfp flags with the context marking
memalloc_no* across the entire kernel, or at least anywhere rcu might
be used.
- audit all callchains and make sure a call_rcu_noalloc is used
anywhere there might be a problem. probably better to have a
call_rcu_gfp with explicit gfp flags parameter, since generally that
needs to be passed down.

But at least to me the lockless magic in mm sounds a lot safer, since
it contains the complexity and doesn't leak it out to callers of
call_rcu.
-Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [Intel-gfx] [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-17  7:52                         ` Daniel Vetter
  0 siblings, 0 replies; 282+ messages in thread
From: Daniel Vetter @ 2020-09-17  7:52 UTC (permalink / raw)
  To: Paul E. McKenney
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, Linux-MM,
	open list:KERNEL SELFTEST FRAMEWORK, linux-hexagon, Will Deacon,
	Ingo Molnar, Anton Ivanov, linux-arch, Herbert Xu, Brian Cain,
	Richard Weinberger, Russell King, Ard Biesheuvel, David Airlie,
	Ingo Molnar, Geert Uytterhoeven, Mel Gorman, intel-gfx,
	Matt Turner, Valentin Schneider, linux-xtensa, Shuah Khan,
	Jeff Dike, linux-um, Josh Triplett, Steven Rostedt, rcu,
	linux-m68k, Ivan Kokshaysky, Thomas Gleixner, Dietmar Eggemann,
	Linux ARM, Richard Henderson, Chris Zankel, Max Filippov,
	Daniel Bristot de Oliveira, LKML, alpha, Mathieu Desnoyers,
	Andrew Morton, Linus Torvalds

On Thu, Sep 17, 2020 at 12:39 AM Paul E. McKenney <paulmck@kernel.org> wrote:
>
> On Wed, Sep 16, 2020 at 11:43:02PM +0200, Daniel Vetter wrote:
> > On Wed, Sep 16, 2020 at 10:58 PM Paul E. McKenney <paulmck@kernel.org> wrote:
> > >
> > > On Wed, Sep 16, 2020 at 10:29:06PM +0200, Daniel Vetter wrote:
> > > > On Wed, Sep 16, 2020 at 5:29 PM Paul E. McKenney <paulmck@kernel.org> wrote:
> > > > >
> > > > > On Wed, Sep 16, 2020 at 09:37:17AM +0200, Daniel Vetter wrote:
> > > > > > On Tue, Sep 15, 2020 at 7:35 PM Linus Torvalds
> > > > > > <torvalds@linux-foundation.org> wrote:
> > > > > > >
> > > > > > > On Tue, Sep 15, 2020 at 1:39 AM Thomas Gleixner <tglx@linutronix.de> wrote:
> > > > > > > >
> > > > > > > > OTOH, having a working 'preemptible()' or maybe better named
> > > > > > > > 'can_schedule()' check makes tons of sense to make decisions about
> > > > > > > > allocation modes or other things.
> > > > > > >
> > > > > > > No. I think that those kinds of decisions about actual behavior are
> > > > > > > always simply fundamentally wrong.
> > > > > > >
> > > > > > > Note that this is very different from having warnings about invalid
> > > > > > > use. THAT is correct. It may not warn in all configurations, but that
> > > > > > > doesn't matter: what matters is that it warns in common enough
> > > > > > > configurations that developers will catch it.
> > > > > > >
> > > > > > > So having a warning in "might_sleep()" that doesn't always trigger,
> > > > > > > because you have a limited configuration that can't even detect the
> > > > > > > situation, that's fine and dandy and intentional.
> > > > > > >
> > > > > > > But having code like
> > > > > > >
> > > > > > >        if (can_schedule())
> > > > > > >            .. do something different ..
> > > > > > >
> > > > > > > is fundamentally complete and utter garbage.
> > > > > > >
> > > > > > > It's one thing if you test for "am I in hardware interrupt context".
> > > > > > > Those tests aren't great either, but at least they make sense.
> > > > > > >
> > > > > > > But a driver - or some library routine - making a difference based on
> > > > > > > some nebulous "can I schedule" is fundamentally and basically WRONG.
> > > > > > >
> > > > > > > If some code changes behavior, it needs to be explicit to the *caller*
> > > > > > > of that code.
> > > > > > >
> > > > > > > So this is why GFP_ATOMIC is fine, but "if (!can_schedule())
> > > > > > > do_something_atomic()" is pure shite.
> > > > > > >
> > > > > > > And I am not IN THE LEAST interested in trying to help people doing
> > > > > > > pure shite. We need to fix them. Like the crypto code is getting
> > > > > > > fixed.
> > > > > >
> > > > > > Just figured I'll throw my +1 in from reading too many (gpu) drivers.
> > > > > > Code that tries to cleverly adjust its behaviour depending upon the
> > > > > > context it's running in is harder to understand and blows up in more
> > > > > > interesting ways. We still have drm_can_sleep() and it's mostly just
> > > > > > used for debug code, and I've largely ended up just deleting
> > > > > > everything that used it because when you're driver is blowing up the
> > > > > > last thing you want is to realize your debug code and output can't be
> > > > > > relied upon. Or worse, that the only Oops you have is the one in the
> > > > > > debug code, because the real one scrolled away - the original idea
> > > > > > behind drm_can_sleep was to make all the modeset code work
> > > > > > automagically both in normal ioctl/kworker context and in the panic
> > > > > > handlers or kgdb callbacks. Wishful thinking at best.
> > > > > >
> > > > > > Also at least for me that extends to everything, e.g. I much prefer
> > > > > > explicit spin_lock and spin_lock_irq vs magic spin_lock_irqsave for
> > > > > > locks shared with interrupt handlers, since the former two gives me
> > > > > > clear information from which contexts such function can be called.
> > > > > > Other end is the memalloc_no*_save/restore functions, where I recently
> > > > > > made a real big fool of myself because I didn't realize how much that
> > > > > > impacts everything that's run within - suddenly "GFP_KERNEL for small
> > > > > > stuff never fails" is wrong everywhere.
> > > > > >
> > > > > > It's all great for debugging and sanity checks (and we run with all
> > > > > > that stuff enabled in our CI), but really semantic changes depending
> > > > > > upon magic context checks freak my out :-)
> > > > >
> > > > > All fair, but some of us need to write code that must handle being
> > > > > invoked from a wide variety of contexts.  Now perhaps you like the idea of
> > > > > call_rcu() for schedulable contexts, call_rcu_nosched() when preemption
> > > > > is disabled, call_rcu_irqs_are_disabled() when interrupts are disabled,
> > > > > call_rcu_raw_atomic() from contexts where (for example) raw spinlocks
> > > > > are held, and so on.  However, from what I can see, most people instead
> > > > > consistently prefer that the RCU API instead be consolidated.
> > > > >
> > > > > Some in-flight cache-efficiency work for kvfree_rcu() and call_rcu()
> > > > > needs to be able to allocate memory occasionally.  It can do that when
> > > > > invoked from some contexts, but not when invoked from others.  Right now,
> > > > > in !PREEMPT kernels, it cannot tell, and must either do things to the
> > > > > memory allocators that some of the MM hate or must unnecessarily invoke
> > > > > workqueues.  Thomas's patches would allow the code to just allocate in
> > > > > the common case when these primitives are invoked from contexts where
> > > > > allocation is permitted.
> > > > >
> > > > > If we want to restrict access to the can_schedule() or whatever primitive,
> > > > > fine and good.  We can add a check to checkpatch.pl, for example.  Maybe
> > > > > we can go back to the old brlock approach of requiring certain people's
> > > > > review for each addition to the kernel.
> > > > >
> > > > > But there really are use cases that it would greatly help.
> > > >
> > > > We can deadlock in random fun places if random stuff we're calling
> > > > suddenly starts allocating. Sometimes. Maybe once in a blue moon, to
> > > > make it extra fun to reproduce. Maybe most driver subsystems are less
> > > > brittle, but gpu drivers definitely need to know about the details for
> > > > exactly this example. And yes gpu drivers use rcu for freeing
> > > > dma_fence structures, and that tends to happen in code that we only
> > > > recently figured out should really not allocate memory.
> > > >
> > > > I think minimally you need to throw in an unconditional
> > > > fs_reclaim_acquire();fs_reclaim_release(); so that everyone who runs
> > > > with full debugging knows what might happen. It's kinda like
> > > > might_sleep, but a lot more specific. might_sleep() alone is not
> > > > enough, because in the specific code paths I'm thinking of (and
> > > > created special lockdep annotations for just recently) sleeping is
> > > > allowed, but any memory allocations with GFP_RECLAIM set are no-go.
> > >
> > > Completely agreed!  Any allocation on any free path must be handled
> > > -extremely- carefully.  To that end...
> > >
> > > First, there is always a fallback in case the allocation fails.  Which
> > > might have performance or corner-case robustness issues, but which will
> > > at least allow forward progress.  Second, we consulted with a number of
> > > MM experts to arrive at appropriate GFP_* flags (and their patience is
> > > greatly appreciated).  Third, the paths that can allocate will do so about
> > > one time of 500, so any issues should be spotted sooner rather than later.
> > >
> > > So you are quite right to be concerned, but I believe we will be doing the
> > > right things.  And based on his previous track record, I am also quite
> > > certain that Mr. Murphy will be on hand to provide me any additional
> > > education that I might require.
> > >
> > > Finally, I have noted down your point about fs_reclaim_acquire() and
> > > fs_reclaim_release().  Whether or not they prove to be needed, I do
> > > appreciate your calling them to my attention.
> >
> > I just realized that since these dma_fence structs are refcounted and
> > userspace can hold references (directly, it can pass them around
> > behind file descriptors) we might never hit such a path until slightly
> > unusual or evil userspace does something interesting. Do you have
> > links to those patches? Some googling didn't turn up anything. I can
> > then figure out whether it's better to risk not spotting issues with
> > call_rcu vs slapping a memalloc_noio_save/restore around all these
> > critical section which force-degrades any allocation to GFP_ATOMIC at
> > most, but has the risk that we run into code that assumes "GFP_KERNEL
> > never fails for small stuff" and has a decidedly less tested fallback
> > path than rcu code.
>
> Here is the previous early draft version, which will change considerably
> for the next version:
>
>         lore.kernel.org/lkml/20200809204354.20137-1-urezki@gmail.com
>
> This does kvfree_rcu(), but we expect to handle call_rcu() similarly.
>
> The version in preparation will use workqueues to do the allocation in a
> known-safe environment and also use lockless access to certain portions
> of the allocator caches (as noted earlier, this last is not much loved
> by some of the MM guys).  Given Thomas's patch, we could with high
> probability allocate directly, perhaps even not needing memory-allocator
> modifications.
>
> Either way, kvfree_rcu(), and later call_rcu(), will avoid asking the
> allocator to do anything that the calling context prohibits.  So what
> types of bugs are you looking for?  Where reclaim calls back into the
> driver or some such?

Yeah pretty much. It's a problem for gpu, fs, block drivers and really
anything else that's remotely involved in memory reclaim somehow.
Generally this is all handled explicitly by passing gfp_t flags down
any call chain, but in some cases it's instead solved with the
memalloc_no* functions. E.g. sunrpc uses that to make sure the network
stack (which generally just assumes it can allocate memory) doesn't,
to avoid recursions back into nfs/sunrpc. To my knowledge there's no
way to check at runtime with which gfp flags you're allowed to
allocate memory, a preemptible check is definitely not enough.
Disabled preemption implies only GFP_ATOMIC is allowed (ignoring nmi
and stuff like that), but the inverse is not true.

So if you want the automagic in call_rcu I think either
- we need to replace all explicit gfp flags with the context marking
memalloc_no* across the entire kernel, or at least anywhere rcu might
be used.
- audit all callchains and make sure a call_rcu_noalloc is used
anywhere there might be a problem. probably better to have a
call_rcu_gfp with explicit gfp flags parameter, since generally that
needs to be passed down.

But at least to me the lockless magic in mm sounds a lot safer, since
it contains the complexity and doesn't leak it out to callers of
call_rcu.
-Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-17  7:52                         ` Daniel Vetter
  0 siblings, 0 replies; 282+ messages in thread
From: Daniel Vetter @ 2020-09-17  7:52 UTC (permalink / raw)
  To: Paul E. McKenney
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, Linux-MM,
	open list:KERNEL SELFTEST FRAMEWORK, linux-hexagon, Will Deacon,
	Ingo Molnar, Anton Ivanov, linux-arch, Herbert Xu, Brian Cain,
	Richard Weinberger, Russell King, Ard Biesheuvel, David Airlie,
	Ingo Molnar, Geert Uytterhoeven, Mel Gorman, intel-gfx

On Thu, Sep 17, 2020 at 12:39 AM Paul E. McKenney <paulmck@kernel.org> wrote:
>
> On Wed, Sep 16, 2020 at 11:43:02PM +0200, Daniel Vetter wrote:
> > On Wed, Sep 16, 2020 at 10:58 PM Paul E. McKenney <paulmck@kernel.org> wrote:
> > >
> > > On Wed, Sep 16, 2020 at 10:29:06PM +0200, Daniel Vetter wrote:
> > > > On Wed, Sep 16, 2020 at 5:29 PM Paul E. McKenney <paulmck@kernel.org> wrote:
> > > > >
> > > > > On Wed, Sep 16, 2020 at 09:37:17AM +0200, Daniel Vetter wrote:
> > > > > > On Tue, Sep 15, 2020 at 7:35 PM Linus Torvalds
> > > > > > <torvalds@linux-foundation.org> wrote:
> > > > > > >
> > > > > > > On Tue, Sep 15, 2020 at 1:39 AM Thomas Gleixner <tglx@linutronix.de> wrote:
> > > > > > > >
> > > > > > > > OTOH, having a working 'preemptible()' or maybe better named
> > > > > > > > 'can_schedule()' check makes tons of sense to make decisions about
> > > > > > > > allocation modes or other things.
> > > > > > >
> > > > > > > No. I think that those kinds of decisions about actual behavior are
> > > > > > > always simply fundamentally wrong.
> > > > > > >
> > > > > > > Note that this is very different from having warnings about invalid
> > > > > > > use. THAT is correct. It may not warn in all configurations, but that
> > > > > > > doesn't matter: what matters is that it warns in common enough
> > > > > > > configurations that developers will catch it.
> > > > > > >
> > > > > > > So having a warning in "might_sleep()" that doesn't always trigger,
> > > > > > > because you have a limited configuration that can't even detect the
> > > > > > > situation, that's fine and dandy and intentional.
> > > > > > >
> > > > > > > But having code like
> > > > > > >
> > > > > > >        if (can_schedule())
> > > > > > >            .. do something different ..
> > > > > > >
> > > > > > > is fundamentally complete and utter garbage.
> > > > > > >
> > > > > > > It's one thing if you test for "am I in hardware interrupt context".
> > > > > > > Those tests aren't great either, but at least they make sense.
> > > > > > >
> > > > > > > But a driver - or some library routine - making a difference based on
> > > > > > > some nebulous "can I schedule" is fundamentally and basically WRONG.
> > > > > > >
> > > > > > > If some code changes behavior, it needs to be explicit to the *caller*
> > > > > > > of that code.
> > > > > > >
> > > > > > > So this is why GFP_ATOMIC is fine, but "if (!can_schedule())
> > > > > > > do_something_atomic()" is pure shite.
> > > > > > >
> > > > > > > And I am not IN THE LEAST interested in trying to help people doing
> > > > > > > pure shite. We need to fix them. Like the crypto code is getting
> > > > > > > fixed.
> > > > > >
> > > > > > Just figured I'll throw my +1 in from reading too many (gpu) drivers.
> > > > > > Code that tries to cleverly adjust its behaviour depending upon the
> > > > > > context it's running in is harder to understand and blows up in more
> > > > > > interesting ways. We still have drm_can_sleep() and it's mostly just
> > > > > > used for debug code, and I've largely ended up just deleting
> > > > > > everything that used it because when you're driver is blowing up the
> > > > > > last thing you want is to realize your debug code and output can't be
> > > > > > relied upon. Or worse, that the only Oops you have is the one in the
> > > > > > debug code, because the real one scrolled away - the original idea
> > > > > > behind drm_can_sleep was to make all the modeset code work
> > > > > > automagically both in normal ioctl/kworker context and in the panic
> > > > > > handlers or kgdb callbacks. Wishful thinking at best.
> > > > > >
> > > > > > Also at least for me that extends to everything, e.g. I much prefer
> > > > > > explicit spin_lock and spin_lock_irq vs magic spin_lock_irqsave for
> > > > > > locks shared with interrupt handlers, since the former two gives me
> > > > > > clear information from which contexts such function can be called.
> > > > > > Other end is the memalloc_no*_save/restore functions, where I recently
> > > > > > made a real big fool of myself because I didn't realize how much that
> > > > > > impacts everything that's run within - suddenly "GFP_KERNEL for small
> > > > > > stuff never fails" is wrong everywhere.
> > > > > >
> > > > > > It's all great for debugging and sanity checks (and we run with all
> > > > > > that stuff enabled in our CI), but really semantic changes depending
> > > > > > upon magic context checks freak my out :-)
> > > > >
> > > > > All fair, but some of us need to write code that must handle being
> > > > > invoked from a wide variety of contexts.  Now perhaps you like the idea of
> > > > > call_rcu() for schedulable contexts, call_rcu_nosched() when preemption
> > > > > is disabled, call_rcu_irqs_are_disabled() when interrupts are disabled,
> > > > > call_rcu_raw_atomic() from contexts where (for example) raw spinlocks
> > > > > are held, and so on.  However, from what I can see, most people instead
> > > > > consistently prefer that the RCU API instead be consolidated.
> > > > >
> > > > > Some in-flight cache-efficiency work for kvfree_rcu() and call_rcu()
> > > > > needs to be able to allocate memory occasionally.  It can do that when
> > > > > invoked from some contexts, but not when invoked from others.  Right now,
> > > > > in !PREEMPT kernels, it cannot tell, and must either do things to the
> > > > > memory allocators that some of the MM hate or must unnecessarily invoke
> > > > > workqueues.  Thomas's patches would allow the code to just allocate in
> > > > > the common case when these primitives are invoked from contexts where
> > > > > allocation is permitted.
> > > > >
> > > > > If we want to restrict access to the can_schedule() or whatever primitive,
> > > > > fine and good.  We can add a check to checkpatch.pl, for example.  Maybe
> > > > > we can go back to the old brlock approach of requiring certain people's
> > > > > review for each addition to the kernel.
> > > > >
> > > > > But there really are use cases that it would greatly help.
> > > >
> > > > We can deadlock in random fun places if random stuff we're calling
> > > > suddenly starts allocating. Sometimes. Maybe once in a blue moon, to
> > > > make it extra fun to reproduce. Maybe most driver subsystems are less
> > > > brittle, but gpu drivers definitely need to know about the details for
> > > > exactly this example. And yes gpu drivers use rcu for freeing
> > > > dma_fence structures, and that tends to happen in code that we only
> > > > recently figured out should really not allocate memory.
> > > >
> > > > I think minimally you need to throw in an unconditional
> > > > fs_reclaim_acquire();fs_reclaim_release(); so that everyone who runs
> > > > with full debugging knows what might happen. It's kinda like
> > > > might_sleep, but a lot more specific. might_sleep() alone is not
> > > > enough, because in the specific code paths I'm thinking of (and
> > > > created special lockdep annotations for just recently) sleeping is
> > > > allowed, but any memory allocations with GFP_RECLAIM set are no-go.
> > >
> > > Completely agreed!  Any allocation on any free path must be handled
> > > -extremely- carefully.  To that end...
> > >
> > > First, there is always a fallback in case the allocation fails.  Which
> > > might have performance or corner-case robustness issues, but which will
> > > at least allow forward progress.  Second, we consulted with a number of
> > > MM experts to arrive at appropriate GFP_* flags (and their patience is
> > > greatly appreciated).  Third, the paths that can allocate will do so about
> > > one time of 500, so any issues should be spotted sooner rather than later.
> > >
> > > So you are quite right to be concerned, but I believe we will be doing the
> > > right things.  And based on his previous track record, I am also quite
> > > certain that Mr. Murphy will be on hand to provide me any additional
> > > education that I might require.
> > >
> > > Finally, I have noted down your point about fs_reclaim_acquire() and
> > > fs_reclaim_release().  Whether or not they prove to be needed, I do
> > > appreciate your calling them to my attention.
> >
> > I just realized that since these dma_fence structs are refcounted and
> > userspace can hold references (directly, it can pass them around
> > behind file descriptors) we might never hit such a path until slightly
> > unusual or evil userspace does something interesting. Do you have
> > links to those patches? Some googling didn't turn up anything. I can
> > then figure out whether it's better to risk not spotting issues with
> > call_rcu vs slapping a memalloc_noio_save/restore around all these
> > critical section which force-degrades any allocation to GFP_ATOMIC at
> > most, but has the risk that we run into code that assumes "GFP_KERNEL
> > never fails for small stuff" and has a decidedly less tested fallback
> > path than rcu code.
>
> Here is the previous early draft version, which will change considerably
> for the next version:
>
>         lore.kernel.org/lkml/20200809204354.20137-1-urezki@gmail.com
>
> This does kvfree_rcu(), but we expect to handle call_rcu() similarly.
>
> The version in preparation will use workqueues to do the allocation in a
> known-safe environment and also use lockless access to certain portions
> of the allocator caches (as noted earlier, this last is not much loved
> by some of the MM guys).  Given Thomas's patch, we could with high
> probability allocate directly, perhaps even not needing memory-allocator
> modifications.
>
> Either way, kvfree_rcu(), and later call_rcu(), will avoid asking the
> allocator to do anything that the calling context prohibits.  So what
> types of bugs are you looking for?  Where reclaim calls back into the
> driver or some such?

Yeah pretty much. It's a problem for gpu, fs, block drivers and really
anything else that's remotely involved in memory reclaim somehow.
Generally this is all handled explicitly by passing gfp_t flags down
any call chain, but in some cases it's instead solved with the
memalloc_no* functions. E.g. sunrpc uses that to make sure the network
stack (which generally just assumes it can allocate memory) doesn't,
to avoid recursions back into nfs/sunrpc. To my knowledge there's no
way to check at runtime with which gfp flags you're allowed to
allocate memory, a preemptible check is definitely not enough.
Disabled preemption implies only GFP_ATOMIC is allowed (ignoring nmi
and stuff like that), but the inverse is not true.

So if you want the automagic in call_rcu I think either
- we need to replace all explicit gfp flags with the context marking
memalloc_no* across the entire kernel, or at least anywhere rcu might
be used.
- audit all callchains and make sure a call_rcu_noalloc is used
anywhere there might be a problem. probably better to have a
call_rcu_gfp with explicit gfp flags parameter, since generally that
needs to be passed down.

But at least to me the lockless magic in mm sounds a lot safer, since
it contains the complexity and doesn't leak it out to callers of
call_rcu.
-Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
  2020-09-17  7:52                         ` Daniel Vetter
                                             ` (2 preceding siblings ...)
  (?)
@ 2020-09-17 16:28                           ` Paul E. McKenney
  -1 siblings, 0 replies; 282+ messages in thread
From: Paul E. McKenney @ 2020-09-17 16:28 UTC (permalink / raw)
  To: Daniel Vetter
  Cc: Linus Torvalds, Thomas Gleixner, Ard Biesheuvel, Herbert Xu,
	LKML, linux-arch, Sebastian Andrzej Siewior, Valentin Schneider,
	Richard Henderson, Ivan Kokshaysky, Matt Turner, alpha,
	Jeff Dike, Richard Weinberger, Anton Ivanov, linux-um,
	Brian Cain, linux-hexagon, Geert Uytterhoeven, linux-m68k,
	Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
	Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman,
	Daniel Bristot de Oliveira, Will Deacon, Andrew Morton, Linux-MM,
	Ingo Molnar, Russell King, Linux ARM, Chris Zankel, Max Filippov,
	linux-xtensa, Jani Nikula, Joonas Lahtinen, Rodrigo Vivi,
	David Airlie, intel-gfx, dri-devel, Josh Triplett,
	Mathieu Desnoyers, Lai Jiangshan, Shuah Khan, rcu,
	open list:KERNEL SELFTEST FRAMEWORK

On Thu, Sep 17, 2020 at 09:52:30AM +0200, Daniel Vetter wrote:
> On Thu, Sep 17, 2020 at 12:39 AM Paul E. McKenney <paulmck@kernel.org> wrote:
> >
> > On Wed, Sep 16, 2020 at 11:43:02PM +0200, Daniel Vetter wrote:
> > > On Wed, Sep 16, 2020 at 10:58 PM Paul E. McKenney <paulmck@kernel.org> wrote:
> > > >
> > > > On Wed, Sep 16, 2020 at 10:29:06PM +0200, Daniel Vetter wrote:
> > > > > On Wed, Sep 16, 2020 at 5:29 PM Paul E. McKenney <paulmck@kernel.org> wrote:
> > > > > >
> > > > > > On Wed, Sep 16, 2020 at 09:37:17AM +0200, Daniel Vetter wrote:
> > > > > > > On Tue, Sep 15, 2020 at 7:35 PM Linus Torvalds
> > > > > > > <torvalds@linux-foundation.org> wrote:
> > > > > > > >
> > > > > > > > On Tue, Sep 15, 2020 at 1:39 AM Thomas Gleixner <tglx@linutronix.de> wrote:
> > > > > > > > >
> > > > > > > > > OTOH, having a working 'preemptible()' or maybe better named
> > > > > > > > > 'can_schedule()' check makes tons of sense to make decisions about
> > > > > > > > > allocation modes or other things.
> > > > > > > >
> > > > > > > > No. I think that those kinds of decisions about actual behavior are
> > > > > > > > always simply fundamentally wrong.
> > > > > > > >
> > > > > > > > Note that this is very different from having warnings about invalid
> > > > > > > > use. THAT is correct. It may not warn in all configurations, but that
> > > > > > > > doesn't matter: what matters is that it warns in common enough
> > > > > > > > configurations that developers will catch it.
> > > > > > > >
> > > > > > > > So having a warning in "might_sleep()" that doesn't always trigger,
> > > > > > > > because you have a limited configuration that can't even detect the
> > > > > > > > situation, that's fine and dandy and intentional.
> > > > > > > >
> > > > > > > > But having code like
> > > > > > > >
> > > > > > > >        if (can_schedule())
> > > > > > > >            .. do something different ..
> > > > > > > >
> > > > > > > > is fundamentally complete and utter garbage.
> > > > > > > >
> > > > > > > > It's one thing if you test for "am I in hardware interrupt context".
> > > > > > > > Those tests aren't great either, but at least they make sense.
> > > > > > > >
> > > > > > > > But a driver - or some library routine - making a difference based on
> > > > > > > > some nebulous "can I schedule" is fundamentally and basically WRONG.
> > > > > > > >
> > > > > > > > If some code changes behavior, it needs to be explicit to the *caller*
> > > > > > > > of that code.
> > > > > > > >
> > > > > > > > So this is why GFP_ATOMIC is fine, but "if (!can_schedule())
> > > > > > > > do_something_atomic()" is pure shite.
> > > > > > > >
> > > > > > > > And I am not IN THE LEAST interested in trying to help people doing
> > > > > > > > pure shite. We need to fix them. Like the crypto code is getting
> > > > > > > > fixed.
> > > > > > >
> > > > > > > Just figured I'll throw my +1 in from reading too many (gpu) drivers.
> > > > > > > Code that tries to cleverly adjust its behaviour depending upon the
> > > > > > > context it's running in is harder to understand and blows up in more
> > > > > > > interesting ways. We still have drm_can_sleep() and it's mostly just
> > > > > > > used for debug code, and I've largely ended up just deleting
> > > > > > > everything that used it because when you're driver is blowing up the
> > > > > > > last thing you want is to realize your debug code and output can't be
> > > > > > > relied upon. Or worse, that the only Oops you have is the one in the
> > > > > > > debug code, because the real one scrolled away - the original idea
> > > > > > > behind drm_can_sleep was to make all the modeset code work
> > > > > > > automagically both in normal ioctl/kworker context and in the panic
> > > > > > > handlers or kgdb callbacks. Wishful thinking at best.
> > > > > > >
> > > > > > > Also at least for me that extends to everything, e.g. I much prefer
> > > > > > > explicit spin_lock and spin_lock_irq vs magic spin_lock_irqsave for
> > > > > > > locks shared with interrupt handlers, since the former two gives me
> > > > > > > clear information from which contexts such function can be called.
> > > > > > > Other end is the memalloc_no*_save/restore functions, where I recently
> > > > > > > made a real big fool of myself because I didn't realize how much that
> > > > > > > impacts everything that's run within - suddenly "GFP_KERNEL for small
> > > > > > > stuff never fails" is wrong everywhere.
> > > > > > >
> > > > > > > It's all great for debugging and sanity checks (and we run with all
> > > > > > > that stuff enabled in our CI), but really semantic changes depending
> > > > > > > upon magic context checks freak my out :-)
> > > > > >
> > > > > > All fair, but some of us need to write code that must handle being
> > > > > > invoked from a wide variety of contexts.  Now perhaps you like the idea of
> > > > > > call_rcu() for schedulable contexts, call_rcu_nosched() when preemption
> > > > > > is disabled, call_rcu_irqs_are_disabled() when interrupts are disabled,
> > > > > > call_rcu_raw_atomic() from contexts where (for example) raw spinlocks
> > > > > > are held, and so on.  However, from what I can see, most people instead
> > > > > > consistently prefer that the RCU API instead be consolidated.
> > > > > >
> > > > > > Some in-flight cache-efficiency work for kvfree_rcu() and call_rcu()
> > > > > > needs to be able to allocate memory occasionally.  It can do that when
> > > > > > invoked from some contexts, but not when invoked from others.  Right now,
> > > > > > in !PREEMPT kernels, it cannot tell, and must either do things to the
> > > > > > memory allocators that some of the MM hate or must unnecessarily invoke
> > > > > > workqueues.  Thomas's patches would allow the code to just allocate in
> > > > > > the common case when these primitives are invoked from contexts where
> > > > > > allocation is permitted.
> > > > > >
> > > > > > If we want to restrict access to the can_schedule() or whatever primitive,
> > > > > > fine and good.  We can add a check to checkpatch.pl, for example.  Maybe
> > > > > > we can go back to the old brlock approach of requiring certain people's
> > > > > > review for each addition to the kernel.
> > > > > >
> > > > > > But there really are use cases that it would greatly help.
> > > > >
> > > > > We can deadlock in random fun places if random stuff we're calling
> > > > > suddenly starts allocating. Sometimes. Maybe once in a blue moon, to
> > > > > make it extra fun to reproduce. Maybe most driver subsystems are less
> > > > > brittle, but gpu drivers definitely need to know about the details for
> > > > > exactly this example. And yes gpu drivers use rcu for freeing
> > > > > dma_fence structures, and that tends to happen in code that we only
> > > > > recently figured out should really not allocate memory.
> > > > >
> > > > > I think minimally you need to throw in an unconditional
> > > > > fs_reclaim_acquire();fs_reclaim_release(); so that everyone who runs
> > > > > with full debugging knows what might happen. It's kinda like
> > > > > might_sleep, but a lot more specific. might_sleep() alone is not
> > > > > enough, because in the specific code paths I'm thinking of (and
> > > > > created special lockdep annotations for just recently) sleeping is
> > > > > allowed, but any memory allocations with GFP_RECLAIM set are no-go.
> > > >
> > > > Completely agreed!  Any allocation on any free path must be handled
> > > > -extremely- carefully.  To that end...
> > > >
> > > > First, there is always a fallback in case the allocation fails.  Which
> > > > might have performance or corner-case robustness issues, but which will
> > > > at least allow forward progress.  Second, we consulted with a number of
> > > > MM experts to arrive at appropriate GFP_* flags (and their patience is
> > > > greatly appreciated).  Third, the paths that can allocate will do so about
> > > > one time of 500, so any issues should be spotted sooner rather than later.
> > > >
> > > > So you are quite right to be concerned, but I believe we will be doing the
> > > > right things.  And based on his previous track record, I am also quite
> > > > certain that Mr. Murphy will be on hand to provide me any additional
> > > > education that I might require.
> > > >
> > > > Finally, I have noted down your point about fs_reclaim_acquire() and
> > > > fs_reclaim_release().  Whether or not they prove to be needed, I do
> > > > appreciate your calling them to my attention.
> > >
> > > I just realized that since these dma_fence structs are refcounted and
> > > userspace can hold references (directly, it can pass them around
> > > behind file descriptors) we might never hit such a path until slightly
> > > unusual or evil userspace does something interesting. Do you have
> > > links to those patches? Some googling didn't turn up anything. I can
> > > then figure out whether it's better to risk not spotting issues with
> > > call_rcu vs slapping a memalloc_noio_save/restore around all these
> > > critical section which force-degrades any allocation to GFP_ATOMIC at
> > > most, but has the risk that we run into code that assumes "GFP_KERNEL
> > > never fails for small stuff" and has a decidedly less tested fallback
> > > path than rcu code.
> >
> > Here is the previous early draft version, which will change considerably
> > for the next version:
> >
> >         lore.kernel.org/lkml/20200809204354.20137-1-urezki@gmail.com
> >
> > This does kvfree_rcu(), but we expect to handle call_rcu() similarly.
> >
> > The version in preparation will use workqueues to do the allocation in a
> > known-safe environment and also use lockless access to certain portions
> > of the allocator caches (as noted earlier, this last is not much loved
> > by some of the MM guys).  Given Thomas's patch, we could with high
> > probability allocate directly, perhaps even not needing memory-allocator
> > modifications.
> >
> > Either way, kvfree_rcu(), and later call_rcu(), will avoid asking the
> > allocator to do anything that the calling context prohibits.  So what
> > types of bugs are you looking for?  Where reclaim calls back into the
> > driver or some such?
> 
> Yeah pretty much. It's a problem for gpu, fs, block drivers and really
> anything else that's remotely involved in memory reclaim somehow.
> Generally this is all handled explicitly by passing gfp_t flags down
> any call chain, but in some cases it's instead solved with the
> memalloc_no* functions. E.g. sunrpc uses that to make sure the network
> stack (which generally just assumes it can allocate memory) doesn't,
> to avoid recursions back into nfs/sunrpc. To my knowledge there's no
> way to check at runtime with which gfp flags you're allowed to
> allocate memory, a preemptible check is definitely not enough.
> Disabled preemption implies only GFP_ATOMIC is allowed (ignoring nmi
> and stuff like that), but the inverse is not true.

Thank you for the confirmation!

> So if you want the automagic in call_rcu I think either
> - we need to replace all explicit gfp flags with the context marking
> memalloc_no* across the entire kernel, or at least anywhere rcu might
> be used.
> - audit all callchains and make sure a call_rcu_noalloc is used
> anywhere there might be a problem. probably better to have a
> call_rcu_gfp with explicit gfp flags parameter, since generally that
> needs to be passed down.
> 
> But at least to me the lockless magic in mm sounds a lot safer, since
> it contains the complexity and doesn't leak it out to callers of
> call_rcu.

Agreed, I greatly prefer Peter Zijlstra's lockless-allocation patch
myself.

In the meantime, it looks like we will start by causing the allocation to
happen in a safe environment.  That may have issues with delays, but is
at least something that can be done entirely within the confines of RCU.

							Thanx, Paul

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-17 16:28                           ` Paul E. McKenney
  0 siblings, 0 replies; 282+ messages in thread
From: Paul E. McKenney @ 2020-09-17 16:28 UTC (permalink / raw)
  To: Daniel Vetter
  Cc: Linus Torvalds, Thomas Gleixner, Ard Biesheuvel, Herbert Xu,
	LKML, linux-arch, Sebastian Andrzej Siewior, Valentin Schneider,
	Richard Henderson, Ivan Kokshaysky, Matt Turner, alpha,
	Jeff Dike, Richard Weinberger, Anton Ivanov, linux-um,
	Brian Cain, linux-hexagon, Geert Uytterhoeven, linux-m68k,
	Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
	Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman,
	Daniel Bristot de Oliveira, Will Deacon, Andrew Morton, Linux-MM,
	Ingo Molnar, Russell King, Linux ARM, Chris Zankel, Max Filippov,
	linux-xtensa, Jani Nikula, Joonas Lahtinen, Rodrigo Vivi,
	David Airlie, intel-gfx, dri-devel, Josh Triplett,
	Mathieu Desnoyers, Lai Jiangshan, Shuah Khan, rcu,
	open list:KERNEL SELFTEST FRAMEWORK

On Thu, Sep 17, 2020 at 09:52:30AM +0200, Daniel Vetter wrote:
> On Thu, Sep 17, 2020 at 12:39 AM Paul E. McKenney <paulmck@kernel.org> wrote:
> >
> > On Wed, Sep 16, 2020 at 11:43:02PM +0200, Daniel Vetter wrote:
> > > On Wed, Sep 16, 2020 at 10:58 PM Paul E. McKenney <paulmck@kernel.org> wrote:
> > > >
> > > > On Wed, Sep 16, 2020 at 10:29:06PM +0200, Daniel Vetter wrote:
> > > > > On Wed, Sep 16, 2020 at 5:29 PM Paul E. McKenney <paulmck@kernel.org> wrote:
> > > > > >
> > > > > > On Wed, Sep 16, 2020 at 09:37:17AM +0200, Daniel Vetter wrote:
> > > > > > > On Tue, Sep 15, 2020 at 7:35 PM Linus Torvalds
> > > > > > > <torvalds@linux-foundation.org> wrote:
> > > > > > > >
> > > > > > > > On Tue, Sep 15, 2020 at 1:39 AM Thomas Gleixner <tglx@linutronix.de> wrote:
> > > > > > > > >
> > > > > > > > > OTOH, having a working 'preemptible()' or maybe better named
> > > > > > > > > 'can_schedule()' check makes tons of sense to make decisions about
> > > > > > > > > allocation modes or other things.
> > > > > > > >
> > > > > > > > No. I think that those kinds of decisions about actual behavior are
> > > > > > > > always simply fundamentally wrong.
> > > > > > > >
> > > > > > > > Note that this is very different from having warnings about invalid
> > > > > > > > use. THAT is correct. It may not warn in all configurations, but that
> > > > > > > > doesn't matter: what matters is that it warns in common enough
> > > > > > > > configurations that developers will catch it.
> > > > > > > >
> > > > > > > > So having a warning in "might_sleep()" that doesn't always trigger,
> > > > > > > > because you have a limited configuration that can't even detect the
> > > > > > > > situation, that's fine and dandy and intentional.
> > > > > > > >
> > > > > > > > But having code like
> > > > > > > >
> > > > > > > >        if (can_schedule())
> > > > > > > >            .. do something different ..
> > > > > > > >
> > > > > > > > is fundamentally complete and utter garbage.
> > > > > > > >
> > > > > > > > It's one thing if you test for "am I in hardware interrupt context".
> > > > > > > > Those tests aren't great either, but at least they make sense.
> > > > > > > >
> > > > > > > > But a driver - or some library routine - making a difference based on
> > > > > > > > some nebulous "can I schedule" is fundamentally and basically WRONG.
> > > > > > > >
> > > > > > > > If some code changes behavior, it needs to be explicit to the *caller*
> > > > > > > > of that code.
> > > > > > > >
> > > > > > > > So this is why GFP_ATOMIC is fine, but "if (!can_schedule())
> > > > > > > > do_something_atomic()" is pure shite.
> > > > > > > >
> > > > > > > > And I am not IN THE LEAST interested in trying to help people doing
> > > > > > > > pure shite. We need to fix them. Like the crypto code is getting
> > > > > > > > fixed.
> > > > > > >
> > > > > > > Just figured I'll throw my +1 in from reading too many (gpu) drivers.
> > > > > > > Code that tries to cleverly adjust its behaviour depending upon the
> > > > > > > context it's running in is harder to understand and blows up in more
> > > > > > > interesting ways. We still have drm_can_sleep() and it's mostly just
> > > > > > > used for debug code, and I've largely ended up just deleting
> > > > > > > everything that used it because when you're driver is blowing up the
> > > > > > > last thing you want is to realize your debug code and output can't be
> > > > > > > relied upon. Or worse, that the only Oops you have is the one in the
> > > > > > > debug code, because the real one scrolled away - the original idea
> > > > > > > behind drm_can_sleep was to make all the modeset code work
> > > > > > > automagically both in normal ioctl/kworker context and in the panic
> > > > > > > handlers or kgdb callbacks. Wishful thinking at best.
> > > > > > >
> > > > > > > Also at least for me that extends to everything, e.g. I much prefer
> > > > > > > explicit spin_lock and spin_lock_irq vs magic spin_lock_irqsave for
> > > > > > > locks shared with interrupt handlers, since the former two gives me
> > > > > > > clear information from which contexts such function can be called.
> > > > > > > Other end is the memalloc_no*_save/restore functions, where I recently
> > > > > > > made a real big fool of myself because I didn't realize how much that
> > > > > > > impacts everything that's run within - suddenly "GFP_KERNEL for small
> > > > > > > stuff never fails" is wrong everywhere.
> > > > > > >
> > > > > > > It's all great for debugging and sanity checks (and we run with all
> > > > > > > that stuff enabled in our CI), but really semantic changes depending
> > > > > > > upon magic context checks freak my out :-)
> > > > > >
> > > > > > All fair, but some of us need to write code that must handle being
> > > > > > invoked from a wide variety of contexts.  Now perhaps you like the idea of
> > > > > > call_rcu() for schedulable contexts, call_rcu_nosched() when preemption
> > > > > > is disabled, call_rcu_irqs_are_disabled() when interrupts are disabled,
> > > > > > call_rcu_raw_atomic() from contexts where (for example) raw spinlocks
> > > > > > are held, and so on.  However, from what I can see, most people instead
> > > > > > consistently prefer that the RCU API instead be consolidated.
> > > > > >
> > > > > > Some in-flight cache-efficiency work for kvfree_rcu() and call_rcu()
> > > > > > needs to be able to allocate memory occasionally.  It can do that when
> > > > > > invoked from some contexts, but not when invoked from others.  Right now,
> > > > > > in !PREEMPT kernels, it cannot tell, and must either do things to the
> > > > > > memory allocators that some of the MM hate or must unnecessarily invoke
> > > > > > workqueues.  Thomas's patches would allow the code to just allocate in
> > > > > > the common case when these primitives are invoked from contexts where
> > > > > > allocation is permitted.
> > > > > >
> > > > > > If we want to restrict access to the can_schedule() or whatever primitive,
> > > > > > fine and good.  We can add a check to checkpatch.pl, for example.  Maybe
> > > > > > we can go back to the old brlock approach of requiring certain people's
> > > > > > review for each addition to the kernel.
> > > > > >
> > > > > > But there really are use cases that it would greatly help.
> > > > >
> > > > > We can deadlock in random fun places if random stuff we're calling
> > > > > suddenly starts allocating. Sometimes. Maybe once in a blue moon, to
> > > > > make it extra fun to reproduce. Maybe most driver subsystems are less
> > > > > brittle, but gpu drivers definitely need to know about the details for
> > > > > exactly this example. And yes gpu drivers use rcu for freeing
> > > > > dma_fence structures, and that tends to happen in code that we only
> > > > > recently figured out should really not allocate memory.
> > > > >
> > > > > I think minimally you need to throw in an unconditional
> > > > > fs_reclaim_acquire();fs_reclaim_release(); so that everyone who runs
> > > > > with full debugging knows what might happen. It's kinda like
> > > > > might_sleep, but a lot more specific. might_sleep() alone is not
> > > > > enough, because in the specific code paths I'm thinking of (and
> > > > > created special lockdep annotations for just recently) sleeping is
> > > > > allowed, but any memory allocations with GFP_RECLAIM set are no-go.
> > > >
> > > > Completely agreed!  Any allocation on any free path must be handled
> > > > -extremely- carefully.  To that end...
> > > >
> > > > First, there is always a fallback in case the allocation fails.  Which
> > > > might have performance or corner-case robustness issues, but which will
> > > > at least allow forward progress.  Second, we consulted with a number of
> > > > MM experts to arrive at appropriate GFP_* flags (and their patience is
> > > > greatly appreciated).  Third, the paths that can allocate will do so about
> > > > one time of 500, so any issues should be spotted sooner rather than later.
> > > >
> > > > So you are quite right to be concerned, but I believe we will be doing the
> > > > right things.  And based on his previous track record, I am also quite
> > > > certain that Mr. Murphy will be on hand to provide me any additional
> > > > education that I might require.
> > > >
> > > > Finally, I have noted down your point about fs_reclaim_acquire() and
> > > > fs_reclaim_release().  Whether or not they prove to be needed, I do
> > > > appreciate your calling them to my attention.
> > >
> > > I just realized that since these dma_fence structs are refcounted and
> > > userspace can hold references (directly, it can pass them around
> > > behind file descriptors) we might never hit such a path until slightly
> > > unusual or evil userspace does something interesting. Do you have
> > > links to those patches? Some googling didn't turn up anything. I can
> > > then figure out whether it's better to risk not spotting issues with
> > > call_rcu vs slapping a memalloc_noio_save/restore around all these
> > > critical section which force-degrades any allocation to GFP_ATOMIC at
> > > most, but has the risk that we run into code that assumes "GFP_KERNEL
> > > never fails for small stuff" and has a decidedly less tested fallback
> > > path than rcu code.
> >
> > Here is the previous early draft version, which will change considerably
> > for the next version:
> >
> >         lore.kernel.org/lkml/20200809204354.20137-1-urezki@gmail.com
> >
> > This does kvfree_rcu(), but we expect to handle call_rcu() similarly.
> >
> > The version in preparation will use workqueues to do the allocation in a
> > known-safe environment and also use lockless access to certain portions
> > of the allocator caches (as noted earlier, this last is not much loved
> > by some of the MM guys).  Given Thomas's patch, we could with high
> > probability allocate directly, perhaps even not needing memory-allocator
> > modifications.
> >
> > Either way, kvfree_rcu(), and later call_rcu(), will avoid asking the
> > allocator to do anything that the calling context prohibits.  So what
> > types of bugs are you looking for?  Where reclaim calls back into the
> > driver or some such?
> 
> Yeah pretty much. It's a problem for gpu, fs, block drivers and really
> anything else that's remotely involved in memory reclaim somehow.
> Generally this is all handled explicitly by passing gfp_t flags down
> any call chain, but in some cases it's instead solved with the
> memalloc_no* functions. E.g. sunrpc uses that to make sure the network
> stack (which generally just assumes it can allocate memory) doesn't,
> to avoid recursions back into nfs/sunrpc. To my knowledge there's no
> way to check at runtime with which gfp flags you're allowed to
> allocate memory, a preemptible check is definitely not enough.
> Disabled preemption implies only GFP_ATOMIC is allowed (ignoring nmi
> and stuff like that), but the inverse is not true.

Thank you for the confirmation!

> So if you want the automagic in call_rcu I think either
> - we need to replace all explicit gfp flags with the context marking
> memalloc_no* across the entire kernel, or at least anywhere rcu might
> be used.
> - audit all callchains and make sure a call_rcu_noalloc is used
> anywhere there might be a problem. probably better to have a
> call_rcu_gfp with explicit gfp flags parameter, since generally that
> needs to be passed down.
> 
> But at least to me the lockless magic in mm sounds a lot safer, since
> it contains the complexity and doesn't leak it out to callers of
> call_rcu.

Agreed, I greatly prefer Peter Zijlstra's lockless-allocation patch
myself.

In the meantime, it looks like we will start by causing the allocation to
happen in a safe environment.  That may have issues with delays, but is
at least something that can be done entirely within the confines of RCU.

							Thanx, Paul


^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-17 16:28                           ` Paul E. McKenney
  0 siblings, 0 replies; 282+ messages in thread
From: Paul E. McKenney @ 2020-09-17 16:28 UTC (permalink / raw)
  To: Daniel Vetter
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, Linux-MM,
	open list:KERNEL SELFTEST FRAMEWORK, linux-hexagon, Will Deacon,
	Ingo Molnar, Anton Ivanov, linux-arch, Vincent Guittot,
	Herbert Xu, Brian Cain, Richard Weinberger, Russell King,
	Ard Biesheuvel, David Airlie, Ingo Molnar, Geert Uytterhoeven,
	Mel Gorman, intel-gfx, Matt Turner, Valentin Schneider,
	linux-xtensa, Shuah Khan, Jeff Dike, linux-um, Josh Triplett,
	Steven Rostedt, rcu, linux-m68k, Ivan Kokshaysky, Rodrigo Vivi,
	Thomas Gleixner, Dietmar Eggemann, Linux ARM, Richard Henderson,
	Chris Zankel, Max Filippov, Daniel Bristot de Oliveira, LKML,
	alpha, Mathieu Desnoyers, Andrew Morton, Linus Torvalds

On Thu, Sep 17, 2020 at 09:52:30AM +0200, Daniel Vetter wrote:
> On Thu, Sep 17, 2020 at 12:39 AM Paul E. McKenney <paulmck@kernel.org> wrote:
> >
> > On Wed, Sep 16, 2020 at 11:43:02PM +0200, Daniel Vetter wrote:
> > > On Wed, Sep 16, 2020 at 10:58 PM Paul E. McKenney <paulmck@kernel.org> wrote:
> > > >
> > > > On Wed, Sep 16, 2020 at 10:29:06PM +0200, Daniel Vetter wrote:
> > > > > On Wed, Sep 16, 2020 at 5:29 PM Paul E. McKenney <paulmck@kernel.org> wrote:
> > > > > >
> > > > > > On Wed, Sep 16, 2020 at 09:37:17AM +0200, Daniel Vetter wrote:
> > > > > > > On Tue, Sep 15, 2020 at 7:35 PM Linus Torvalds
> > > > > > > <torvalds@linux-foundation.org> wrote:
> > > > > > > >
> > > > > > > > On Tue, Sep 15, 2020 at 1:39 AM Thomas Gleixner <tglx@linutronix.de> wrote:
> > > > > > > > >
> > > > > > > > > OTOH, having a working 'preemptible()' or maybe better named
> > > > > > > > > 'can_schedule()' check makes tons of sense to make decisions about
> > > > > > > > > allocation modes or other things.
> > > > > > > >
> > > > > > > > No. I think that those kinds of decisions about actual behavior are
> > > > > > > > always simply fundamentally wrong.
> > > > > > > >
> > > > > > > > Note that this is very different from having warnings about invalid
> > > > > > > > use. THAT is correct. It may not warn in all configurations, but that
> > > > > > > > doesn't matter: what matters is that it warns in common enough
> > > > > > > > configurations that developers will catch it.
> > > > > > > >
> > > > > > > > So having a warning in "might_sleep()" that doesn't always trigger,
> > > > > > > > because you have a limited configuration that can't even detect the
> > > > > > > > situation, that's fine and dandy and intentional.
> > > > > > > >
> > > > > > > > But having code like
> > > > > > > >
> > > > > > > >        if (can_schedule())
> > > > > > > >            .. do something different ..
> > > > > > > >
> > > > > > > > is fundamentally complete and utter garbage.
> > > > > > > >
> > > > > > > > It's one thing if you test for "am I in hardware interrupt context".
> > > > > > > > Those tests aren't great either, but at least they make sense.
> > > > > > > >
> > > > > > > > But a driver - or some library routine - making a difference based on
> > > > > > > > some nebulous "can I schedule" is fundamentally and basically WRONG.
> > > > > > > >
> > > > > > > > If some code changes behavior, it needs to be explicit to the *caller*
> > > > > > > > of that code.
> > > > > > > >
> > > > > > > > So this is why GFP_ATOMIC is fine, but "if (!can_schedule())
> > > > > > > > do_something_atomic()" is pure shite.
> > > > > > > >
> > > > > > > > And I am not IN THE LEAST interested in trying to help people doing
> > > > > > > > pure shite. We need to fix them. Like the crypto code is getting
> > > > > > > > fixed.
> > > > > > >
> > > > > > > Just figured I'll throw my +1 in from reading too many (gpu) drivers.
> > > > > > > Code that tries to cleverly adjust its behaviour depending upon the
> > > > > > > context it's running in is harder to understand and blows up in more
> > > > > > > interesting ways. We still have drm_can_sleep() and it's mostly just
> > > > > > > used for debug code, and I've largely ended up just deleting
> > > > > > > everything that used it because when you're driver is blowing up the
> > > > > > > last thing you want is to realize your debug code and output can't be
> > > > > > > relied upon. Or worse, that the only Oops you have is the one in the
> > > > > > > debug code, because the real one scrolled away - the original idea
> > > > > > > behind drm_can_sleep was to make all the modeset code work
> > > > > > > automagically both in normal ioctl/kworker context and in the panic
> > > > > > > handlers or kgdb callbacks. Wishful thinking at best.
> > > > > > >
> > > > > > > Also at least for me that extends to everything, e.g. I much prefer
> > > > > > > explicit spin_lock and spin_lock_irq vs magic spin_lock_irqsave for
> > > > > > > locks shared with interrupt handlers, since the former two gives me
> > > > > > > clear information from which contexts such function can be called.
> > > > > > > Other end is the memalloc_no*_save/restore functions, where I recently
> > > > > > > made a real big fool of myself because I didn't realize how much that
> > > > > > > impacts everything that's run within - suddenly "GFP_KERNEL for small
> > > > > > > stuff never fails" is wrong everywhere.
> > > > > > >
> > > > > > > It's all great for debugging and sanity checks (and we run with all
> > > > > > > that stuff enabled in our CI), but really semantic changes depending
> > > > > > > upon magic context checks freak my out :-)
> > > > > >
> > > > > > All fair, but some of us need to write code that must handle being
> > > > > > invoked from a wide variety of contexts.  Now perhaps you like the idea of
> > > > > > call_rcu() for schedulable contexts, call_rcu_nosched() when preemption
> > > > > > is disabled, call_rcu_irqs_are_disabled() when interrupts are disabled,
> > > > > > call_rcu_raw_atomic() from contexts where (for example) raw spinlocks
> > > > > > are held, and so on.  However, from what I can see, most people instead
> > > > > > consistently prefer that the RCU API instead be consolidated.
> > > > > >
> > > > > > Some in-flight cache-efficiency work for kvfree_rcu() and call_rcu()
> > > > > > needs to be able to allocate memory occasionally.  It can do that when
> > > > > > invoked from some contexts, but not when invoked from others.  Right now,
> > > > > > in !PREEMPT kernels, it cannot tell, and must either do things to the
> > > > > > memory allocators that some of the MM hate or must unnecessarily invoke
> > > > > > workqueues.  Thomas's patches would allow the code to just allocate in
> > > > > > the common case when these primitives are invoked from contexts where
> > > > > > allocation is permitted.
> > > > > >
> > > > > > If we want to restrict access to the can_schedule() or whatever primitive,
> > > > > > fine and good.  We can add a check to checkpatch.pl, for example.  Maybe
> > > > > > we can go back to the old brlock approach of requiring certain people's
> > > > > > review for each addition to the kernel.
> > > > > >
> > > > > > But there really are use cases that it would greatly help.
> > > > >
> > > > > We can deadlock in random fun places if random stuff we're calling
> > > > > suddenly starts allocating. Sometimes. Maybe once in a blue moon, to
> > > > > make it extra fun to reproduce. Maybe most driver subsystems are less
> > > > > brittle, but gpu drivers definitely need to know about the details for
> > > > > exactly this example. And yes gpu drivers use rcu for freeing
> > > > > dma_fence structures, and that tends to happen in code that we only
> > > > > recently figured out should really not allocate memory.
> > > > >
> > > > > I think minimally you need to throw in an unconditional
> > > > > fs_reclaim_acquire();fs_reclaim_release(); so that everyone who runs
> > > > > with full debugging knows what might happen. It's kinda like
> > > > > might_sleep, but a lot more specific. might_sleep() alone is not
> > > > > enough, because in the specific code paths I'm thinking of (and
> > > > > created special lockdep annotations for just recently) sleeping is
> > > > > allowed, but any memory allocations with GFP_RECLAIM set are no-go.
> > > >
> > > > Completely agreed!  Any allocation on any free path must be handled
> > > > -extremely- carefully.  To that end...
> > > >
> > > > First, there is always a fallback in case the allocation fails.  Which
> > > > might have performance or corner-case robustness issues, but which will
> > > > at least allow forward progress.  Second, we consulted with a number of
> > > > MM experts to arrive at appropriate GFP_* flags (and their patience is
> > > > greatly appreciated).  Third, the paths that can allocate will do so about
> > > > one time of 500, so any issues should be spotted sooner rather than later.
> > > >
> > > > So you are quite right to be concerned, but I believe we will be doing the
> > > > right things.  And based on his previous track record, I am also quite
> > > > certain that Mr. Murphy will be on hand to provide me any additional
> > > > education that I might require.
> > > >
> > > > Finally, I have noted down your point about fs_reclaim_acquire() and
> > > > fs_reclaim_release().  Whether or not they prove to be needed, I do
> > > > appreciate your calling them to my attention.
> > >
> > > I just realized that since these dma_fence structs are refcounted and
> > > userspace can hold references (directly, it can pass them around
> > > behind file descriptors) we might never hit such a path until slightly
> > > unusual or evil userspace does something interesting. Do you have
> > > links to those patches? Some googling didn't turn up anything. I can
> > > then figure out whether it's better to risk not spotting issues with
> > > call_rcu vs slapping a memalloc_noio_save/restore around all these
> > > critical section which force-degrades any allocation to GFP_ATOMIC at
> > > most, but has the risk that we run into code that assumes "GFP_KERNEL
> > > never fails for small stuff" and has a decidedly less tested fallback
> > > path than rcu code.
> >
> > Here is the previous early draft version, which will change considerably
> > for the next version:
> >
> >         lore.kernel.org/lkml/20200809204354.20137-1-urezki@gmail.com
> >
> > This does kvfree_rcu(), but we expect to handle call_rcu() similarly.
> >
> > The version in preparation will use workqueues to do the allocation in a
> > known-safe environment and also use lockless access to certain portions
> > of the allocator caches (as noted earlier, this last is not much loved
> > by some of the MM guys).  Given Thomas's patch, we could with high
> > probability allocate directly, perhaps even not needing memory-allocator
> > modifications.
> >
> > Either way, kvfree_rcu(), and later call_rcu(), will avoid asking the
> > allocator to do anything that the calling context prohibits.  So what
> > types of bugs are you looking for?  Where reclaim calls back into the
> > driver or some such?
> 
> Yeah pretty much. It's a problem for gpu, fs, block drivers and really
> anything else that's remotely involved in memory reclaim somehow.
> Generally this is all handled explicitly by passing gfp_t flags down
> any call chain, but in some cases it's instead solved with the
> memalloc_no* functions. E.g. sunrpc uses that to make sure the network
> stack (which generally just assumes it can allocate memory) doesn't,
> to avoid recursions back into nfs/sunrpc. To my knowledge there's no
> way to check at runtime with which gfp flags you're allowed to
> allocate memory, a preemptible check is definitely not enough.
> Disabled preemption implies only GFP_ATOMIC is allowed (ignoring nmi
> and stuff like that), but the inverse is not true.

Thank you for the confirmation!

> So if you want the automagic in call_rcu I think either
> - we need to replace all explicit gfp flags with the context marking
> memalloc_no* across the entire kernel, or at least anywhere rcu might
> be used.
> - audit all callchains and make sure a call_rcu_noalloc is used
> anywhere there might be a problem. probably better to have a
> call_rcu_gfp with explicit gfp flags parameter, since generally that
> needs to be passed down.
> 
> But at least to me the lockless magic in mm sounds a lot safer, since
> it contains the complexity and doesn't leak it out to callers of
> call_rcu.

Agreed, I greatly prefer Peter Zijlstra's lockless-allocation patch
myself.

In the meantime, it looks like we will start by causing the allocation to
happen in a safe environment.  That may have issues with delays, but is
at least something that can be done entirely within the confines of RCU.

							Thanx, Paul
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [Intel-gfx] [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-17 16:28                           ` Paul E. McKenney
  0 siblings, 0 replies; 282+ messages in thread
From: Paul E. McKenney @ 2020-09-17 16:28 UTC (permalink / raw)
  To: Daniel Vetter
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, Linux-MM,
	open list:KERNEL SELFTEST FRAMEWORK, linux-hexagon, Will Deacon,
	Ingo Molnar, Anton Ivanov, linux-arch, Herbert Xu, Brian Cain,
	Richard Weinberger, Russell King, Ard Biesheuvel, David Airlie,
	Ingo Molnar, Geert Uytterhoeven, Mel Gorman, intel-gfx,
	Matt Turner, Valentin Schneider, linux-xtensa, Shuah Khan,
	Jeff Dike, linux-um, Josh Triplett, Steven Rostedt, rcu,
	linux-m68k, Ivan Kokshaysky, Thomas Gleixner, Dietmar Eggemann,
	Linux ARM, Richard Henderson, Chris Zankel, Max Filippov,
	Daniel Bristot de Oliveira, LKML, alpha, Mathieu Desnoyers,
	Andrew Morton, Linus Torvalds

On Thu, Sep 17, 2020 at 09:52:30AM +0200, Daniel Vetter wrote:
> On Thu, Sep 17, 2020 at 12:39 AM Paul E. McKenney <paulmck@kernel.org> wrote:
> >
> > On Wed, Sep 16, 2020 at 11:43:02PM +0200, Daniel Vetter wrote:
> > > On Wed, Sep 16, 2020 at 10:58 PM Paul E. McKenney <paulmck@kernel.org> wrote:
> > > >
> > > > On Wed, Sep 16, 2020 at 10:29:06PM +0200, Daniel Vetter wrote:
> > > > > On Wed, Sep 16, 2020 at 5:29 PM Paul E. McKenney <paulmck@kernel.org> wrote:
> > > > > >
> > > > > > On Wed, Sep 16, 2020 at 09:37:17AM +0200, Daniel Vetter wrote:
> > > > > > > On Tue, Sep 15, 2020 at 7:35 PM Linus Torvalds
> > > > > > > <torvalds@linux-foundation.org> wrote:
> > > > > > > >
> > > > > > > > On Tue, Sep 15, 2020 at 1:39 AM Thomas Gleixner <tglx@linutronix.de> wrote:
> > > > > > > > >
> > > > > > > > > OTOH, having a working 'preemptible()' or maybe better named
> > > > > > > > > 'can_schedule()' check makes tons of sense to make decisions about
> > > > > > > > > allocation modes or other things.
> > > > > > > >
> > > > > > > > No. I think that those kinds of decisions about actual behavior are
> > > > > > > > always simply fundamentally wrong.
> > > > > > > >
> > > > > > > > Note that this is very different from having warnings about invalid
> > > > > > > > use. THAT is correct. It may not warn in all configurations, but that
> > > > > > > > doesn't matter: what matters is that it warns in common enough
> > > > > > > > configurations that developers will catch it.
> > > > > > > >
> > > > > > > > So having a warning in "might_sleep()" that doesn't always trigger,
> > > > > > > > because you have a limited configuration that can't even detect the
> > > > > > > > situation, that's fine and dandy and intentional.
> > > > > > > >
> > > > > > > > But having code like
> > > > > > > >
> > > > > > > >        if (can_schedule())
> > > > > > > >            .. do something different ..
> > > > > > > >
> > > > > > > > is fundamentally complete and utter garbage.
> > > > > > > >
> > > > > > > > It's one thing if you test for "am I in hardware interrupt context".
> > > > > > > > Those tests aren't great either, but at least they make sense.
> > > > > > > >
> > > > > > > > But a driver - or some library routine - making a difference based on
> > > > > > > > some nebulous "can I schedule" is fundamentally and basically WRONG.
> > > > > > > >
> > > > > > > > If some code changes behavior, it needs to be explicit to the *caller*
> > > > > > > > of that code.
> > > > > > > >
> > > > > > > > So this is why GFP_ATOMIC is fine, but "if (!can_schedule())
> > > > > > > > do_something_atomic()" is pure shite.
> > > > > > > >
> > > > > > > > And I am not IN THE LEAST interested in trying to help people doing
> > > > > > > > pure shite. We need to fix them. Like the crypto code is getting
> > > > > > > > fixed.
> > > > > > >
> > > > > > > Just figured I'll throw my +1 in from reading too many (gpu) drivers.
> > > > > > > Code that tries to cleverly adjust its behaviour depending upon the
> > > > > > > context it's running in is harder to understand and blows up in more
> > > > > > > interesting ways. We still have drm_can_sleep() and it's mostly just
> > > > > > > used for debug code, and I've largely ended up just deleting
> > > > > > > everything that used it because when you're driver is blowing up the
> > > > > > > last thing you want is to realize your debug code and output can't be
> > > > > > > relied upon. Or worse, that the only Oops you have is the one in the
> > > > > > > debug code, because the real one scrolled away - the original idea
> > > > > > > behind drm_can_sleep was to make all the modeset code work
> > > > > > > automagically both in normal ioctl/kworker context and in the panic
> > > > > > > handlers or kgdb callbacks. Wishful thinking at best.
> > > > > > >
> > > > > > > Also at least for me that extends to everything, e.g. I much prefer
> > > > > > > explicit spin_lock and spin_lock_irq vs magic spin_lock_irqsave for
> > > > > > > locks shared with interrupt handlers, since the former two gives me
> > > > > > > clear information from which contexts such function can be called.
> > > > > > > Other end is the memalloc_no*_save/restore functions, where I recently
> > > > > > > made a real big fool of myself because I didn't realize how much that
> > > > > > > impacts everything that's run within - suddenly "GFP_KERNEL for small
> > > > > > > stuff never fails" is wrong everywhere.
> > > > > > >
> > > > > > > It's all great for debugging and sanity checks (and we run with all
> > > > > > > that stuff enabled in our CI), but really semantic changes depending
> > > > > > > upon magic context checks freak my out :-)
> > > > > >
> > > > > > All fair, but some of us need to write code that must handle being
> > > > > > invoked from a wide variety of contexts.  Now perhaps you like the idea of
> > > > > > call_rcu() for schedulable contexts, call_rcu_nosched() when preemption
> > > > > > is disabled, call_rcu_irqs_are_disabled() when interrupts are disabled,
> > > > > > call_rcu_raw_atomic() from contexts where (for example) raw spinlocks
> > > > > > are held, and so on.  However, from what I can see, most people instead
> > > > > > consistently prefer that the RCU API instead be consolidated.
> > > > > >
> > > > > > Some in-flight cache-efficiency work for kvfree_rcu() and call_rcu()
> > > > > > needs to be able to allocate memory occasionally.  It can do that when
> > > > > > invoked from some contexts, but not when invoked from others.  Right now,
> > > > > > in !PREEMPT kernels, it cannot tell, and must either do things to the
> > > > > > memory allocators that some of the MM hate or must unnecessarily invoke
> > > > > > workqueues.  Thomas's patches would allow the code to just allocate in
> > > > > > the common case when these primitives are invoked from contexts where
> > > > > > allocation is permitted.
> > > > > >
> > > > > > If we want to restrict access to the can_schedule() or whatever primitive,
> > > > > > fine and good.  We can add a check to checkpatch.pl, for example.  Maybe
> > > > > > we can go back to the old brlock approach of requiring certain people's
> > > > > > review for each addition to the kernel.
> > > > > >
> > > > > > But there really are use cases that it would greatly help.
> > > > >
> > > > > We can deadlock in random fun places if random stuff we're calling
> > > > > suddenly starts allocating. Sometimes. Maybe once in a blue moon, to
> > > > > make it extra fun to reproduce. Maybe most driver subsystems are less
> > > > > brittle, but gpu drivers definitely need to know about the details for
> > > > > exactly this example. And yes gpu drivers use rcu for freeing
> > > > > dma_fence structures, and that tends to happen in code that we only
> > > > > recently figured out should really not allocate memory.
> > > > >
> > > > > I think minimally you need to throw in an unconditional
> > > > > fs_reclaim_acquire();fs_reclaim_release(); so that everyone who runs
> > > > > with full debugging knows what might happen. It's kinda like
> > > > > might_sleep, but a lot more specific. might_sleep() alone is not
> > > > > enough, because in the specific code paths I'm thinking of (and
> > > > > created special lockdep annotations for just recently) sleeping is
> > > > > allowed, but any memory allocations with GFP_RECLAIM set are no-go.
> > > >
> > > > Completely agreed!  Any allocation on any free path must be handled
> > > > -extremely- carefully.  To that end...
> > > >
> > > > First, there is always a fallback in case the allocation fails.  Which
> > > > might have performance or corner-case robustness issues, but which will
> > > > at least allow forward progress.  Second, we consulted with a number of
> > > > MM experts to arrive at appropriate GFP_* flags (and their patience is
> > > > greatly appreciated).  Third, the paths that can allocate will do so about
> > > > one time of 500, so any issues should be spotted sooner rather than later.
> > > >
> > > > So you are quite right to be concerned, but I believe we will be doing the
> > > > right things.  And based on his previous track record, I am also quite
> > > > certain that Mr. Murphy will be on hand to provide me any additional
> > > > education that I might require.
> > > >
> > > > Finally, I have noted down your point about fs_reclaim_acquire() and
> > > > fs_reclaim_release().  Whether or not they prove to be needed, I do
> > > > appreciate your calling them to my attention.
> > >
> > > I just realized that since these dma_fence structs are refcounted and
> > > userspace can hold references (directly, it can pass them around
> > > behind file descriptors) we might never hit such a path until slightly
> > > unusual or evil userspace does something interesting. Do you have
> > > links to those patches? Some googling didn't turn up anything. I can
> > > then figure out whether it's better to risk not spotting issues with
> > > call_rcu vs slapping a memalloc_noio_save/restore around all these
> > > critical section which force-degrades any allocation to GFP_ATOMIC at
> > > most, but has the risk that we run into code that assumes "GFP_KERNEL
> > > never fails for small stuff" and has a decidedly less tested fallback
> > > path than rcu code.
> >
> > Here is the previous early draft version, which will change considerably
> > for the next version:
> >
> >         lore.kernel.org/lkml/20200809204354.20137-1-urezki@gmail.com
> >
> > This does kvfree_rcu(), but we expect to handle call_rcu() similarly.
> >
> > The version in preparation will use workqueues to do the allocation in a
> > known-safe environment and also use lockless access to certain portions
> > of the allocator caches (as noted earlier, this last is not much loved
> > by some of the MM guys).  Given Thomas's patch, we could with high
> > probability allocate directly, perhaps even not needing memory-allocator
> > modifications.
> >
> > Either way, kvfree_rcu(), and later call_rcu(), will avoid asking the
> > allocator to do anything that the calling context prohibits.  So what
> > types of bugs are you looking for?  Where reclaim calls back into the
> > driver or some such?
> 
> Yeah pretty much. It's a problem for gpu, fs, block drivers and really
> anything else that's remotely involved in memory reclaim somehow.
> Generally this is all handled explicitly by passing gfp_t flags down
> any call chain, but in some cases it's instead solved with the
> memalloc_no* functions. E.g. sunrpc uses that to make sure the network
> stack (which generally just assumes it can allocate memory) doesn't,
> to avoid recursions back into nfs/sunrpc. To my knowledge there's no
> way to check at runtime with which gfp flags you're allowed to
> allocate memory, a preemptible check is definitely not enough.
> Disabled preemption implies only GFP_ATOMIC is allowed (ignoring nmi
> and stuff like that), but the inverse is not true.

Thank you for the confirmation!

> So if you want the automagic in call_rcu I think either
> - we need to replace all explicit gfp flags with the context marking
> memalloc_no* across the entire kernel, or at least anywhere rcu might
> be used.
> - audit all callchains and make sure a call_rcu_noalloc is used
> anywhere there might be a problem. probably better to have a
> call_rcu_gfp with explicit gfp flags parameter, since generally that
> needs to be passed down.
> 
> But at least to me the lockless magic in mm sounds a lot safer, since
> it contains the complexity and doesn't leak it out to callers of
> call_rcu.

Agreed, I greatly prefer Peter Zijlstra's lockless-allocation patch
myself.

In the meantime, it looks like we will start by causing the allocation to
happen in a safe environment.  That may have issues with delays, but is
at least something that can be done entirely within the confines of RCU.

							Thanx, Paul
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-17 16:28                           ` Paul E. McKenney
  0 siblings, 0 replies; 282+ messages in thread
From: Paul E. McKenney @ 2020-09-17 16:28 UTC (permalink / raw)
  To: Daniel Vetter
  Cc: Linus Torvalds, Thomas Gleixner, Ard Biesheuvel, Herbert Xu,
	LKML, linux-arch, Sebastian Andrzej Siewior, Valentin Schneider,
	Richard Henderson, Ivan Kokshaysky, Matt Turner, alpha,
	Jeff Dike, Richard Weinberger, Anton Ivanov, linux-um,
	Brian Cain, linux-hexagon, Geert Uytterhoeven

On Thu, Sep 17, 2020 at 09:52:30AM +0200, Daniel Vetter wrote:
> On Thu, Sep 17, 2020 at 12:39 AM Paul E. McKenney <paulmck@kernel.org> wrote:
> >
> > On Wed, Sep 16, 2020 at 11:43:02PM +0200, Daniel Vetter wrote:
> > > On Wed, Sep 16, 2020 at 10:58 PM Paul E. McKenney <paulmck@kernel.org> wrote:
> > > >
> > > > On Wed, Sep 16, 2020 at 10:29:06PM +0200, Daniel Vetter wrote:
> > > > > On Wed, Sep 16, 2020 at 5:29 PM Paul E. McKenney <paulmck@kernel.org> wrote:
> > > > > >
> > > > > > On Wed, Sep 16, 2020 at 09:37:17AM +0200, Daniel Vetter wrote:
> > > > > > > On Tue, Sep 15, 2020 at 7:35 PM Linus Torvalds
> > > > > > > <torvalds@linux-foundation.org> wrote:
> > > > > > > >
> > > > > > > > On Tue, Sep 15, 2020 at 1:39 AM Thomas Gleixner <tglx@linutronix.de> wrote:
> > > > > > > > >
> > > > > > > > > OTOH, having a working 'preemptible()' or maybe better named
> > > > > > > > > 'can_schedule()' check makes tons of sense to make decisions about
> > > > > > > > > allocation modes or other things.
> > > > > > > >
> > > > > > > > No. I think that those kinds of decisions about actual behavior are
> > > > > > > > always simply fundamentally wrong.
> > > > > > > >
> > > > > > > > Note that this is very different from having warnings about invalid
> > > > > > > > use. THAT is correct. It may not warn in all configurations, but that
> > > > > > > > doesn't matter: what matters is that it warns in common enough
> > > > > > > > configurations that developers will catch it.
> > > > > > > >
> > > > > > > > So having a warning in "might_sleep()" that doesn't always trigger,
> > > > > > > > because you have a limited configuration that can't even detect the
> > > > > > > > situation, that's fine and dandy and intentional.
> > > > > > > >
> > > > > > > > But having code like
> > > > > > > >
> > > > > > > >        if (can_schedule())
> > > > > > > >            .. do something different ..
> > > > > > > >
> > > > > > > > is fundamentally complete and utter garbage.
> > > > > > > >
> > > > > > > > It's one thing if you test for "am I in hardware interrupt context".
> > > > > > > > Those tests aren't great either, but at least they make sense.
> > > > > > > >
> > > > > > > > But a driver - or some library routine - making a difference based on
> > > > > > > > some nebulous "can I schedule" is fundamentally and basically WRONG.
> > > > > > > >
> > > > > > > > If some code changes behavior, it needs to be explicit to the *caller*
> > > > > > > > of that code.
> > > > > > > >
> > > > > > > > So this is why GFP_ATOMIC is fine, but "if (!can_schedule())
> > > > > > > > do_something_atomic()" is pure shite.
> > > > > > > >
> > > > > > > > And I am not IN THE LEAST interested in trying to help people doing
> > > > > > > > pure shite. We need to fix them. Like the crypto code is getting
> > > > > > > > fixed.
> > > > > > >
> > > > > > > Just figured I'll throw my +1 in from reading too many (gpu) drivers.
> > > > > > > Code that tries to cleverly adjust its behaviour depending upon the
> > > > > > > context it's running in is harder to understand and blows up in more
> > > > > > > interesting ways. We still have drm_can_sleep() and it's mostly just
> > > > > > > used for debug code, and I've largely ended up just deleting
> > > > > > > everything that used it because when you're driver is blowing up the
> > > > > > > last thing you want is to realize your debug code and output can't be
> > > > > > > relied upon. Or worse, that the only Oops you have is the one in the
> > > > > > > debug code, because the real one scrolled away - the original idea
> > > > > > > behind drm_can_sleep was to make all the modeset code work
> > > > > > > automagically both in normal ioctl/kworker context and in the panic
> > > > > > > handlers or kgdb callbacks. Wishful thinking at best.
> > > > > > >
> > > > > > > Also at least for me that extends to everything, e.g. I much prefer
> > > > > > > explicit spin_lock and spin_lock_irq vs magic spin_lock_irqsave for
> > > > > > > locks shared with interrupt handlers, since the former two gives me
> > > > > > > clear information from which contexts such function can be called.
> > > > > > > Other end is the memalloc_no*_save/restore functions, where I recently
> > > > > > > made a real big fool of myself because I didn't realize how much that
> > > > > > > impacts everything that's run within - suddenly "GFP_KERNEL for small
> > > > > > > stuff never fails" is wrong everywhere.
> > > > > > >
> > > > > > > It's all great for debugging and sanity checks (and we run with all
> > > > > > > that stuff enabled in our CI), but really semantic changes depending
> > > > > > > upon magic context checks freak my out :-)
> > > > > >
> > > > > > All fair, but some of us need to write code that must handle being
> > > > > > invoked from a wide variety of contexts.  Now perhaps you like the idea of
> > > > > > call_rcu() for schedulable contexts, call_rcu_nosched() when preemption
> > > > > > is disabled, call_rcu_irqs_are_disabled() when interrupts are disabled,
> > > > > > call_rcu_raw_atomic() from contexts where (for example) raw spinlocks
> > > > > > are held, and so on.  However, from what I can see, most people instead
> > > > > > consistently prefer that the RCU API instead be consolidated.
> > > > > >
> > > > > > Some in-flight cache-efficiency work for kvfree_rcu() and call_rcu()
> > > > > > needs to be able to allocate memory occasionally.  It can do that when
> > > > > > invoked from some contexts, but not when invoked from others.  Right now,
> > > > > > in !PREEMPT kernels, it cannot tell, and must either do things to the
> > > > > > memory allocators that some of the MM hate or must unnecessarily invoke
> > > > > > workqueues.  Thomas's patches would allow the code to just allocate in
> > > > > > the common case when these primitives are invoked from contexts where
> > > > > > allocation is permitted.
> > > > > >
> > > > > > If we want to restrict access to the can_schedule() or whatever primitive,
> > > > > > fine and good.  We can add a check to checkpatch.pl, for example.  Maybe
> > > > > > we can go back to the old brlock approach of requiring certain people's
> > > > > > review for each addition to the kernel.
> > > > > >
> > > > > > But there really are use cases that it would greatly help.
> > > > >
> > > > > We can deadlock in random fun places if random stuff we're calling
> > > > > suddenly starts allocating. Sometimes. Maybe once in a blue moon, to
> > > > > make it extra fun to reproduce. Maybe most driver subsystems are less
> > > > > brittle, but gpu drivers definitely need to know about the details for
> > > > > exactly this example. And yes gpu drivers use rcu for freeing
> > > > > dma_fence structures, and that tends to happen in code that we only
> > > > > recently figured out should really not allocate memory.
> > > > >
> > > > > I think minimally you need to throw in an unconditional
> > > > > fs_reclaim_acquire();fs_reclaim_release(); so that everyone who runs
> > > > > with full debugging knows what might happen. It's kinda like
> > > > > might_sleep, but a lot more specific. might_sleep() alone is not
> > > > > enough, because in the specific code paths I'm thinking of (and
> > > > > created special lockdep annotations for just recently) sleeping is
> > > > > allowed, but any memory allocations with GFP_RECLAIM set are no-go.
> > > >
> > > > Completely agreed!  Any allocation on any free path must be handled
> > > > -extremely- carefully.  To that end...
> > > >
> > > > First, there is always a fallback in case the allocation fails.  Which
> > > > might have performance or corner-case robustness issues, but which will
> > > > at least allow forward progress.  Second, we consulted with a number of
> > > > MM experts to arrive at appropriate GFP_* flags (and their patience is
> > > > greatly appreciated).  Third, the paths that can allocate will do so about
> > > > one time of 500, so any issues should be spotted sooner rather than later.
> > > >
> > > > So you are quite right to be concerned, but I believe we will be doing the
> > > > right things.  And based on his previous track record, I am also quite
> > > > certain that Mr. Murphy will be on hand to provide me any additional
> > > > education that I might require.
> > > >
> > > > Finally, I have noted down your point about fs_reclaim_acquire() and
> > > > fs_reclaim_release().  Whether or not they prove to be needed, I do
> > > > appreciate your calling them to my attention.
> > >
> > > I just realized that since these dma_fence structs are refcounted and
> > > userspace can hold references (directly, it can pass them around
> > > behind file descriptors) we might never hit such a path until slightly
> > > unusual or evil userspace does something interesting. Do you have
> > > links to those patches? Some googling didn't turn up anything. I can
> > > then figure out whether it's better to risk not spotting issues with
> > > call_rcu vs slapping a memalloc_noio_save/restore around all these
> > > critical section which force-degrades any allocation to GFP_ATOMIC at
> > > most, but has the risk that we run into code that assumes "GFP_KERNEL
> > > never fails for small stuff" and has a decidedly less tested fallback
> > > path than rcu code.
> >
> > Here is the previous early draft version, which will change considerably
> > for the next version:
> >
> >         lore.kernel.org/lkml/20200809204354.20137-1-urezki@gmail.com
> >
> > This does kvfree_rcu(), but we expect to handle call_rcu() similarly.
> >
> > The version in preparation will use workqueues to do the allocation in a
> > known-safe environment and also use lockless access to certain portions
> > of the allocator caches (as noted earlier, this last is not much loved
> > by some of the MM guys).  Given Thomas's patch, we could with high
> > probability allocate directly, perhaps even not needing memory-allocator
> > modifications.
> >
> > Either way, kvfree_rcu(), and later call_rcu(), will avoid asking the
> > allocator to do anything that the calling context prohibits.  So what
> > types of bugs are you looking for?  Where reclaim calls back into the
> > driver or some such?
> 
> Yeah pretty much. It's a problem for gpu, fs, block drivers and really
> anything else that's remotely involved in memory reclaim somehow.
> Generally this is all handled explicitly by passing gfp_t flags down
> any call chain, but in some cases it's instead solved with the
> memalloc_no* functions. E.g. sunrpc uses that to make sure the network
> stack (which generally just assumes it can allocate memory) doesn't,
> to avoid recursions back into nfs/sunrpc. To my knowledge there's no
> way to check at runtime with which gfp flags you're allowed to
> allocate memory, a preemptible check is definitely not enough.
> Disabled preemption implies only GFP_ATOMIC is allowed (ignoring nmi
> and stuff like that), but the inverse is not true.

Thank you for the confirmation!

> So if you want the automagic in call_rcu I think either
> - we need to replace all explicit gfp flags with the context marking
> memalloc_no* across the entire kernel, or at least anywhere rcu might
> be used.
> - audit all callchains and make sure a call_rcu_noalloc is used
> anywhere there might be a problem. probably better to have a
> call_rcu_gfp with explicit gfp flags parameter, since generally that
> needs to be passed down.
> 
> But at least to me the lockless magic in mm sounds a lot safer, since
> it contains the complexity and doesn't leak it out to callers of
> call_rcu.

Agreed, I greatly prefer Peter Zijlstra's lockless-allocation patch
myself.

In the meantime, it looks like we will start by causing the allocation to
happen in a safe environment.  That may have issues with delays, but is
at least something that can be done entirely within the confines of RCU.

							Thanx, Paul

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
  2020-09-16 21:43                     ` Daniel Vetter
                                         ` (18 preceding siblings ...)
  (?)
@ 2020-09-29  8:19                       ` Michal Hocko
  -1 siblings, 0 replies; 282+ messages in thread
From: Michal Hocko @ 2020-09-29  8:19 UTC (permalink / raw)
  To: Daniel Vetter
  Cc: Paul E. McKenney, Juri Lelli, Peter Zijlstra,
	Sebastian Andrzej Siewior, Lai Jiangshan, dri-devel, Ben Segall,
	Linux-MM, open list:KERNEL SELFTEST FRAMEWORK, linux-hexagon,
	Will Deacon, Ingo Molnar, Anton Ivanov, linux-arch,
	Vincent Guittot, Herbert Xu, Brian Cain, Richard Weinberger,
	Russell King, Ard Biesheuvel, David Airlie, Ingo Molnar,
	Geert Uytterhoeven, Mel Gorman, intel-gfx, Matt Turner,
	Valentin Schneider, linux-xtensa, Shuah Khan, Jeff Dike,
	linux-um, Josh Triplett, Steven Rostedt, rcu, linux-m68k,
	Ivan Kokshaysky, Rodrigo Vivi, Thomas Gleixner, Dietmar Eggemann,
	Linux ARM, Richard Henderson, Chris Zankel, Max Filippov,
	Daniel Bristot de Oliveira, LKML, alpha, Mathieu Desnoyers,
	Andrew Morton, Linus Torvalds

On Wed 16-09-20 23:43:02, Daniel Vetter wrote:
> I can
> then figure out whether it's better to risk not spotting issues with
> call_rcu vs slapping a memalloc_noio_save/restore around all these
> critical section which force-degrades any allocation to GFP_ATOMIC at

did you mean memalloc_noreclaim_* here?

> most, but has the risk that we run into code that assumes "GFP_KERNEL
> never fails for small stuff" and has a decidedly less tested fallback
> path than rcu code.

Even if the above then please note that memalloc_noreclaim_* or
PF_MEMALLOC should be used with an extreme care. Essentially only for
internal memory reclaimers. It grants access to _all_ the available
memory so any abuse can be detrimental to the overall system operation.
Allocation failure in this mode means that we are out of memory and any
code relying on such an allocation has to carefuly consider failure.
This is not a random allocation mode.

-- 
Michal Hocko
SUSE Labs

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-29  8:19                       ` Michal Hocko
  0 siblings, 0 replies; 282+ messages in thread
From: Michal Hocko @ 2020-09-29  8:19 UTC (permalink / raw)
  To: Daniel Vetter
  Cc: Paul E. McKenney, Juri Lelli, Peter Zijlstra,
	Sebastian Andrzej Siewior, Lai Jiangshan, dri-devel, Ben Segall,
	Linux-MM, open list:KERNEL SELFTEST FRAMEWORK, linux-hexagon,
	Will Deacon, Ingo Molnar, Anton Ivanov, linux-arch,
	Vincent Guittot, Herbert Xu, Brian Cain, Richard Weinberger,
	Russell King, Ard Biesheuvel, David Airlie, Ingo Molnar,
	Geert Uytterhoeven, Mel Gorman, intel-gfx, Matt Turner,
	Valentin Schneider, linux-xtensa, Shuah Khan, Jeff Dike,
	linux-um, Josh Triplett, Steven Rostedt, rcu, linux-m68k,
	Ivan Kokshaysky, Rodrigo Vivi, Thomas Gleixner, Dietmar Eggemann,
	Linux ARM, Richard Henderson, Chris Zankel, Max Filippov,
	Daniel Bristot de Oliveira, LKML, alpha, Mathieu Desnoyers,
	Andrew Morton, Linus Torvalds

On Wed 16-09-20 23:43:02, Daniel Vetter wrote:
> I can
> then figure out whether it's better to risk not spotting issues with
> call_rcu vs slapping a memalloc_noio_save/restore around all these
> critical section which force-degrades any allocation to GFP_ATOMIC at

did you mean memalloc_noreclaim_* here?

> most, but has the risk that we run into code that assumes "GFP_KERNEL
> never fails for small stuff" and has a decidedly less tested fallback
> path than rcu code.

Even if the above then please note that memalloc_noreclaim_* or
PF_MEMALLOC should be used with an extreme care. Essentially only for
internal memory reclaimers. It grants access to _all_ the available
memory so any abuse can be detrimental to the overall system operation.
Allocation failure in this mode means that we are out of memory and any
code relying on such an allocation has to carefuly consider failure.
This is not a random allocation mode.

-- 
Michal Hocko
SUSE Labs


^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-29  8:19                       ` Michal Hocko
  0 siblings, 0 replies; 282+ messages in thread
From: Michal Hocko @ 2020-09-29  8:19 UTC (permalink / raw)
  To: Daniel Vetter
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, Linux-MM,
	open list:KERNEL SELFTEST FRAMEWORK, Will Deacon, Ard Biesheuvel,
	Anton Ivanov, linux-arch, Vincent Guittot, Herbert Xu,
	Brian Cain, linux-hexagon, Russell King, Ingo Molnar,
	David Airlie, Ingo Molnar, Geert Uytterhoeven, Mel Gorman,
	Matt Turner, Valentin Schneider, linux-xtensa, Shuah Khan,
	Paul E. McKenney, Jeff Dike, intel-gfx, alpha, linux-um,
	Josh Triplett, Steven Rostedt, rcu, linux-m68k, Ivan Kokshaysky,
	Rodrigo Vivi, Thomas Gleixner, Dietmar Eggemann, Linux ARM,
	Richard Henderson, Chris Zankel, Max Filippov, Linus Torvalds,
	LKML, Richard Weinberger, Mathieu Desnoyers, Andrew Morton,
	Daniel Bristot de Oliveira

On Wed 16-09-20 23:43:02, Daniel Vetter wrote:
> I can
> then figure out whether it's better to risk not spotting issues with
> call_rcu vs slapping a memalloc_noio_save/restore around all these
> critical section which force-degrades any allocation to GFP_ATOMIC at

did you mean memalloc_noreclaim_* here?

> most, but has the risk that we run into code that assumes "GFP_KERNEL
> never fails for small stuff" and has a decidedly less tested fallback
> path than rcu code.

Even if the above then please note that memalloc_noreclaim_* or
PF_MEMALLOC should be used with an extreme care. Essentially only for
internal memory reclaimers. It grants access to _all_ the available
memory so any abuse can be detrimental to the overall system operation.
Allocation failure in this mode means that we are out of memory and any
code relying on such an allocation has to carefuly consider failure.
This is not a random allocation mode.

-- 
Michal Hocko
SUSE Labs
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [Intel-gfx] [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-29  8:19                       ` Michal Hocko
  0 siblings, 0 replies; 282+ messages in thread
From: Michal Hocko @ 2020-09-29  8:19 UTC (permalink / raw)
  To: Daniel Vetter
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, Linux-MM,
	open list:KERNEL SELFTEST FRAMEWORK, Will Deacon, Ard Biesheuvel,
	Anton Ivanov, linux-arch, Herbert Xu, Brian Cain, linux-hexagon,
	Russell King, Ingo Molnar, David Airlie, Ingo Molnar,
	Geert Uytterhoeven, Mel Gorman, Matt Turner, Valentin Schneider,
	linux-xtensa, Shuah Khan, Paul E. McKenney, Jeff Dike, intel-gfx,
	alpha, linux-um, Josh Triplett, Steven Rostedt, rcu, linux-m68k,
	Ivan Kokshaysky, Thomas Gleixner, Dietmar Eggemann, Linux ARM,
	Richard Henderson, Chris Zankel, Max Filippov, Linus Torvalds,
	LKML, Richard Weinberger, Mathieu Desnoyers, Andrew Morton,
	Daniel Bristot de Oliveira

On Wed 16-09-20 23:43:02, Daniel Vetter wrote:
> I can
> then figure out whether it's better to risk not spotting issues with
> call_rcu vs slapping a memalloc_noio_save/restore around all these
> critical section which force-degrades any allocation to GFP_ATOMIC at

did you mean memalloc_noreclaim_* here?

> most, but has the risk that we run into code that assumes "GFP_KERNEL
> never fails for small stuff" and has a decidedly less tested fallback
> path than rcu code.

Even if the above then please note that memalloc_noreclaim_* or
PF_MEMALLOC should be used with an extreme care. Essentially only for
internal memory reclaimers. It grants access to _all_ the available
memory so any abuse can be detrimental to the overall system operation.
Allocation failure in this mode means that we are out of memory and any
code relying on such an allocation has to carefuly consider failure.
This is not a random allocation mode.

-- 
Michal Hocko
SUSE Labs
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-29  8:19                       ` Michal Hocko
  0 siblings, 0 replies; 282+ messages in thread
From: Michal Hocko @ 2020-09-29  8:19 UTC (permalink / raw)
  To: Daniel Vetter
  Cc: Paul E. McKenney, Juri Lelli, Peter Zijlstra,
	Sebastian Andrzej Siewior, Lai Jiangshan, dri-devel, Ben Segall,
	Linux-MM, open list:KERNEL SELFTEST FRAMEWORK, linux-hexagon,
	Will Deacon, Ingo Molnar, Anton Ivanov, linux-arch,
	Vincent Guittot, Herbert Xu, Brian Cain, Richard Weinberger,
	Russell King

On Wed 16-09-20 23:43:02, Daniel Vetter wrote:
> I can
> then figure out whether it's better to risk not spotting issues with
> call_rcu vs slapping a memalloc_noio_save/restore around all these
> critical section which force-degrades any allocation to GFP_ATOMIC at

did you mean memalloc_noreclaim_* here?

> most, but has the risk that we run into code that assumes "GFP_KERNEL
> never fails for small stuff" and has a decidedly less tested fallback
> path than rcu code.

Even if the above then please note that memalloc_noreclaim_* or
PF_MEMALLOC should be used with an extreme care. Essentially only for
internal memory reclaimers. It grants access to _all_ the available
memory so any abuse can be detrimental to the overall system operation.
Allocation failure in this mode means that we are out of memory and any
code relying on such an allocation has to carefuly consider failure.
This is not a random allocation mode.

-- 
Michal Hocko
SUSE Labs

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-29  8:19                       ` Michal Hocko
  0 siblings, 0 replies; 282+ messages in thread
From: Michal Hocko @ 2020-09-29  8:19 UTC (permalink / raw)
  To: Daniel Vetter
  Cc: Paul E. McKenney, Juri Lelli, Peter Zijlstra,
	Sebastian Andrzej Siewior, Lai Jiangshan, dri-devel, Ben Segall,
	Linux-MM, open list:KERNEL SELFTEST FRAMEWORK, linux-hexagon,
	Will Deacon, Ingo Molnar, Anton Ivanov, linux-arch,
	Vincent Guittot, Herbert Xu, Brian Cain, Richard Weinberger,
	Russell King, Ard Biesheuvel, David Airlie, Ingo Molnar,
	Geert Uytterhoeven, Mel Gorman, intel-gfx, Matt Turner,
	Valentin Schneider, linux-xtensa, Shuah Khan, Jeff Dike,
	linux-um, Josh Triplett, Steven Rostedt, rcu, linux-m68k,
	Ivan Kokshaysky, Rodrigo Vivi, Thomas Gleixner, Dietmar Eggemann,
	Linux ARM, Richard Henderson, Chris Zankel, Max Filippov,
	Daniel Bristot de Oliveira, LKML, alpha, Mathieu Desnoyers,
	Andrew Morton, Linus Torvalds

On Wed 16-09-20 23:43:02, Daniel Vetter wrote:
> I can
> then figure out whether it's better to risk not spotting issues with
> call_rcu vs slapping a memalloc_noio_save/restore around all these
> critical section which force-degrades any allocation to GFP_ATOMIC at

did you mean memalloc_noreclaim_* here?

> most, but has the risk that we run into code that assumes "GFP_KERNEL
> never fails for small stuff" and has a decidedly less tested fallback
> path than rcu code.

Even if the above then please note that memalloc_noreclaim_* or
PF_MEMALLOC should be used with an extreme care. Essentially only for
internal memory reclaimers. It grants access to _all_ the available
memory so any abuse can be detrimental to the overall system operation.
Allocation failure in this mode means that we are out of memory and any
code relying on such an allocation has to carefuly consider failure.
This is not a random allocation mode.

-- 
Michal Hocko
SUSE Labs

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-29  8:19                       ` Michal Hocko
  0 siblings, 0 replies; 282+ messages in thread
From: Michal Hocko @ 2020-09-29  8:19 UTC (permalink / raw)
  To: Daniel Vetter
  Cc: Paul E. McKenney, Juri Lelli, Peter Zijlstra,
	Sebastian Andrzej Siewior, Lai Jiangshan, dri-devel, Ben Segall,
	Linux-MM, open list:KERNEL SELFTEST FRAMEWORK, linux-hexagon,
	Will Deacon, Ingo Molnar, Anton Ivanov, linux-arch,
	Vincent Guittot, Herbert Xu, Brian Cain, Richard Weinberger,
	Russell King, Ard Biesheuvel, David Airlie, Ingo Molnar,
	Geert Uytterhoeven, Mel Gorman, intel-gfx, Matt Turner,
	Valentin Schneider, linux-xtensa, Shuah Khan, Jeff Dike,
	linux-um, Josh Triplett, Steven Rostedt, rcu, linux-m68k,
	Ivan Kokshaysky, Rodrigo Vivi, Thomas Gleixner, Dietmar Eggemann,
	Linux ARM, Richard Henderson, Chris Zankel, Max Filippov,
	Daniel Bristot de Oliveira, LKML, alpha, Mathieu Desnoyers,
	Andrew Morton, Linus Torvalds

On Wed 16-09-20 23:43:02, Daniel Vetter wrote:
> I can
> then figure out whether it's better to risk not spotting issues with
> call_rcu vs slapping a memalloc_noio_save/restore around all these
> critical section which force-degrades any allocation to GFP_ATOMIC at

did you mean memalloc_noreclaim_* here?

> most, but has the risk that we run into code that assumes "GFP_KERNEL
> never fails for small stuff" and has a decidedly less tested fallback
> path than rcu code.

Even if the above then please note that memalloc_noreclaim_* or
PF_MEMALLOC should be used with an extreme care. Essentially only for
internal memory reclaimers. It grants access to _all_ the available
memory so any abuse can be detrimental to the overall system operation.
Allocation failure in this mode means that we are out of memory and any
code relying on such an allocation has to carefuly consider failure.
This is not a random allocation mode.

-- 
Michal Hocko
SUSE Labs


^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-29  8:19                       ` Michal Hocko
  0 siblings, 0 replies; 282+ messages in thread
From: Michal Hocko @ 2020-09-29  8:19 UTC (permalink / raw)
  To: Daniel Vetter
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, Linux-MM,
	open list:KERNEL SELFTEST FRAMEWORK, Will Deacon, Ard Biesheuvel,
	Anton Ivanov, linux-arch, Vincent Guittot, Herbert Xu,
	Brian Cain, linux-hexagon, Russell King, Ingo Molnar,
	David Airlie, Ingo Molnar, Geert Uytterhoeven, Mel Gorman,
	Matt Turner, Valentin Schneider, linux-xtensa, Shuah Khan,
	Paul E. McKenney, Jeff Dike, intel-gfx, alpha, linux-um,
	Josh Triplett, Steven Rostedt, rcu, linux-m68k, Ivan Kokshaysky,
	Rodrigo Vivi, Thomas Gleixner, Dietmar Eggemann, Linux ARM,
	Richard Henderson, Chris Zankel, Max Filippov, Linus Torvalds,
	LKML, Richard Weinberger, Mathieu Desnoyers, Andrew Morton,
	Daniel Bristot de Oliveira

On Wed 16-09-20 23:43:02, Daniel Vetter wrote:
> I can
> then figure out whether it's better to risk not spotting issues with
> call_rcu vs slapping a memalloc_noio_save/restore around all these
> critical section which force-degrades any allocation to GFP_ATOMIC at

did you mean memalloc_noreclaim_* here?

> most, but has the risk that we run into code that assumes "GFP_KERNEL
> never fails for small stuff" and has a decidedly less tested fallback
> path than rcu code.

Even if the above then please note that memalloc_noreclaim_* or
PF_MEMALLOC should be used with an extreme care. Essentially only for
internal memory reclaimers. It grants access to _all_ the available
memory so any abuse can be detrimental to the overall system operation.
Allocation failure in this mode means that we are out of memory and any
code relying on such an allocation has to carefuly consider failure.
This is not a random allocation mode.

-- 
Michal Hocko
SUSE Labs
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [Intel-gfx] [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-29  8:19                       ` Michal Hocko
  0 siblings, 0 replies; 282+ messages in thread
From: Michal Hocko @ 2020-09-29  8:19 UTC (permalink / raw)
  To: Daniel Vetter
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, Linux-MM,
	open list:KERNEL SELFTEST FRAMEWORK, Will Deacon, Ard Biesheuvel,
	Anton Ivanov, linux-arch, Herbert Xu, Brian Cain, linux-hexagon,
	Russell King, Ingo Molnar, David Airlie, Ingo Molnar,
	Geert Uytterhoeven, Mel Gorman, Matt Turner, Valentin Schneider,
	linux-xtensa, Shuah Khan, Paul E. McKenney, Jeff Dike, intel-gfx,
	alpha, linux-um, Josh Triplett, Steven Rostedt, rcu, linux-m68k,
	Ivan Kokshaysky, Thomas Gleixner, Dietmar Eggemann, Linux ARM,
	Richard Henderson, Chris Zankel, Max Filippov, Linus Torvalds,
	LKML, Richard Weinberger, Mathieu Desnoyers, Andrew Morton,
	Daniel Bristot de Oliveira

On Wed 16-09-20 23:43:02, Daniel Vetter wrote:
> I can
> then figure out whether it's better to risk not spotting issues with
> call_rcu vs slapping a memalloc_noio_save/restore around all these
> critical section which force-degrades any allocation to GFP_ATOMIC at

did you mean memalloc_noreclaim_* here?

> most, but has the risk that we run into code that assumes "GFP_KERNEL
> never fails for small stuff" and has a decidedly less tested fallback
> path than rcu code.

Even if the above then please note that memalloc_noreclaim_* or
PF_MEMALLOC should be used with an extreme care. Essentially only for
internal memory reclaimers. It grants access to _all_ the available
memory so any abuse can be detrimental to the overall system operation.
Allocation failure in this mode means that we are out of memory and any
code relying on such an allocation has to carefuly consider failure.
This is not a random allocation mode.

-- 
Michal Hocko
SUSE Labs
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-29  8:19                       ` Michal Hocko
  0 siblings, 0 replies; 282+ messages in thread
From: Michal Hocko @ 2020-09-29  8:20 UTC (permalink / raw)
  To: Daniel Vetter
  Cc: Paul E. McKenney, Juri Lelli, Peter Zijlstra,
	Sebastian Andrzej Siewior, Lai Jiangshan, dri-devel, Ben Segall,
	Linux-MM, open list:KERNEL SELFTEST FRAMEWORK, linux-hexagon,
	Will Deacon, Ingo Molnar, Anton Ivanov, linux-arch,
	Vincent Guittot, Herbert Xu, Brian Cain, Richard Weinberger,
	Russell King, Ard Biesheuvel, David Airlie, Ingo Molnar,
	Geert Uytterhoeven, Mel Gorman, intel-gfx, Matt Turner,
	Valentin Schneider, linux-xtensa, Shuah Khan, Jeff Dike,
	linux-um, Josh Triplett, Steven Rostedt, rcu, linux-m68k,
	Ivan Kokshaysky, Rodrigo Vivi, Thomas Gleixner, Dietmar Eggemann,
	Linux ARM, Richard Henderson, Chris Zankel, Max Filippov,
	Daniel Bristot de Oliveira, LKML, alpha, Mathieu Desnoyers,
	Andrew Morton, Linus Torvalds

On Wed 16-09-20 23:43:02, Daniel Vetter wrote:
> I can
> then figure out whether it's better to risk not spotting issues with
> call_rcu vs slapping a memalloc_noio_save/restore around all these
> critical section which force-degrades any allocation to GFP_ATOMIC at

did you mean memalloc_noreclaim_* here?

> most, but has the risk that we run into code that assumes "GFP_KERNEL
> never fails for small stuff" and has a decidedly less tested fallback
> path than rcu code.

Even if the above then please note that memalloc_noreclaim_* or
PF_MEMALLOC should be used with an extreme care. Essentially only for
internal memory reclaimers. It grants access to _all_ the available
memory so any abuse can be detrimental to the overall system operation.
Allocation failure in this mode means that we are out of memory and any
code relying on such an allocation has to carefuly consider failure.
This is not a random allocation mode.

-- 
Michal Hocko
SUSE Labs

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-29  8:19                       ` Michal Hocko
  0 siblings, 0 replies; 282+ messages in thread
From: Michal Hocko @ 2020-09-29  8:20 UTC (permalink / raw)
  To: Daniel Vetter
  Cc: Paul E. McKenney, Juri Lelli, Peter Zijlstra,
	Sebastian Andrzej Siewior, Lai Jiangshan, dri-devel, Ben Segall,
	Linux-MM, open list:KERNEL SELFTEST FRAMEWORK, linux-hexagon,
	Will Deacon, Ingo Molnar, Anton Ivanov, linux-arch,
	Vincent Guittot, Herbert Xu, Brian Cain, Richard Weinberger,
	Russell King, Ard Biesheuvel, David Airlie, Ingo Molnar,
	Geert Uytterhoeven, Mel Gorman, intel-gfx, Matt Turner,
	Valentin Schneider, linux-xtensa, Shuah Khan, Jeff Dike,
	linux-um, Josh Triplett, Steven Rostedt, rcu, linux-m68k,
	Ivan Kokshaysky, Rodrigo Vivi, Thomas Gleixner, Dietmar Eggemann,
	Linux ARM, Richard Henderson, Chris Zankel, Max Filippov,
	Daniel Bristot de Oliveira, LKML, alpha, Mathieu Desnoyers,
	Andrew Morton, Linus Torvalds

On Wed 16-09-20 23:43:02, Daniel Vetter wrote:
> I can
> then figure out whether it's better to risk not spotting issues with
> call_rcu vs slapping a memalloc_noio_save/restore around all these
> critical section which force-degrades any allocation to GFP_ATOMIC at

did you mean memalloc_noreclaim_* here?

> most, but has the risk that we run into code that assumes "GFP_KERNEL
> never fails for small stuff" and has a decidedly less tested fallback
> path than rcu code.

Even if the above then please note that memalloc_noreclaim_* or
PF_MEMALLOC should be used with an extreme care. Essentially only for
internal memory reclaimers. It grants access to _all_ the available
memory so any abuse can be detrimental to the overall system operation.
Allocation failure in this mode means that we are out of memory and any
code relying on such an allocation has to carefuly consider failure.
This is not a random allocation mode.

-- 
Michal Hocko
SUSE Labs


^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-29  8:19                       ` Michal Hocko
  0 siblings, 0 replies; 282+ messages in thread
From: Michal Hocko @ 2020-09-29  8:20 UTC (permalink / raw)
  To: Daniel Vetter
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, Linux-MM,
	open list:KERNEL SELFTEST FRAMEWORK, Will Deacon, Ard Biesheuvel,
	Anton Ivanov, linux-arch, Vincent Guittot, Herbert Xu,
	Brian Cain, linux-hexagon, Russell King, Ingo Molnar,
	David Airlie, Ingo Molnar, Geert Uytterhoeven, Mel Gorman,
	Matt Turner, Valentin Schneider, linux-xtensa, Shuah Khan,
	Paul E. McKenney, Jeff Dike, intel-gfx, alpha, linux-um,
	Josh Triplett, Steven Rostedt, rcu, linux-m68k, Ivan Kokshaysky,
	Rodrigo Vivi, Thomas Gleixner, Dietmar Eggemann, Linux ARM,
	Richard Henderson, Chris Zankel, Max Filippov, Linus Torvalds,
	LKML, Richard Weinberger, Mathieu Desnoyers, Andrew Morton,
	Daniel Bristot de Oliveira

On Wed 16-09-20 23:43:02, Daniel Vetter wrote:
> I can
> then figure out whether it's better to risk not spotting issues with
> call_rcu vs slapping a memalloc_noio_save/restore around all these
> critical section which force-degrades any allocation to GFP_ATOMIC at

did you mean memalloc_noreclaim_* here?

> most, but has the risk that we run into code that assumes "GFP_KERNEL
> never fails for small stuff" and has a decidedly less tested fallback
> path than rcu code.

Even if the above then please note that memalloc_noreclaim_* or
PF_MEMALLOC should be used with an extreme care. Essentially only for
internal memory reclaimers. It grants access to _all_ the available
memory so any abuse can be detrimental to the overall system operation.
Allocation failure in this mode means that we are out of memory and any
code relying on such an allocation has to carefuly consider failure.
This is not a random allocation mode.

-- 
Michal Hocko
SUSE Labs
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [Intel-gfx] [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-29  8:19                       ` Michal Hocko
  0 siblings, 0 replies; 282+ messages in thread
From: Michal Hocko @ 2020-09-29  8:20 UTC (permalink / raw)
  To: Daniel Vetter
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, Linux-MM,
	open list:KERNEL SELFTEST FRAMEWORK, Will Deacon, Ard Biesheuvel,
	Anton Ivanov, linux-arch, Herbert Xu, Brian Cain, linux-hexagon,
	Russell King, Ingo Molnar, David Airlie, Ingo Molnar,
	Geert Uytterhoeven, Mel Gorman, Matt Turner, Valentin Schneider,
	linux-xtensa, Shuah Khan, Paul E. McKenney, Jeff Dike, intel-gfx,
	alpha, linux-um, Josh Triplett, Steven Rostedt, rcu, linux-m68k,
	Ivan Kokshaysky, Thomas Gleixner, Dietmar Eggemann, Linux ARM,
	Richard Henderson, Chris Zankel, Max Filippov, Linus Torvalds,
	LKML, Richard Weinberger, Mathieu Desnoyers, Andrew Morton,
	Daniel Bristot de Oliveira

On Wed 16-09-20 23:43:02, Daniel Vetter wrote:
> I can
> then figure out whether it's better to risk not spotting issues with
> call_rcu vs slapping a memalloc_noio_save/restore around all these
> critical section which force-degrades any allocation to GFP_ATOMIC at

did you mean memalloc_noreclaim_* here?

> most, but has the risk that we run into code that assumes "GFP_KERNEL
> never fails for small stuff" and has a decidedly less tested fallback
> path than rcu code.

Even if the above then please note that memalloc_noreclaim_* or
PF_MEMALLOC should be used with an extreme care. Essentially only for
internal memory reclaimers. It grants access to _all_ the available
memory so any abuse can be detrimental to the overall system operation.
Allocation failure in this mode means that we are out of memory and any
code relying on such an allocation has to carefuly consider failure.
This is not a random allocation mode.

-- 
Michal Hocko
SUSE Labs
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-29  8:19                       ` Michal Hocko
  0 siblings, 0 replies; 282+ messages in thread
From: Michal Hocko @ 2020-09-29  8:21 UTC (permalink / raw)
  To: Daniel Vetter
  Cc: Paul E. McKenney, Juri Lelli, Peter Zijlstra,
	Sebastian Andrzej Siewior, Lai Jiangshan, dri-devel, Ben Segall,
	Linux-MM, open list:KERNEL SELFTEST FRAMEWORK, linux-hexagon,
	Will Deacon, Ingo Molnar, Anton Ivanov, linux-arch,
	Vincent Guittot, Herbert Xu, Brian Cain, Richard Weinberger,
	Russell King, Ard Biesheuvel, David Airlie, Ingo Molnar,
	Geert Uytterhoeven, Mel Gorman, intel-gfx, Matt Turner,
	Valentin Schneider, linux-xtensa, Shuah Khan, Jeff Dike,
	linux-um, Josh Triplett, Steven Rostedt, rcu, linux-m68k,
	Ivan Kokshaysky, Rodrigo Vivi, Thomas Gleixner, Dietmar Eggemann,
	Linux ARM, Richard Henderson, Chris Zankel, Max Filippov,
	Daniel Bristot de Oliveira, LKML, alpha, Mathieu Desnoyers,
	Andrew Morton, Linus Torvalds

On Wed 16-09-20 23:43:02, Daniel Vetter wrote:
> I can
> then figure out whether it's better to risk not spotting issues with
> call_rcu vs slapping a memalloc_noio_save/restore around all these
> critical section which force-degrades any allocation to GFP_ATOMIC at

did you mean memalloc_noreclaim_* here?

> most, but has the risk that we run into code that assumes "GFP_KERNEL
> never fails for small stuff" and has a decidedly less tested fallback
> path than rcu code.

Even if the above then please note that memalloc_noreclaim_* or
PF_MEMALLOC should be used with an extreme care. Essentially only for
internal memory reclaimers. It grants access to _all_ the available
memory so any abuse can be detrimental to the overall system operation.
Allocation failure in this mode means that we are out of memory and any
code relying on such an allocation has to carefuly consider failure.
This is not a random allocation mode.

-- 
Michal Hocko
SUSE Labs

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-29  8:19                       ` Michal Hocko
  0 siblings, 0 replies; 282+ messages in thread
From: Michal Hocko @ 2020-09-29  8:21 UTC (permalink / raw)
  To: Daniel Vetter
  Cc: Paul E. McKenney, Juri Lelli, Peter Zijlstra,
	Sebastian Andrzej Siewior, Lai Jiangshan, dri-devel, Ben Segall,
	Linux-MM, open list:KERNEL SELFTEST FRAMEWORK, linux-hexagon,
	Will Deacon, Ingo Molnar, Anton Ivanov, linux-arch,
	Vincent Guittot, Herbert Xu, Brian Cain, Richard Weinberger,
	Russell King, Ard Biesheuvel, David Airlie, Ingo Molnar,
	Geert Uytterhoeven, Mel Gorman, intel-gfx, Matt Turner,
	Valentin Schneider, linux-xtensa, Shuah Khan, Jeff Dike,
	linux-um, Josh Triplett, Steven Rostedt, rcu, linux-m68k,
	Ivan Kokshaysky, Rodrigo Vivi, Thomas Gleixner, Dietmar Eggemann,
	Linux ARM, Richard Henderson, Chris Zankel, Max Filippov,
	Daniel Bristot de Oliveira, LKML, alpha, Mathieu Desnoyers,
	Andrew Morton, Linus Torvalds

On Wed 16-09-20 23:43:02, Daniel Vetter wrote:
> I can
> then figure out whether it's better to risk not spotting issues with
> call_rcu vs slapping a memalloc_noio_save/restore around all these
> critical section which force-degrades any allocation to GFP_ATOMIC at

did you mean memalloc_noreclaim_* here?

> most, but has the risk that we run into code that assumes "GFP_KERNEL
> never fails for small stuff" and has a decidedly less tested fallback
> path than rcu code.

Even if the above then please note that memalloc_noreclaim_* or
PF_MEMALLOC should be used with an extreme care. Essentially only for
internal memory reclaimers. It grants access to _all_ the available
memory so any abuse can be detrimental to the overall system operation.
Allocation failure in this mode means that we are out of memory and any
code relying on such an allocation has to carefuly consider failure.
This is not a random allocation mode.

-- 
Michal Hocko
SUSE Labs


^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-29  8:19                       ` Michal Hocko
  0 siblings, 0 replies; 282+ messages in thread
From: Michal Hocko @ 2020-09-29  8:21 UTC (permalink / raw)
  To: Daniel Vetter
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, Linux-MM,
	open list:KERNEL SELFTEST FRAMEWORK, Will Deacon, Ard Biesheuvel,
	Anton Ivanov, linux-arch, Vincent Guittot, Herbert Xu,
	Brian Cain, linux-hexagon, Russell King, Ingo Molnar,
	David Airlie, Ingo Molnar, Geert Uytterhoeven, Mel Gorman,
	Matt Turner, Valentin Schneider, linux-xtensa, Shuah Khan,
	Paul E. McKenney, Jeff Dike, intel-gfx, alpha, linux-um,
	Josh Triplett, Steven Rostedt, rcu, linux-m68k, Ivan Kokshaysky,
	Rodrigo Vivi, Thomas Gleixner, Dietmar Eggemann, Linux ARM,
	Richard Henderson, Chris Zankel, Max Filippov, Linus Torvalds,
	LKML, Richard Weinberger, Mathieu Desnoyers, Andrew Morton,
	Daniel Bristot de Oliveira

On Wed 16-09-20 23:43:02, Daniel Vetter wrote:
> I can
> then figure out whether it's better to risk not spotting issues with
> call_rcu vs slapping a memalloc_noio_save/restore around all these
> critical section which force-degrades any allocation to GFP_ATOMIC at

did you mean memalloc_noreclaim_* here?

> most, but has the risk that we run into code that assumes "GFP_KERNEL
> never fails for small stuff" and has a decidedly less tested fallback
> path than rcu code.

Even if the above then please note that memalloc_noreclaim_* or
PF_MEMALLOC should be used with an extreme care. Essentially only for
internal memory reclaimers. It grants access to _all_ the available
memory so any abuse can be detrimental to the overall system operation.
Allocation failure in this mode means that we are out of memory and any
code relying on such an allocation has to carefuly consider failure.
This is not a random allocation mode.

-- 
Michal Hocko
SUSE Labs
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [Intel-gfx] [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-29  8:19                       ` Michal Hocko
  0 siblings, 0 replies; 282+ messages in thread
From: Michal Hocko @ 2020-09-29  8:21 UTC (permalink / raw)
  To: Daniel Vetter
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, Linux-MM,
	open list:KERNEL SELFTEST FRAMEWORK, Will Deacon, Ard Biesheuvel,
	Anton Ivanov, linux-arch, Herbert Xu, Brian Cain, linux-hexagon,
	Russell King, Ingo Molnar, David Airlie, Ingo Molnar,
	Geert Uytterhoeven, Mel Gorman, Matt Turner, Valentin Schneider,
	linux-xtensa, Shuah Khan, Paul E. McKenney, Jeff Dike, intel-gfx,
	alpha, linux-um, Josh Triplett, Steven Rostedt, rcu, linux-m68k,
	Ivan Kokshaysky, Thomas Gleixner, Dietmar Eggemann, Linux ARM,
	Richard Henderson, Chris Zankel, Max Filippov, Linus Torvalds,
	LKML, Richard Weinberger, Mathieu Desnoyers, Andrew Morton,
	Daniel Bristot de Oliveira

On Wed 16-09-20 23:43:02, Daniel Vetter wrote:
> I can
> then figure out whether it's better to risk not spotting issues with
> call_rcu vs slapping a memalloc_noio_save/restore around all these
> critical section which force-degrades any allocation to GFP_ATOMIC at

did you mean memalloc_noreclaim_* here?

> most, but has the risk that we run into code that assumes "GFP_KERNEL
> never fails for small stuff" and has a decidedly less tested fallback
> path than rcu code.

Even if the above then please note that memalloc_noreclaim_* or
PF_MEMALLOC should be used with an extreme care. Essentially only for
internal memory reclaimers. It grants access to _all_ the available
memory so any abuse can be detrimental to the overall system operation.
Allocation failure in this mode means that we are out of memory and any
code relying on such an allocation has to carefuly consider failure.
This is not a random allocation mode.

-- 
Michal Hocko
SUSE Labs
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-29  8:19                       ` Michal Hocko
  0 siblings, 0 replies; 282+ messages in thread
From: Michal Hocko @ 2020-09-29  8:23 UTC (permalink / raw)
  To: Daniel Vetter
  Cc: Paul E. McKenney, Juri Lelli, Peter Zijlstra,
	Sebastian Andrzej Siewior, Lai Jiangshan, dri-devel, Ben Segall,
	Linux-MM, open list:KERNEL SELFTEST FRAMEWORK, linux-hexagon,
	Will Deacon, Ingo Molnar, Anton Ivanov, linux-arch,
	Vincent Guittot, Herbert Xu, Brian Cain, Richard Weinberger,
	Russell King, Ard Biesheuvel, David Airlie, Ingo Molnar,
	Geert Uytterhoeven, Mel Gorman, intel-gfx, Matt Turner,
	Valentin Schneider, linux-xtensa, Shuah Khan, Jeff Dike,
	linux-um, Josh Triplett, Steven Rostedt, rcu, linux-m68k,
	Ivan Kokshaysky, Rodrigo Vivi, Thomas Gleixner, Dietmar Eggemann,
	Linux ARM, Richard Henderson, Chris Zankel, Max Filippov,
	Daniel Bristot de Oliveira, LKML, alpha, Mathieu Desnoyers,
	Andrew Morton, Linus Torvalds

On Wed 16-09-20 23:43:02, Daniel Vetter wrote:
> I can
> then figure out whether it's better to risk not spotting issues with
> call_rcu vs slapping a memalloc_noio_save/restore around all these
> critical section which force-degrades any allocation to GFP_ATOMIC at

did you mean memalloc_noreclaim_* here?

> most, but has the risk that we run into code that assumes "GFP_KERNEL
> never fails for small stuff" and has a decidedly less tested fallback
> path than rcu code.

Even if the above then please note that memalloc_noreclaim_* or
PF_MEMALLOC should be used with an extreme care. Essentially only for
internal memory reclaimers. It grants access to _all_ the available
memory so any abuse can be detrimental to the overall system operation.
Allocation failure in this mode means that we are out of memory and any
code relying on such an allocation has to carefuly consider failure.
This is not a random allocation mode.

-- 
Michal Hocko
SUSE Labs

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-29  8:19                       ` Michal Hocko
  0 siblings, 0 replies; 282+ messages in thread
From: Michal Hocko @ 2020-09-29  8:23 UTC (permalink / raw)
  To: Daniel Vetter
  Cc: Paul E. McKenney, Juri Lelli, Peter Zijlstra,
	Sebastian Andrzej Siewior, Lai Jiangshan, dri-devel, Ben Segall,
	Linux-MM, open list:KERNEL SELFTEST FRAMEWORK, linux-hexagon,
	Will Deacon, Ingo Molnar, Anton Ivanov, linux-arch,
	Vincent Guittot, Herbert Xu, Brian Cain, Richard Weinberger,
	Russell King, Ard Biesheuvel, David Airlie, Ingo Molnar,
	Geert Uytterhoeven, Mel Gorman, intel-gfx, Matt Turner,
	Valentin Schneider, linux-xtensa, Shuah Khan, Jeff Dike,
	linux-um, Josh Triplett, Steven Rostedt, rcu, linux-m68k,
	Ivan Kokshaysky, Rodrigo Vivi, Thomas Gleixner, Dietmar Eggemann,
	Linux ARM, Richard Henderson, Chris Zankel, Max Filippov,
	Daniel Bristot de Oliveira, LKML, alpha, Mathieu Desnoyers,
	Andrew Morton, Linus Torvalds

On Wed 16-09-20 23:43:02, Daniel Vetter wrote:
> I can
> then figure out whether it's better to risk not spotting issues with
> call_rcu vs slapping a memalloc_noio_save/restore around all these
> critical section which force-degrades any allocation to GFP_ATOMIC at

did you mean memalloc_noreclaim_* here?

> most, but has the risk that we run into code that assumes "GFP_KERNEL
> never fails for small stuff" and has a decidedly less tested fallback
> path than rcu code.

Even if the above then please note that memalloc_noreclaim_* or
PF_MEMALLOC should be used with an extreme care. Essentially only for
internal memory reclaimers. It grants access to _all_ the available
memory so any abuse can be detrimental to the overall system operation.
Allocation failure in this mode means that we are out of memory and any
code relying on such an allocation has to carefuly consider failure.
This is not a random allocation mode.

-- 
Michal Hocko
SUSE Labs


^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-29  8:19                       ` Michal Hocko
  0 siblings, 0 replies; 282+ messages in thread
From: Michal Hocko @ 2020-09-29  8:23 UTC (permalink / raw)
  To: Daniel Vetter
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, Linux-MM,
	open list:KERNEL SELFTEST FRAMEWORK, Will Deacon, Ard Biesheuvel,
	Anton Ivanov, linux-arch, Vincent Guittot, Herbert Xu,
	Brian Cain, linux-hexagon, Russell King, Ingo Molnar,
	David Airlie, Ingo Molnar, Geert Uytterhoeven, Mel Gorman,
	Matt Turner, Valentin Schneider, linux-xtensa, Shuah Khan,
	Paul E. McKenney, Jeff Dike, intel-gfx, alpha, linux-um,
	Josh Triplett, Steven Rostedt, rcu, linux-m68k, Ivan Kokshaysky,
	Rodrigo Vivi, Thomas Gleixner, Dietmar Eggemann, Linux ARM,
	Richard Henderson, Chris Zankel, Max Filippov, Linus Torvalds,
	LKML, Richard Weinberger, Mathieu Desnoyers, Andrew Morton,
	Daniel Bristot de Oliveira

On Wed 16-09-20 23:43:02, Daniel Vetter wrote:
> I can
> then figure out whether it's better to risk not spotting issues with
> call_rcu vs slapping a memalloc_noio_save/restore around all these
> critical section which force-degrades any allocation to GFP_ATOMIC at

did you mean memalloc_noreclaim_* here?

> most, but has the risk that we run into code that assumes "GFP_KERNEL
> never fails for small stuff" and has a decidedly less tested fallback
> path than rcu code.

Even if the above then please note that memalloc_noreclaim_* or
PF_MEMALLOC should be used with an extreme care. Essentially only for
internal memory reclaimers. It grants access to _all_ the available
memory so any abuse can be detrimental to the overall system operation.
Allocation failure in this mode means that we are out of memory and any
code relying on such an allocation has to carefuly consider failure.
This is not a random allocation mode.

-- 
Michal Hocko
SUSE Labs
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [Intel-gfx] [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-29  8:19                       ` Michal Hocko
  0 siblings, 0 replies; 282+ messages in thread
From: Michal Hocko @ 2020-09-29  8:23 UTC (permalink / raw)
  To: Daniel Vetter
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, Linux-MM,
	open list:KERNEL SELFTEST FRAMEWORK, Will Deacon, Ard Biesheuvel,
	Anton Ivanov, linux-arch, Herbert Xu, Brian Cain, linux-hexagon,
	Russell King, Ingo Molnar, David Airlie, Ingo Molnar,
	Geert Uytterhoeven, Mel Gorman, Matt Turner, Valentin Schneider,
	linux-xtensa, Shuah Khan, Paul E. McKenney, Jeff Dike, intel-gfx,
	alpha, linux-um, Josh Triplett, Steven Rostedt, rcu, linux-m68k,
	Ivan Kokshaysky, Thomas Gleixner, Dietmar Eggemann, Linux ARM,
	Richard Henderson, Chris Zankel, Max Filippov, Linus Torvalds,
	LKML, Richard Weinberger, Mathieu Desnoyers, Andrew Morton,
	Daniel Bristot de Oliveira

On Wed 16-09-20 23:43:02, Daniel Vetter wrote:
> I can
> then figure out whether it's better to risk not spotting issues with
> call_rcu vs slapping a memalloc_noio_save/restore around all these
> critical section which force-degrades any allocation to GFP_ATOMIC at

did you mean memalloc_noreclaim_* here?

> most, but has the risk that we run into code that assumes "GFP_KERNEL
> never fails for small stuff" and has a decidedly less tested fallback
> path than rcu code.

Even if the above then please note that memalloc_noreclaim_* or
PF_MEMALLOC should be used with an extreme care. Essentially only for
internal memory reclaimers. It grants access to _all_ the available
memory so any abuse can be detrimental to the overall system operation.
Allocation failure in this mode means that we are out of memory and any
code relying on such an allocation has to carefuly consider failure.
This is not a random allocation mode.

-- 
Michal Hocko
SUSE Labs
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
  2020-09-29  8:19                       ` Michal Hocko
                                           ` (2 preceding siblings ...)
  (?)
@ 2020-09-29  9:00                         ` Daniel Vetter
  -1 siblings, 0 replies; 282+ messages in thread
From: Daniel Vetter @ 2020-09-29  9:00 UTC (permalink / raw)
  To: Michal Hocko
  Cc: Daniel Vetter, Paul E. McKenney, Juri Lelli, Peter Zijlstra,
	Sebastian Andrzej Siewior, Lai Jiangshan, dri-devel, Ben Segall,
	Linux-MM, open list:KERNEL SELFTEST FRAMEWORK, linux-hexagon,
	Will Deacon, Ingo Molnar, Anton Ivanov, linux-arch,
	Vincent Guittot, Herbert Xu, Brian Cain, Richard Weinberger,
	Russell King, Ard Biesheuvel, David Airlie, Ingo Molnar,
	Geert Uytterhoeven, Mel Gorman, intel-gfx, Matt Turner,
	Valentin Schneider, linux-xtensa, Shuah Khan, Jeff Dike,
	linux-um, Josh Triplett, Steven Rostedt, rcu, linux-m68k,
	Ivan Kokshaysky, Rodrigo Vivi, Thomas Gleixner, Dietmar Eggemann,
	Linux ARM, Richard Henderson, Chris Zankel, Max Filippov,
	Daniel Bristot de Oliveira, LKML, alpha, Mathieu Desnoyers,
	Andrew Morton, Linus Torvalds

On Tue, Sep 29, 2020 at 10:19:38AM +0200, Michal Hocko wrote:
> On Wed 16-09-20 23:43:02, Daniel Vetter wrote:
> > I can
> > then figure out whether it's better to risk not spotting issues with
> > call_rcu vs slapping a memalloc_noio_save/restore around all these
> > critical section which force-degrades any allocation to GFP_ATOMIC at
> 
> did you mean memalloc_noreclaim_* here?

Yeah I picked the wrong one of that family of functions.

> > most, but has the risk that we run into code that assumes "GFP_KERNEL
> > never fails for small stuff" and has a decidedly less tested fallback
> > path than rcu code.
> 
> Even if the above then please note that memalloc_noreclaim_* or
> PF_MEMALLOC should be used with an extreme care. Essentially only for
> internal memory reclaimers. It grants access to _all_ the available
> memory so any abuse can be detrimental to the overall system operation.
> Allocation failure in this mode means that we are out of memory and any
> code relying on such an allocation has to carefuly consider failure.
> This is not a random allocation mode.

Agreed, that's why I don't like having these kind of automagic critical
sections. It's a bit a shotgun approach. Paul said that the code would
handle failures, but the problem is that it applies everywhere.

Anyway my understanding is that call_rcu will be reworked and gain a pile
of tricks so that these problems for the callchains leading to call_rcu
all disappear.
-Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-29  9:00                         ` Daniel Vetter
  0 siblings, 0 replies; 282+ messages in thread
From: Daniel Vetter @ 2020-09-29  9:00 UTC (permalink / raw)
  To: Michal Hocko
  Cc: Daniel Vetter, Paul E. McKenney, Juri Lelli, Peter Zijlstra,
	Sebastian Andrzej Siewior, Lai Jiangshan, dri-devel, Ben Segall,
	Linux-MM, open list:KERNEL SELFTEST FRAMEWORK, linux-hexagon,
	Will Deacon, Ingo Molnar, Anton Ivanov, linux-arch,
	Vincent Guittot, Herbert Xu, Brian Cain, Richard Weinberger,
	Russell King, Ard Biesheuvel, David Airlie, Ingo Molnar,
	Geert Uytterhoeven, Mel Gorman, intel-gfx, Matt Turner,
	Valentin Schneider, linux-xtensa, Shuah Khan, Jeff Dike,
	linux-um, Josh Triplett, Steven Rostedt, rcu, linux-m68k,
	Ivan Kokshaysky, Rodrigo Vivi, Thomas Gleixner, Dietmar Eggemann,
	Linux ARM, Richard Henderson, Chris Zankel, Max Filippov,
	Daniel Bristot de Oliveira, LKML, alpha, Mathieu Desnoyers,
	Andrew Morton, Linus Torvalds

On Tue, Sep 29, 2020 at 10:19:38AM +0200, Michal Hocko wrote:
> On Wed 16-09-20 23:43:02, Daniel Vetter wrote:
> > I can
> > then figure out whether it's better to risk not spotting issues with
> > call_rcu vs slapping a memalloc_noio_save/restore around all these
> > critical section which force-degrades any allocation to GFP_ATOMIC at
> 
> did you mean memalloc_noreclaim_* here?

Yeah I picked the wrong one of that family of functions.

> > most, but has the risk that we run into code that assumes "GFP_KERNEL
> > never fails for small stuff" and has a decidedly less tested fallback
> > path than rcu code.
> 
> Even if the above then please note that memalloc_noreclaim_* or
> PF_MEMALLOC should be used with an extreme care. Essentially only for
> internal memory reclaimers. It grants access to _all_ the available
> memory so any abuse can be detrimental to the overall system operation.
> Allocation failure in this mode means that we are out of memory and any
> code relying on such an allocation has to carefuly consider failure.
> This is not a random allocation mode.

Agreed, that's why I don't like having these kind of automagic critical
sections. It's a bit a shotgun approach. Paul said that the code would
handle failures, but the problem is that it applies everywhere.

Anyway my understanding is that call_rcu will be reworked and gain a pile
of tricks so that these problems for the callchains leading to call_rcu
all disappear.
-Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch


^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-29  9:00                         ` Daniel Vetter
  0 siblings, 0 replies; 282+ messages in thread
From: Daniel Vetter @ 2020-09-29  9:00 UTC (permalink / raw)
  To: Michal Hocko
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, Linux-MM,
	open list:KERNEL SELFTEST FRAMEWORK, Will Deacon, Ard Biesheuvel,
	Anton Ivanov, linux-arch, Vincent Guittot, Herbert Xu,
	Brian Cain, linux-hexagon, Russell King, Ingo Molnar,
	David Airlie, Ingo Molnar, Geert Uytterhoeven, Mel Gorman,
	Matt Turner, Valentin Schneider, linux-xtensa, Shuah Khan,
	Paul E. McKenney, Jeff Dike, intel-gfx, alpha, linux-um,
	Josh Triplett, Steven Rostedt, rcu, linux-m68k, Ivan Kokshaysky,
	Rodrigo Vivi, Thomas Gleixner, Dietmar Eggemann, Linux ARM,
	Richard Henderson, Chris Zankel, Max Filippov, Linus Torvalds,
	LKML, Richard Weinberger, Mathieu Desnoyers, Andrew Morton,
	Daniel Bristot de Oliveira

On Tue, Sep 29, 2020 at 10:19:38AM +0200, Michal Hocko wrote:
> On Wed 16-09-20 23:43:02, Daniel Vetter wrote:
> > I can
> > then figure out whether it's better to risk not spotting issues with
> > call_rcu vs slapping a memalloc_noio_save/restore around all these
> > critical section which force-degrades any allocation to GFP_ATOMIC at
> 
> did you mean memalloc_noreclaim_* here?

Yeah I picked the wrong one of that family of functions.

> > most, but has the risk that we run into code that assumes "GFP_KERNEL
> > never fails for small stuff" and has a decidedly less tested fallback
> > path than rcu code.
> 
> Even if the above then please note that memalloc_noreclaim_* or
> PF_MEMALLOC should be used with an extreme care. Essentially only for
> internal memory reclaimers. It grants access to _all_ the available
> memory so any abuse can be detrimental to the overall system operation.
> Allocation failure in this mode means that we are out of memory and any
> code relying on such an allocation has to carefuly consider failure.
> This is not a random allocation mode.

Agreed, that's why I don't like having these kind of automagic critical
sections. It's a bit a shotgun approach. Paul said that the code would
handle failures, but the problem is that it applies everywhere.

Anyway my understanding is that call_rcu will be reworked and gain a pile
of tricks so that these problems for the callchains leading to call_rcu
all disappear.
-Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [Intel-gfx] [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-29  9:00                         ` Daniel Vetter
  0 siblings, 0 replies; 282+ messages in thread
From: Daniel Vetter @ 2020-09-29  9:00 UTC (permalink / raw)
  To: Michal Hocko
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, Linux-MM,
	open list:KERNEL SELFTEST FRAMEWORK, Will Deacon, Ard Biesheuvel,
	Anton Ivanov, linux-arch, Herbert Xu, Brian Cain, linux-hexagon,
	Russell King, Ingo Molnar, David Airlie, Ingo Molnar,
	Geert Uytterhoeven, Mel Gorman, Matt Turner, Valentin Schneider,
	linux-xtensa, Shuah Khan, Paul E. McKenney, Jeff Dike, intel-gfx,
	alpha, linux-um, Josh Triplett, Steven Rostedt, rcu, linux-m68k,
	Ivan Kokshaysky, Thomas Gleixner, Dietmar Eggemann, Linux ARM,
	Richard Henderson, Chris Zankel, Max Filippov, Linus Torvalds,
	LKML, Richard Weinberger, Mathieu Desnoyers, Andrew Morton,
	Daniel Bristot de Oliveira

On Tue, Sep 29, 2020 at 10:19:38AM +0200, Michal Hocko wrote:
> On Wed 16-09-20 23:43:02, Daniel Vetter wrote:
> > I can
> > then figure out whether it's better to risk not spotting issues with
> > call_rcu vs slapping a memalloc_noio_save/restore around all these
> > critical section which force-degrades any allocation to GFP_ATOMIC at
> 
> did you mean memalloc_noreclaim_* here?

Yeah I picked the wrong one of that family of functions.

> > most, but has the risk that we run into code that assumes "GFP_KERNEL
> > never fails for small stuff" and has a decidedly less tested fallback
> > path than rcu code.
> 
> Even if the above then please note that memalloc_noreclaim_* or
> PF_MEMALLOC should be used with an extreme care. Essentially only for
> internal memory reclaimers. It grants access to _all_ the available
> memory so any abuse can be detrimental to the overall system operation.
> Allocation failure in this mode means that we are out of memory and any
> code relying on such an allocation has to carefuly consider failure.
> This is not a random allocation mode.

Agreed, that's why I don't like having these kind of automagic critical
sections. It's a bit a shotgun approach. Paul said that the code would
handle failures, but the problem is that it applies everywhere.

Anyway my understanding is that call_rcu will be reworked and gain a pile
of tricks so that these problems for the callchains leading to call_rcu
all disappear.
-Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-29  9:00                         ` Daniel Vetter
  0 siblings, 0 replies; 282+ messages in thread
From: Daniel Vetter @ 2020-09-29  9:00 UTC (permalink / raw)
  To: Michal Hocko
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, Linux-MM,
	open list:KERNEL SELFTEST FRAMEWORK, Will Deacon, Ard Biesheuvel,
	Anton Ivanov, linux-arch, Herbert Xu, Brian Cain, linux-hexagon,
	Russell King, Ingo Molnar, David Airlie, Ingo Molnar,
	Geert Uytterhoeven, Mel Gorman, Matt Turner, Valentin Schneider

On Tue, Sep 29, 2020 at 10:19:38AM +0200, Michal Hocko wrote:
> On Wed 16-09-20 23:43:02, Daniel Vetter wrote:
> > I can
> > then figure out whether it's better to risk not spotting issues with
> > call_rcu vs slapping a memalloc_noio_save/restore around all these
> > critical section which force-degrades any allocation to GFP_ATOMIC at
> 
> did you mean memalloc_noreclaim_* here?

Yeah I picked the wrong one of that family of functions.

> > most, but has the risk that we run into code that assumes "GFP_KERNEL
> > never fails for small stuff" and has a decidedly less tested fallback
> > path than rcu code.
> 
> Even if the above then please note that memalloc_noreclaim_* or
> PF_MEMALLOC should be used with an extreme care. Essentially only for
> internal memory reclaimers. It grants access to _all_ the available
> memory so any abuse can be detrimental to the overall system operation.
> Allocation failure in this mode means that we are out of memory and any
> code relying on such an allocation has to carefuly consider failure.
> This is not a random allocation mode.

Agreed, that's why I don't like having these kind of automagic critical
sections. It's a bit a shotgun approach. Paul said that the code would
handle failures, but the problem is that it applies everywhere.

Anyway my understanding is that call_rcu will be reworked and gain a pile
of tricks so that these problems for the callchains leading to call_rcu
all disappear.
-Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
  2020-09-29  9:00                         ` Daniel Vetter
                                             ` (2 preceding siblings ...)
  (?)
@ 2020-09-29 14:54                           ` Michal Hocko
  -1 siblings, 0 replies; 282+ messages in thread
From: Michal Hocko @ 2020-09-29 14:54 UTC (permalink / raw)
  To: Daniel Vetter
  Cc: Paul E. McKenney, Juri Lelli, Peter Zijlstra,
	Sebastian Andrzej Siewior, Lai Jiangshan, dri-devel, Ben Segall,
	Linux-MM, open list:KERNEL SELFTEST FRAMEWORK, linux-hexagon,
	Will Deacon, Ingo Molnar, Anton Ivanov, linux-arch,
	Vincent Guittot, Herbert Xu, Brian Cain, Richard Weinberger,
	Russell King, Ard Biesheuvel, David Airlie, Ingo Molnar,
	Geert Uytterhoeven, Mel Gorman, intel-gfx, Matt Turner,
	Valentin Schneider, linux-xtensa, Shuah Khan, Jeff Dike,
	linux-um, Josh Triplett, Steven Rostedt, rcu, linux-m68k,
	Ivan Kokshaysky, Rodrigo Vivi, Thomas Gleixner, Dietmar Eggemann,
	Linux ARM, Richard Henderson, Chris Zankel, Max Filippov,
	Daniel Bristot de Oliveira, LKML, alpha, Mathieu Desnoyers,
	Andrew Morton, Linus Torvalds

On Tue 29-09-20 11:00:03, Daniel Vetter wrote:
> On Tue, Sep 29, 2020 at 10:19:38AM +0200, Michal Hocko wrote:
> > On Wed 16-09-20 23:43:02, Daniel Vetter wrote:
> > > I can
> > > then figure out whether it's better to risk not spotting issues with
> > > call_rcu vs slapping a memalloc_noio_save/restore around all these
> > > critical section which force-degrades any allocation to GFP_ATOMIC at
> > 
> > did you mean memalloc_noreclaim_* here?
> 
> Yeah I picked the wrong one of that family of functions.
> 
> > > most, but has the risk that we run into code that assumes "GFP_KERNEL
> > > never fails for small stuff" and has a decidedly less tested fallback
> > > path than rcu code.
> > 
> > Even if the above then please note that memalloc_noreclaim_* or
> > PF_MEMALLOC should be used with an extreme care. Essentially only for
> > internal memory reclaimers. It grants access to _all_ the available
> > memory so any abuse can be detrimental to the overall system operation.
> > Allocation failure in this mode means that we are out of memory and any
> > code relying on such an allocation has to carefuly consider failure.
> > This is not a random allocation mode.
> 
> Agreed, that's why I don't like having these kind of automagic critical
> sections. It's a bit a shotgun approach. Paul said that the code would
> handle failures, but the problem is that it applies everywhere.

Ohh, in the ideal world we wouldn't need anything like that. But then
the reality fires:
* PF_MEMALLOC (resp memalloc_noreclaim_* for that matter) is primarily used
  to make sure that allocations from inside the memory reclaim - yeah that
  happens - will not recurse.
* PF_MEMALLOC_NO{FS,IO} (resp memalloc_no{fs,io}*) are used to mark no
  fs/io reclaim recursion critical sections because controling that for
  each allocation inside fs transaction (or other sensitive) or IO
  contexts turned out to be unmaintainable and people simply fallen into
  using NOFS/NOIO unconditionally which is causing reclaim imbalance
  problems.
* PF_MEMALLOC_NOCMA (resp memalloc_nocma*) is used for long term pinning
  when CMA pages cannot be pinned because that would break the CMA
  guarantees. Communicating this to all potential allocations during
  pinning is simply unfeasible.

So you are absolutely right that these critical sections with side
effects on all allocations are far from ideal from the API point of view
but they are mostly mirroring a demand for functionality which is
_practically_ impossible to achieve with our current code base. Not that
we couldn't get back to drawing board and come up with a saner thing and
rework the world...
-- 
Michal Hocko
SUSE Labs

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-29 14:54                           ` Michal Hocko
  0 siblings, 0 replies; 282+ messages in thread
From: Michal Hocko @ 2020-09-29 14:54 UTC (permalink / raw)
  To: Daniel Vetter
  Cc: Paul E. McKenney, Juri Lelli, Peter Zijlstra,
	Sebastian Andrzej Siewior, Lai Jiangshan, dri-devel, Ben Segall,
	Linux-MM, open list:KERNEL SELFTEST FRAMEWORK, linux-hexagon,
	Will Deacon, Ingo Molnar, Anton Ivanov, linux-arch,
	Vincent Guittot, Herbert Xu, Brian Cain, Richard Weinberger,
	Russell King, Ard Biesheuvel, David Airlie, Ingo Molnar,
	Geert Uytterhoeven, Mel Gorman, intel-gfx, Matt Turner,
	Valentin Schneider, linux-xtensa, Shuah Khan, Jeff Dike,
	linux-um, Josh Triplett, Steven Rostedt, rcu, linux-m68k,
	Ivan Kokshaysky, Rodrigo Vivi, Thomas Gleixner, Dietmar Eggemann,
	Linux ARM, Richard Henderson, Chris Zankel, Max Filippov,
	Daniel Bristot de Oliveira, LKML, alpha, Mathieu Desnoyers,
	Andrew Morton, Linus Torvalds

On Tue 29-09-20 11:00:03, Daniel Vetter wrote:
> On Tue, Sep 29, 2020 at 10:19:38AM +0200, Michal Hocko wrote:
> > On Wed 16-09-20 23:43:02, Daniel Vetter wrote:
> > > I can
> > > then figure out whether it's better to risk not spotting issues with
> > > call_rcu vs slapping a memalloc_noio_save/restore around all these
> > > critical section which force-degrades any allocation to GFP_ATOMIC at
> > 
> > did you mean memalloc_noreclaim_* here?
> 
> Yeah I picked the wrong one of that family of functions.
> 
> > > most, but has the risk that we run into code that assumes "GFP_KERNEL
> > > never fails for small stuff" and has a decidedly less tested fallback
> > > path than rcu code.
> > 
> > Even if the above then please note that memalloc_noreclaim_* or
> > PF_MEMALLOC should be used with an extreme care. Essentially only for
> > internal memory reclaimers. It grants access to _all_ the available
> > memory so any abuse can be detrimental to the overall system operation.
> > Allocation failure in this mode means that we are out of memory and any
> > code relying on such an allocation has to carefuly consider failure.
> > This is not a random allocation mode.
> 
> Agreed, that's why I don't like having these kind of automagic critical
> sections. It's a bit a shotgun approach. Paul said that the code would
> handle failures, but the problem is that it applies everywhere.

Ohh, in the ideal world we wouldn't need anything like that. But then
the reality fires:
* PF_MEMALLOC (resp memalloc_noreclaim_* for that matter) is primarily used
  to make sure that allocations from inside the memory reclaim - yeah that
  happens - will not recurse.
* PF_MEMALLOC_NO{FS,IO} (resp memalloc_no{fs,io}*) are used to mark no
  fs/io reclaim recursion critical sections because controling that for
  each allocation inside fs transaction (or other sensitive) or IO
  contexts turned out to be unmaintainable and people simply fallen into
  using NOFS/NOIO unconditionally which is causing reclaim imbalance
  problems.
* PF_MEMALLOC_NOCMA (resp memalloc_nocma*) is used for long term pinning
  when CMA pages cannot be pinned because that would break the CMA
  guarantees. Communicating this to all potential allocations during
  pinning is simply unfeasible.

So you are absolutely right that these critical sections with side
effects on all allocations are far from ideal from the API point of view
but they are mostly mirroring a demand for functionality which is
_practically_ impossible to achieve with our current code base. Not that
we couldn't get back to drawing board and come up with a saner thing and
rework the world...
-- 
Michal Hocko
SUSE Labs


^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-29 14:54                           ` Michal Hocko
  0 siblings, 0 replies; 282+ messages in thread
From: Michal Hocko @ 2020-09-29 14:54 UTC (permalink / raw)
  To: Daniel Vetter
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, Linux-MM,
	open list:KERNEL SELFTEST FRAMEWORK, Will Deacon, Ard Biesheuvel,
	Anton Ivanov, linux-arch, Vincent Guittot, Herbert Xu,
	Brian Cain, linux-hexagon, Russell King, Ingo Molnar,
	David Airlie, Ingo Molnar, Geert Uytterhoeven, Mel Gorman,
	Matt Turner, Valentin Schneider, linux-xtensa, Shuah Khan,
	Paul E. McKenney, Jeff Dike, intel-gfx, alpha, linux-um,
	Josh Triplett, Steven Rostedt, rcu, linux-m68k, Ivan Kokshaysky,
	Rodrigo Vivi, Thomas Gleixner, Dietmar Eggemann, Linux ARM,
	Richard Henderson, Chris Zankel, Max Filippov, Linus Torvalds,
	LKML, Richard Weinberger, Mathieu Desnoyers, Andrew Morton,
	Daniel Bristot de Oliveira

On Tue 29-09-20 11:00:03, Daniel Vetter wrote:
> On Tue, Sep 29, 2020 at 10:19:38AM +0200, Michal Hocko wrote:
> > On Wed 16-09-20 23:43:02, Daniel Vetter wrote:
> > > I can
> > > then figure out whether it's better to risk not spotting issues with
> > > call_rcu vs slapping a memalloc_noio_save/restore around all these
> > > critical section which force-degrades any allocation to GFP_ATOMIC at
> > 
> > did you mean memalloc_noreclaim_* here?
> 
> Yeah I picked the wrong one of that family of functions.
> 
> > > most, but has the risk that we run into code that assumes "GFP_KERNEL
> > > never fails for small stuff" and has a decidedly less tested fallback
> > > path than rcu code.
> > 
> > Even if the above then please note that memalloc_noreclaim_* or
> > PF_MEMALLOC should be used with an extreme care. Essentially only for
> > internal memory reclaimers. It grants access to _all_ the available
> > memory so any abuse can be detrimental to the overall system operation.
> > Allocation failure in this mode means that we are out of memory and any
> > code relying on such an allocation has to carefuly consider failure.
> > This is not a random allocation mode.
> 
> Agreed, that's why I don't like having these kind of automagic critical
> sections. It's a bit a shotgun approach. Paul said that the code would
> handle failures, but the problem is that it applies everywhere.

Ohh, in the ideal world we wouldn't need anything like that. But then
the reality fires:
* PF_MEMALLOC (resp memalloc_noreclaim_* for that matter) is primarily used
  to make sure that allocations from inside the memory reclaim - yeah that
  happens - will not recurse.
* PF_MEMALLOC_NO{FS,IO} (resp memalloc_no{fs,io}*) are used to mark no
  fs/io reclaim recursion critical sections because controling that for
  each allocation inside fs transaction (or other sensitive) or IO
  contexts turned out to be unmaintainable and people simply fallen into
  using NOFS/NOIO unconditionally which is causing reclaim imbalance
  problems.
* PF_MEMALLOC_NOCMA (resp memalloc_nocma*) is used for long term pinning
  when CMA pages cannot be pinned because that would break the CMA
  guarantees. Communicating this to all potential allocations during
  pinning is simply unfeasible.

So you are absolutely right that these critical sections with side
effects on all allocations are far from ideal from the API point of view
but they are mostly mirroring a demand for functionality which is
_practically_ impossible to achieve with our current code base. Not that
we couldn't get back to drawing board and come up with a saner thing and
rework the world...
-- 
Michal Hocko
SUSE Labs
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [Intel-gfx] [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-29 14:54                           ` Michal Hocko
  0 siblings, 0 replies; 282+ messages in thread
From: Michal Hocko @ 2020-09-29 14:54 UTC (permalink / raw)
  To: Daniel Vetter
  Cc: Juri Lelli, Peter Zijlstra, Sebastian Andrzej Siewior,
	Lai Jiangshan, dri-devel, Ben Segall, Linux-MM,
	open list:KERNEL SELFTEST FRAMEWORK, Will Deacon, Ard Biesheuvel,
	Anton Ivanov, linux-arch, Herbert Xu, Brian Cain, linux-hexagon,
	Russell King, Ingo Molnar, David Airlie, Ingo Molnar,
	Geert Uytterhoeven, Mel Gorman, Matt Turner, Valentin Schneider,
	linux-xtensa, Shuah Khan, Paul E. McKenney, Jeff Dike, intel-gfx,
	alpha, linux-um, Josh Triplett, Steven Rostedt, rcu, linux-m68k,
	Ivan Kokshaysky, Thomas Gleixner, Dietmar Eggemann, Linux ARM,
	Richard Henderson, Chris Zankel, Max Filippov, Linus Torvalds,
	LKML, Richard Weinberger, Mathieu Desnoyers, Andrew Morton,
	Daniel Bristot de Oliveira

On Tue 29-09-20 11:00:03, Daniel Vetter wrote:
> On Tue, Sep 29, 2020 at 10:19:38AM +0200, Michal Hocko wrote:
> > On Wed 16-09-20 23:43:02, Daniel Vetter wrote:
> > > I can
> > > then figure out whether it's better to risk not spotting issues with
> > > call_rcu vs slapping a memalloc_noio_save/restore around all these
> > > critical section which force-degrades any allocation to GFP_ATOMIC at
> > 
> > did you mean memalloc_noreclaim_* here?
> 
> Yeah I picked the wrong one of that family of functions.
> 
> > > most, but has the risk that we run into code that assumes "GFP_KERNEL
> > > never fails for small stuff" and has a decidedly less tested fallback
> > > path than rcu code.
> > 
> > Even if the above then please note that memalloc_noreclaim_* or
> > PF_MEMALLOC should be used with an extreme care. Essentially only for
> > internal memory reclaimers. It grants access to _all_ the available
> > memory so any abuse can be detrimental to the overall system operation.
> > Allocation failure in this mode means that we are out of memory and any
> > code relying on such an allocation has to carefuly consider failure.
> > This is not a random allocation mode.
> 
> Agreed, that's why I don't like having these kind of automagic critical
> sections. It's a bit a shotgun approach. Paul said that the code would
> handle failures, but the problem is that it applies everywhere.

Ohh, in the ideal world we wouldn't need anything like that. But then
the reality fires:
* PF_MEMALLOC (resp memalloc_noreclaim_* for that matter) is primarily used
  to make sure that allocations from inside the memory reclaim - yeah that
  happens - will not recurse.
* PF_MEMALLOC_NO{FS,IO} (resp memalloc_no{fs,io}*) are used to mark no
  fs/io reclaim recursion critical sections because controling that for
  each allocation inside fs transaction (or other sensitive) or IO
  contexts turned out to be unmaintainable and people simply fallen into
  using NOFS/NOIO unconditionally which is causing reclaim imbalance
  problems.
* PF_MEMALLOC_NOCMA (resp memalloc_nocma*) is used for long term pinning
  when CMA pages cannot be pinned because that would break the CMA
  guarantees. Communicating this to all potential allocations during
  pinning is simply unfeasible.

So you are absolutely right that these critical sections with side
effects on all allocations are far from ideal from the API point of view
but they are mostly mirroring a demand for functionality which is
_practically_ impossible to achieve with our current code base. Not that
we couldn't get back to drawing board and come up with a saner thing and
rework the world...
-- 
Michal Hocko
SUSE Labs
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

^ permalink raw reply	[flat|nested] 282+ messages in thread

* Re: [patch 00/13] preempt: Make preempt count unconditional
@ 2020-09-29 14:54                           ` Michal Hocko
  0 siblings, 0 replies; 282+ messages in thread
From: Michal Hocko @ 2020-09-29 14:54 UTC (permalink / raw)
  To: Daniel Vetter
  Cc: Paul E. McKenney, Juri Lelli, Peter Zijlstra,
	Sebastian Andrzej Siewior, Lai Jiangshan, dri-devel, Ben Segall,
	Linux-MM, open list:KERNEL SELFTEST FRAMEWORK, linux-hexagon,
	Will Deacon, Ingo Molnar, Anton Ivanov, linux-arch,
	Vincent Guittot, Herbert Xu, Brian Cain, Richard Weinberger,
	Russell King

On Tue 29-09-20 11:00:03, Daniel Vetter wrote:
> On Tue, Sep 29, 2020 at 10:19:38AM +0200, Michal Hocko wrote:
> > On Wed 16-09-20 23:43:02, Daniel Vetter wrote:
> > > I can
> > > then figure out whether it's better to risk not spotting issues with
> > > call_rcu vs slapping a memalloc_noio_save/restore around all these
> > > critical section which force-degrades any allocation to GFP_ATOMIC at
> > 
> > did you mean memalloc_noreclaim_* here?
> 
> Yeah I picked the wrong one of that family of functions.
> 
> > > most, but has the risk that we run into code that assumes "GFP_KERNEL
> > > never fails for small stuff" and has a decidedly less tested fallback
> > > path than rcu code.
> > 
> > Even if the above then please note that memalloc_noreclaim_* or
> > PF_MEMALLOC should be used with an extreme care. Essentially only for
> > internal memory reclaimers. It grants access to _all_ the available
> > memory so any abuse can be detrimental to the overall system operation.
> > Allocation failure in this mode means that we are out of memory and any
> > code relying on such an allocation has to carefuly consider failure.
> > This is not a random allocation mode.
> 
> Agreed, that's why I don't like having these kind of automagic critical
> sections. It's a bit a shotgun approach. Paul said that the code would
> handle failures, but the problem is that it applies everywhere.

Ohh, in the ideal world we wouldn't need anything like that. But then
the reality fires:
* PF_MEMALLOC (resp memalloc_noreclaim_* for that matter) is primarily used
  to make sure that allocations from inside the memory reclaim - yeah that
  happens - will not recurse.
* PF_MEMALLOC_NO{FS,IO} (resp memalloc_no{fs,io}*) are used to mark no
  fs/io reclaim recursion critical sections because controling that for
  each allocation inside fs transaction (or other sensitive) or IO
  contexts turned out to be unmaintainable and people simply fallen into
  using NOFS/NOIO unconditionally which is causing reclaim imbalance
  problems.
* PF_MEMALLOC_NOCMA (resp memalloc_nocma*) is used for long term pinning
  when CMA pages cannot be pinned because that would break the CMA
  guarantees. Communicating this to all potential allocations during
  pinning is simply unfeasible.

So you are absolutely right that these critical sections with side
effects on all allocations are far from ideal from the API point of view
but they are mostly mirroring a demand for functionality which is
_practically_ impossible to achieve with our current code base. Not that
we couldn't get back to drawing board and come up with a saner thing and
rework the world...
-- 
Michal Hocko
SUSE Labs

^ permalink raw reply	[flat|nested] 282+ messages in thread

end of thread, other threads:[~2020-09-30  7:52 UTC | newest]

Thread overview: 282+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-09-14 20:42 [patch 00/13] preempt: Make preempt count unconditional Thomas Gleixner
2020-09-14 20:42 ` Thomas Gleixner
2020-09-14 20:42 ` [Intel-gfx] " Thomas Gleixner
2020-09-14 20:42 ` Thomas Gleixner
2020-09-14 20:42 ` Thomas Gleixner
2020-09-14 20:42 ` [patch 01/13] lib/debug: Remove pointless ARCH_NO_PREEMPT dependencies Thomas Gleixner
2020-09-14 20:42   ` Thomas Gleixner
2020-09-14 20:42   ` [Intel-gfx] " Thomas Gleixner
2020-09-14 20:42   ` Thomas Gleixner
2020-09-14 20:42   ` Thomas Gleixner
2020-09-14 20:42 ` [patch 02/13] preempt: Make preempt count unconditional Thomas Gleixner
2020-09-14 20:42   ` Thomas Gleixner
2020-09-14 20:42   ` [Intel-gfx] " Thomas Gleixner
2020-09-14 20:42   ` Thomas Gleixner
2020-09-14 20:42   ` Thomas Gleixner
2020-09-14 20:42 ` [patch 03/13] preempt: Clenaup PREEMPT_COUNT leftovers Thomas Gleixner
2020-09-14 20:42   ` Thomas Gleixner
2020-09-14 20:42   ` [Intel-gfx] " Thomas Gleixner
2020-09-14 20:42   ` Thomas Gleixner
2020-09-14 20:42   ` Thomas Gleixner
2020-09-16 10:56   ` Valentin Schneider
2020-09-16 10:56     ` Valentin Schneider
2020-09-16 10:56     ` [Intel-gfx] " Valentin Schneider
2020-09-16 10:56     ` Valentin Schneider
2020-09-14 20:42 ` [patch 04/13] lockdep: " Thomas Gleixner
2020-09-14 20:42   ` Thomas Gleixner
2020-09-14 20:42   ` [Intel-gfx] " Thomas Gleixner
2020-09-14 20:42   ` Thomas Gleixner
2020-09-14 20:42   ` Thomas Gleixner
2020-09-15 16:11   ` Will Deacon
2020-09-15 16:11     ` Will Deacon
2020-09-15 16:11     ` [Intel-gfx] " Will Deacon
2020-09-15 16:11     ` Will Deacon
2020-09-15 16:11     ` Will Deacon
2020-09-14 20:42 ` [patch 05/13] mm/pagemap: " Thomas Gleixner
2020-09-14 20:42   ` Thomas Gleixner
2020-09-14 20:42   ` [Intel-gfx] " Thomas Gleixner
2020-09-14 20:42   ` Thomas Gleixner
2020-09-14 20:42   ` Thomas Gleixner
2020-09-15  0:52   ` kernel test robot
2020-09-15  0:52     ` kernel test robot
2020-09-15  2:40   ` kernel test robot
2020-09-15  5:28   ` kernel test robot
2020-09-14 20:42 ` [patch 06/13] locking/bitspinlock: " Thomas Gleixner
2020-09-14 20:42   ` Thomas Gleixner
2020-09-14 20:42   ` [Intel-gfx] " Thomas Gleixner
2020-09-14 20:42   ` Thomas Gleixner
2020-09-14 20:42   ` Thomas Gleixner
2020-09-15 16:10   ` Will Deacon
2020-09-15 16:10     ` Will Deacon
2020-09-15 16:10     ` [Intel-gfx] " Will Deacon
2020-09-15 16:10     ` Will Deacon
2020-09-15 16:10     ` Will Deacon
2020-09-14 20:42 ` [patch 07/13] uaccess: " Thomas Gleixner
2020-09-14 20:42   ` Thomas Gleixner
2020-09-14 20:42   ` [Intel-gfx] " Thomas Gleixner
2020-09-14 20:42   ` Thomas Gleixner
2020-09-14 20:42   ` Thomas Gleixner
2020-09-14 20:42 ` [patch 08/13] sched: " Thomas Gleixner
2020-09-14 20:42   ` Thomas Gleixner
2020-09-14 20:42   ` [Intel-gfx] " Thomas Gleixner
2020-09-14 20:42   ` Thomas Gleixner
2020-09-14 20:42   ` Thomas Gleixner
2020-09-16 10:56   ` Valentin Schneider
2020-09-16 10:56     ` Valentin Schneider
2020-09-16 10:56     ` [Intel-gfx] " Valentin Schneider
2020-09-16 10:56     ` Valentin Schneider
2020-09-14 20:42 ` [patch 09/13] ARM: " Thomas Gleixner
2020-09-14 20:42   ` Thomas Gleixner
2020-09-14 20:42   ` [Intel-gfx] " Thomas Gleixner
2020-09-14 20:42   ` Thomas Gleixner
2020-09-14 20:42   ` Thomas Gleixner
2020-09-14 20:42 ` [patch 10/13] xtensa: " Thomas Gleixner
2020-09-14 20:42   ` Thomas Gleixner
2020-09-14 20:42   ` [Intel-gfx] " Thomas Gleixner
2020-09-14 20:42   ` Thomas Gleixner
2020-09-14 20:42   ` Thomas Gleixner
2020-09-14 20:42 ` [patch 11/13] drm/i915: " Thomas Gleixner
2020-09-14 20:42   ` Thomas Gleixner
2020-09-14 20:42   ` [Intel-gfx] " Thomas Gleixner
2020-09-14 20:42   ` Thomas Gleixner
2020-09-14 20:42   ` Thomas Gleixner
2020-09-14 20:42 ` [patch 12/13] rcutorture: " Thomas Gleixner
2020-09-14 20:42   ` Thomas Gleixner
2020-09-14 20:42   ` [Intel-gfx] " Thomas Gleixner
2020-09-14 20:42   ` Thomas Gleixner
2020-09-14 20:42   ` Thomas Gleixner
2020-09-14 20:42 ` [patch 13/13] preempt: Remove PREEMPT_COUNT from Kconfig Thomas Gleixner
2020-09-14 20:42   ` Thomas Gleixner
2020-09-14 20:42   ` [Intel-gfx] " Thomas Gleixner
2020-09-14 20:42   ` Thomas Gleixner
2020-09-14 20:42   ` Thomas Gleixner
2020-09-14 20:54 ` [patch 00/13] preempt: Make preempt count unconditional Steven Rostedt
2020-09-14 20:54   ` Steven Rostedt
2020-09-14 20:54   ` [Intel-gfx] " Steven Rostedt
2020-09-14 20:54   ` Steven Rostedt
2020-09-14 20:54   ` Steven Rostedt
2020-09-14 20:59 ` Linus Torvalds
2020-09-14 20:59   ` Linus Torvalds
2020-09-14 20:59   ` [Intel-gfx] " Linus Torvalds
2020-09-14 20:59   ` Linus Torvalds
2020-09-14 20:59   ` Linus Torvalds
2020-09-14 21:55   ` Thomas Gleixner
2020-09-14 21:55     ` Thomas Gleixner
2020-09-14 21:55     ` Thomas Gleixner
2020-09-14 21:55     ` [Intel-gfx] " Thomas Gleixner
2020-09-14 21:55     ` Thomas Gleixner
2020-09-14 21:55     ` Thomas Gleixner
2020-09-14 22:24     ` Linus Torvalds
2020-09-14 22:24       ` Linus Torvalds
2020-09-14 22:24       ` [Intel-gfx] " Linus Torvalds
2020-09-14 22:24       ` Linus Torvalds
2020-09-14 22:37       ` Linus Torvalds
2020-09-14 22:37         ` Linus Torvalds
2020-09-14 22:37         ` [Intel-gfx] " Linus Torvalds
2020-09-14 22:37         ` Linus Torvalds
2020-09-15  3:21         ` [PATCH] crypto: lib/chacha20poly1305 - Set SG_MITER_ATOMIC unconditionally Herbert Xu
2020-09-15  3:21           ` Herbert Xu
2020-09-15  3:21           ` Herbert Xu
2020-09-15  3:21           ` [Intel-gfx] " Herbert Xu
2020-09-15  3:21           ` Herbert Xu
2020-09-15  3:30         ` Herbert Xu
2020-09-15  6:03           ` Ard Biesheuvel
2020-09-15  6:40             ` Herbert Xu
2020-09-15  6:45           ` Linus Torvalds
2020-09-15  6:55             ` Linus Torvalds
2020-09-15  7:05               ` Herbert Xu
2020-09-15  7:10                 ` Ard Biesheuvel
2020-09-15  9:34                 ` Thomas Gleixner
2020-09-15 10:02                   ` Ard Biesheuvel
2020-09-15 10:05                     ` Herbert Xu
2020-09-15 10:08                       ` Ard Biesheuvel
2020-09-15 10:10                         ` Herbert Xu
2020-09-15 19:04                           ` Thomas Gleixner
2020-09-15  7:08               ` Ard Biesheuvel
2020-09-15  6:20         ` [patch 00/13] preempt: Make preempt count unconditional Ard Biesheuvel
2020-09-15  6:20           ` Ard Biesheuvel
2020-09-15  6:20           ` [Intel-gfx] " Ard Biesheuvel
2020-09-15  6:20           ` Ard Biesheuvel
2020-09-15  6:20           ` Ard Biesheuvel
2020-09-15  6:22           ` Herbert Xu
2020-09-15  6:22             ` Herbert Xu
2020-09-15  6:22             ` Herbert Xu
2020-09-15  6:22             ` [Intel-gfx] " Herbert Xu
2020-09-15  6:22             ` Herbert Xu
2020-09-15  6:39             ` Linus Torvalds
2020-09-15  6:39               ` Linus Torvalds
2020-09-15  6:39               ` [Intel-gfx] " Linus Torvalds
2020-09-15  6:39               ` Linus Torvalds
2020-09-15  6:39               ` Linus Torvalds
2020-09-15  7:24               ` Thomas Gleixner
2020-09-15  7:24                 ` Thomas Gleixner
2020-09-15  7:24                 ` [Intel-gfx] " Thomas Gleixner
2020-09-15 17:29                 ` Linus Torvalds
2020-09-15 17:29                   ` Linus Torvalds
2020-09-15 17:29                   ` [Intel-gfx] " Linus Torvalds
2020-09-15 17:29                   ` Linus Torvalds
2020-09-15 17:29                   ` Linus Torvalds
2020-09-15  8:39       ` Thomas Gleixner
2020-09-15  8:39         ` Thomas Gleixner
2020-09-15  8:39         ` Thomas Gleixner
2020-09-15  8:39         ` [Intel-gfx] " Thomas Gleixner
2020-09-15  8:39         ` Thomas Gleixner
2020-09-15 17:35         ` Linus Torvalds
2020-09-15 17:35           ` Linus Torvalds
2020-09-15 17:35           ` [Intel-gfx] " Linus Torvalds
2020-09-15 17:35           ` Linus Torvalds
2020-09-15 17:35           ` Linus Torvalds
2020-09-15 19:57           ` Thomas Gleixner
2020-09-15 19:57             ` Thomas Gleixner
2020-09-15 19:57             ` Thomas Gleixner
2020-09-15 19:57             ` [Intel-gfx] " Thomas Gleixner
2020-09-15 19:57             ` Thomas Gleixner
2020-09-15 19:57             ` Thomas Gleixner
2020-09-16 18:34             ` Linus Torvalds
2020-09-16 18:34               ` Linus Torvalds
2020-09-16 18:34               ` [Intel-gfx] " Linus Torvalds
2020-09-16 18:34               ` Linus Torvalds
2020-09-16 18:34               ` Linus Torvalds
2020-09-16  7:37           ` Daniel Vetter
2020-09-16  7:37             ` Daniel Vetter
2020-09-16  7:37             ` [Intel-gfx] " Daniel Vetter
2020-09-16  7:37             ` Daniel Vetter
2020-09-16  7:37             ` Daniel Vetter
2020-09-16 15:29             ` Paul E. McKenney
2020-09-16 15:29               ` Paul E. McKenney
2020-09-16 15:29               ` [Intel-gfx] " Paul E. McKenney
2020-09-16 15:29               ` Paul E. McKenney
2020-09-16 15:29               ` Paul E. McKenney
2020-09-16 18:32               ` Linus Torvalds
2020-09-16 18:32                 ` Linus Torvalds
2020-09-16 18:32                 ` [Intel-gfx] " Linus Torvalds
2020-09-16 18:32                 ` Linus Torvalds
2020-09-16 18:32                 ` Linus Torvalds
2020-09-16 20:43                 ` Paul E. McKenney
2020-09-16 20:43                   ` Paul E. McKenney
2020-09-16 20:43                   ` [Intel-gfx] " Paul E. McKenney
2020-09-16 20:43                   ` Paul E. McKenney
2020-09-16 20:43                   ` Paul E. McKenney
2020-09-17  6:38                 ` Ard Biesheuvel
2020-09-17  6:38                   ` Ard Biesheuvel
2020-09-17  6:38                   ` [Intel-gfx] " Ard Biesheuvel
2020-09-17  6:38                   ` Ard Biesheuvel
2020-09-17  6:38                   ` Ard Biesheuvel
2020-09-16 20:29               ` Daniel Vetter
2020-09-16 20:29                 ` Daniel Vetter
2020-09-16 20:29                 ` [Intel-gfx] " Daniel Vetter
2020-09-16 20:29                 ` Daniel Vetter
2020-09-16 20:29                 ` Daniel Vetter
2020-09-16 20:58                 ` Paul E. McKenney
2020-09-16 20:58                   ` Paul E. McKenney
2020-09-16 20:58                   ` [Intel-gfx] " Paul E. McKenney
2020-09-16 20:58                   ` Paul E. McKenney
2020-09-16 20:58                   ` Paul E. McKenney
2020-09-16 21:43                   ` Daniel Vetter
2020-09-16 21:43                     ` Daniel Vetter
2020-09-16 21:43                     ` [Intel-gfx] " Daniel Vetter
2020-09-16 21:43                     ` Daniel Vetter
2020-09-16 21:43                     ` Daniel Vetter
2020-09-16 22:39                     ` Paul E. McKenney
2020-09-16 22:39                       ` Paul E. McKenney
2020-09-16 22:39                       ` [Intel-gfx] " Paul E. McKenney
2020-09-16 22:39                       ` Paul E. McKenney
2020-09-16 22:39                       ` Paul E. McKenney
2020-09-17  7:52                       ` Daniel Vetter
2020-09-17  7:52                         ` Daniel Vetter
2020-09-17  7:52                         ` [Intel-gfx] " Daniel Vetter
2020-09-17  7:52                         ` Daniel Vetter
2020-09-17  7:52                         ` Daniel Vetter
2020-09-17 16:28                         ` Paul E. McKenney
2020-09-17 16:28                           ` Paul E. McKenney
2020-09-17 16:28                           ` [Intel-gfx] " Paul E. McKenney
2020-09-17 16:28                           ` Paul E. McKenney
2020-09-17 16:28                           ` Paul E. McKenney
2020-09-29  8:19                     ` Michal Hocko
2020-09-29  8:23                       ` [Intel-gfx] " Michal Hocko
2020-09-29  8:23                       ` Michal Hocko
2020-09-29  8:23                       ` Michal Hocko
2020-09-29  8:23                       ` Michal Hocko
2020-09-29  8:21                       ` [Intel-gfx] " Michal Hocko
2020-09-29  8:21                       ` Michal Hocko
2020-09-29  8:21                       ` Michal Hocko
2020-09-29  8:21                       ` Michal Hocko
2020-09-29  8:20                       ` [Intel-gfx] " Michal Hocko
2020-09-29  8:20                       ` Michal Hocko
2020-09-29  8:20                       ` Michal Hocko
2020-09-29  8:20                       ` Michal Hocko
2020-09-29  8:19                       ` [Intel-gfx] " Michal Hocko
2020-09-29  8:19                       ` Michal Hocko
2020-09-29  8:19                       ` Michal Hocko
2020-09-29  8:19                       ` Michal Hocko
2020-09-29  8:19                       ` Michal Hocko
2020-09-29  8:19                       ` [Intel-gfx] " Michal Hocko
2020-09-29  8:19                       ` Michal Hocko
2020-09-29  8:19                       ` Michal Hocko
2020-09-29  9:00                       ` Daniel Vetter
2020-09-29  9:00                         ` Daniel Vetter
2020-09-29  9:00                         ` [Intel-gfx] " Daniel Vetter
2020-09-29  9:00                         ` Daniel Vetter
2020-09-29  9:00                         ` Daniel Vetter
2020-09-29 14:54                         ` Michal Hocko
2020-09-29 14:54                           ` Michal Hocko
2020-09-29 14:54                           ` [Intel-gfx] " Michal Hocko
2020-09-29 14:54                           ` Michal Hocko
2020-09-29 14:54                           ` Michal Hocko
2020-09-16 19:23     ` Matthew Wilcox
2020-09-16 19:23       ` Matthew Wilcox
2020-09-16 19:23       ` [Intel-gfx] " Matthew Wilcox
2020-09-16 19:23       ` Matthew Wilcox
2020-09-16 19:23       ` Matthew Wilcox
2020-09-16 20:48       ` Paul E. McKenney
2020-09-16 20:48         ` Paul E. McKenney
2020-09-16 20:48         ` [Intel-gfx] " Paul E. McKenney
2020-09-16 20:48         ` Paul E. McKenney
2020-09-16 20:48         ` Paul E. McKenney
2020-09-15 17:25   ` Paul E. McKenney
2020-09-15 17:25     ` Paul E. McKenney
2020-09-15 17:25     ` [Intel-gfx] " Paul E. McKenney
2020-09-15 17:25     ` Paul E. McKenney
2020-09-15 17:25     ` Paul E. McKenney
2020-09-15 17:25     ` Paul E. McKenney
2020-09-14 22:01 ` [Intel-gfx] ✗ Fi.CI.BUILD: failure for " Patchwork

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.