* kernel/futex/requeue.c:462 futex_requeue() warn: bitwise AND condition is false here
@ 2021-11-17 17:12 kernel test robot
0 siblings, 0 replies; 2+ messages in thread
From: kernel test robot @ 2021-11-17 17:12 UTC (permalink / raw)
To: kbuild
[-- Attachment #1: Type: text/plain, Size: 26814 bytes --]
CC: kbuild-all(a)lists.01.org
CC: linux-kernel(a)vger.kernel.org
TO: Peter Zijlstra <peterz@infradead.org>
CC: "André Almeida" <andrealmeid@collabora.com>
tree: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git master
head: ee1703cda8dc777e937dec172da55beaf1a74919
commit: e5c6828493b5fa6a3c4606b43e80ab6c5ec1111f futex: Split out requeue
date: 6 weeks ago
:::::: branch date: 26 minutes ago
:::::: commit date: 6 weeks ago
config: m68k-randconfig-m031-20211117 (attached as .config)
compiler: m68k-linux-gcc (GCC) 11.2.0
If you fix the issue, kindly add following tag as appropriate
Reported-by: kernel test robot <lkp@intel.com>
Reported-by: Dan Carpenter <dan.carpenter@oracle.com>
smatch warnings:
kernel/futex/requeue.c:462 futex_requeue() warn: bitwise AND condition is false here
vim +462 kernel/futex/requeue.c
e5c6828493b5fa Peter Zijlstra 2021-09-23 345
e5c6828493b5fa Peter Zijlstra 2021-09-23 346 /**
e5c6828493b5fa Peter Zijlstra 2021-09-23 347 * futex_requeue() - Requeue waiters from uaddr1 to uaddr2
e5c6828493b5fa Peter Zijlstra 2021-09-23 348 * @uaddr1: source futex user address
e5c6828493b5fa Peter Zijlstra 2021-09-23 349 * @flags: futex flags (FLAGS_SHARED, etc.)
e5c6828493b5fa Peter Zijlstra 2021-09-23 350 * @uaddr2: target futex user address
e5c6828493b5fa Peter Zijlstra 2021-09-23 351 * @nr_wake: number of waiters to wake (must be 1 for requeue_pi)
e5c6828493b5fa Peter Zijlstra 2021-09-23 352 * @nr_requeue: number of waiters to requeue (0-INT_MAX)
e5c6828493b5fa Peter Zijlstra 2021-09-23 353 * @cmpval: @uaddr1 expected value (or %NULL)
e5c6828493b5fa Peter Zijlstra 2021-09-23 354 * @requeue_pi: if we are attempting to requeue from a non-pi futex to a
e5c6828493b5fa Peter Zijlstra 2021-09-23 355 * pi futex (pi to pi requeue is not supported)
e5c6828493b5fa Peter Zijlstra 2021-09-23 356 *
e5c6828493b5fa Peter Zijlstra 2021-09-23 357 * Requeue waiters on uaddr1 to uaddr2. In the requeue_pi case, try to acquire
e5c6828493b5fa Peter Zijlstra 2021-09-23 358 * uaddr2 atomically on behalf of the top waiter.
e5c6828493b5fa Peter Zijlstra 2021-09-23 359 *
e5c6828493b5fa Peter Zijlstra 2021-09-23 360 * Return:
e5c6828493b5fa Peter Zijlstra 2021-09-23 361 * - >=0 - on success, the number of tasks requeued or woken;
e5c6828493b5fa Peter Zijlstra 2021-09-23 362 * - <0 - on error
e5c6828493b5fa Peter Zijlstra 2021-09-23 363 */
e5c6828493b5fa Peter Zijlstra 2021-09-23 364 int futex_requeue(u32 __user *uaddr1, unsigned int flags, u32 __user *uaddr2,
e5c6828493b5fa Peter Zijlstra 2021-09-23 365 int nr_wake, int nr_requeue, u32 *cmpval, int requeue_pi)
e5c6828493b5fa Peter Zijlstra 2021-09-23 366 {
e5c6828493b5fa Peter Zijlstra 2021-09-23 367 union futex_key key1 = FUTEX_KEY_INIT, key2 = FUTEX_KEY_INIT;
e5c6828493b5fa Peter Zijlstra 2021-09-23 368 int task_count = 0, ret;
e5c6828493b5fa Peter Zijlstra 2021-09-23 369 struct futex_pi_state *pi_state = NULL;
e5c6828493b5fa Peter Zijlstra 2021-09-23 370 struct futex_hash_bucket *hb1, *hb2;
e5c6828493b5fa Peter Zijlstra 2021-09-23 371 struct futex_q *this, *next;
e5c6828493b5fa Peter Zijlstra 2021-09-23 372 DEFINE_WAKE_Q(wake_q);
e5c6828493b5fa Peter Zijlstra 2021-09-23 373
e5c6828493b5fa Peter Zijlstra 2021-09-23 374 if (nr_wake < 0 || nr_requeue < 0)
e5c6828493b5fa Peter Zijlstra 2021-09-23 375 return -EINVAL;
e5c6828493b5fa Peter Zijlstra 2021-09-23 376
e5c6828493b5fa Peter Zijlstra 2021-09-23 377 /*
e5c6828493b5fa Peter Zijlstra 2021-09-23 378 * When PI not supported: return -ENOSYS if requeue_pi is true,
e5c6828493b5fa Peter Zijlstra 2021-09-23 379 * consequently the compiler knows requeue_pi is always false past
e5c6828493b5fa Peter Zijlstra 2021-09-23 380 * this point which will optimize away all the conditional code
e5c6828493b5fa Peter Zijlstra 2021-09-23 381 * further down.
e5c6828493b5fa Peter Zijlstra 2021-09-23 382 */
e5c6828493b5fa Peter Zijlstra 2021-09-23 383 if (!IS_ENABLED(CONFIG_FUTEX_PI) && requeue_pi)
e5c6828493b5fa Peter Zijlstra 2021-09-23 384 return -ENOSYS;
e5c6828493b5fa Peter Zijlstra 2021-09-23 385
e5c6828493b5fa Peter Zijlstra 2021-09-23 386 if (requeue_pi) {
e5c6828493b5fa Peter Zijlstra 2021-09-23 387 /*
e5c6828493b5fa Peter Zijlstra 2021-09-23 388 * Requeue PI only works on two distinct uaddrs. This
e5c6828493b5fa Peter Zijlstra 2021-09-23 389 * check is only valid for private futexes. See below.
e5c6828493b5fa Peter Zijlstra 2021-09-23 390 */
e5c6828493b5fa Peter Zijlstra 2021-09-23 391 if (uaddr1 == uaddr2)
e5c6828493b5fa Peter Zijlstra 2021-09-23 392 return -EINVAL;
e5c6828493b5fa Peter Zijlstra 2021-09-23 393
e5c6828493b5fa Peter Zijlstra 2021-09-23 394 /*
e5c6828493b5fa Peter Zijlstra 2021-09-23 395 * futex_requeue() allows the caller to define the number
e5c6828493b5fa Peter Zijlstra 2021-09-23 396 * of waiters to wake up via the @nr_wake argument. With
e5c6828493b5fa Peter Zijlstra 2021-09-23 397 * REQUEUE_PI, waking up more than one waiter is creating
e5c6828493b5fa Peter Zijlstra 2021-09-23 398 * more problems than it solves. Waking up a waiter makes
e5c6828493b5fa Peter Zijlstra 2021-09-23 399 * only sense if the PI futex @uaddr2 is uncontended as
e5c6828493b5fa Peter Zijlstra 2021-09-23 400 * this allows the requeue code to acquire the futex
e5c6828493b5fa Peter Zijlstra 2021-09-23 401 * @uaddr2 before waking the waiter. The waiter can then
e5c6828493b5fa Peter Zijlstra 2021-09-23 402 * return to user space without further action. A secondary
e5c6828493b5fa Peter Zijlstra 2021-09-23 403 * wakeup would just make the futex_wait_requeue_pi()
e5c6828493b5fa Peter Zijlstra 2021-09-23 404 * handling more complex, because that code would have to
e5c6828493b5fa Peter Zijlstra 2021-09-23 405 * look up pi_state and do more or less all the handling
e5c6828493b5fa Peter Zijlstra 2021-09-23 406 * which the requeue code has to do for the to be requeued
e5c6828493b5fa Peter Zijlstra 2021-09-23 407 * waiters. So restrict the number of waiters to wake to
e5c6828493b5fa Peter Zijlstra 2021-09-23 408 * one, and only wake it up when the PI futex is
e5c6828493b5fa Peter Zijlstra 2021-09-23 409 * uncontended. Otherwise requeue it and let the unlock of
e5c6828493b5fa Peter Zijlstra 2021-09-23 410 * the PI futex handle the wakeup.
e5c6828493b5fa Peter Zijlstra 2021-09-23 411 *
e5c6828493b5fa Peter Zijlstra 2021-09-23 412 * All REQUEUE_PI users, e.g. pthread_cond_signal() and
e5c6828493b5fa Peter Zijlstra 2021-09-23 413 * pthread_cond_broadcast() must use nr_wake=1.
e5c6828493b5fa Peter Zijlstra 2021-09-23 414 */
e5c6828493b5fa Peter Zijlstra 2021-09-23 415 if (nr_wake != 1)
e5c6828493b5fa Peter Zijlstra 2021-09-23 416 return -EINVAL;
e5c6828493b5fa Peter Zijlstra 2021-09-23 417
e5c6828493b5fa Peter Zijlstra 2021-09-23 418 /*
e5c6828493b5fa Peter Zijlstra 2021-09-23 419 * requeue_pi requires a pi_state, try to allocate it now
e5c6828493b5fa Peter Zijlstra 2021-09-23 420 * without any locks in case it fails.
e5c6828493b5fa Peter Zijlstra 2021-09-23 421 */
e5c6828493b5fa Peter Zijlstra 2021-09-23 422 if (refill_pi_state_cache())
e5c6828493b5fa Peter Zijlstra 2021-09-23 423 return -ENOMEM;
e5c6828493b5fa Peter Zijlstra 2021-09-23 424 }
e5c6828493b5fa Peter Zijlstra 2021-09-23 425
e5c6828493b5fa Peter Zijlstra 2021-09-23 426 retry:
e5c6828493b5fa Peter Zijlstra 2021-09-23 427 ret = get_futex_key(uaddr1, flags & FLAGS_SHARED, &key1, FUTEX_READ);
e5c6828493b5fa Peter Zijlstra 2021-09-23 428 if (unlikely(ret != 0))
e5c6828493b5fa Peter Zijlstra 2021-09-23 429 return ret;
e5c6828493b5fa Peter Zijlstra 2021-09-23 430 ret = get_futex_key(uaddr2, flags & FLAGS_SHARED, &key2,
e5c6828493b5fa Peter Zijlstra 2021-09-23 431 requeue_pi ? FUTEX_WRITE : FUTEX_READ);
e5c6828493b5fa Peter Zijlstra 2021-09-23 432 if (unlikely(ret != 0))
e5c6828493b5fa Peter Zijlstra 2021-09-23 433 return ret;
e5c6828493b5fa Peter Zijlstra 2021-09-23 434
e5c6828493b5fa Peter Zijlstra 2021-09-23 435 /*
e5c6828493b5fa Peter Zijlstra 2021-09-23 436 * The check above which compares uaddrs is not sufficient for
e5c6828493b5fa Peter Zijlstra 2021-09-23 437 * shared futexes. We need to compare the keys:
e5c6828493b5fa Peter Zijlstra 2021-09-23 438 */
e5c6828493b5fa Peter Zijlstra 2021-09-23 439 if (requeue_pi && futex_match(&key1, &key2))
e5c6828493b5fa Peter Zijlstra 2021-09-23 440 return -EINVAL;
e5c6828493b5fa Peter Zijlstra 2021-09-23 441
e5c6828493b5fa Peter Zijlstra 2021-09-23 442 hb1 = futex_hash(&key1);
e5c6828493b5fa Peter Zijlstra 2021-09-23 443 hb2 = futex_hash(&key2);
e5c6828493b5fa Peter Zijlstra 2021-09-23 444
e5c6828493b5fa Peter Zijlstra 2021-09-23 445 retry_private:
e5c6828493b5fa Peter Zijlstra 2021-09-23 446 futex_hb_waiters_inc(hb2);
e5c6828493b5fa Peter Zijlstra 2021-09-23 447 double_lock_hb(hb1, hb2);
e5c6828493b5fa Peter Zijlstra 2021-09-23 448
e5c6828493b5fa Peter Zijlstra 2021-09-23 449 if (likely(cmpval != NULL)) {
e5c6828493b5fa Peter Zijlstra 2021-09-23 450 u32 curval;
e5c6828493b5fa Peter Zijlstra 2021-09-23 451
e5c6828493b5fa Peter Zijlstra 2021-09-23 452 ret = futex_get_value_locked(&curval, uaddr1);
e5c6828493b5fa Peter Zijlstra 2021-09-23 453
e5c6828493b5fa Peter Zijlstra 2021-09-23 454 if (unlikely(ret)) {
e5c6828493b5fa Peter Zijlstra 2021-09-23 455 double_unlock_hb(hb1, hb2);
e5c6828493b5fa Peter Zijlstra 2021-09-23 456 futex_hb_waiters_dec(hb2);
e5c6828493b5fa Peter Zijlstra 2021-09-23 457
e5c6828493b5fa Peter Zijlstra 2021-09-23 458 ret = get_user(curval, uaddr1);
e5c6828493b5fa Peter Zijlstra 2021-09-23 459 if (ret)
e5c6828493b5fa Peter Zijlstra 2021-09-23 460 return ret;
e5c6828493b5fa Peter Zijlstra 2021-09-23 461
e5c6828493b5fa Peter Zijlstra 2021-09-23 @462 if (!(flags & FLAGS_SHARED))
e5c6828493b5fa Peter Zijlstra 2021-09-23 463 goto retry_private;
e5c6828493b5fa Peter Zijlstra 2021-09-23 464
e5c6828493b5fa Peter Zijlstra 2021-09-23 465 goto retry;
e5c6828493b5fa Peter Zijlstra 2021-09-23 466 }
e5c6828493b5fa Peter Zijlstra 2021-09-23 467 if (curval != *cmpval) {
e5c6828493b5fa Peter Zijlstra 2021-09-23 468 ret = -EAGAIN;
e5c6828493b5fa Peter Zijlstra 2021-09-23 469 goto out_unlock;
e5c6828493b5fa Peter Zijlstra 2021-09-23 470 }
e5c6828493b5fa Peter Zijlstra 2021-09-23 471 }
e5c6828493b5fa Peter Zijlstra 2021-09-23 472
e5c6828493b5fa Peter Zijlstra 2021-09-23 473 if (requeue_pi) {
e5c6828493b5fa Peter Zijlstra 2021-09-23 474 struct task_struct *exiting = NULL;
e5c6828493b5fa Peter Zijlstra 2021-09-23 475
e5c6828493b5fa Peter Zijlstra 2021-09-23 476 /*
e5c6828493b5fa Peter Zijlstra 2021-09-23 477 * Attempt to acquire uaddr2 and wake the top waiter. If we
e5c6828493b5fa Peter Zijlstra 2021-09-23 478 * intend to requeue waiters, force setting the FUTEX_WAITERS
e5c6828493b5fa Peter Zijlstra 2021-09-23 479 * bit. We force this here where we are able to easily handle
e5c6828493b5fa Peter Zijlstra 2021-09-23 480 * faults rather in the requeue loop below.
e5c6828493b5fa Peter Zijlstra 2021-09-23 481 *
e5c6828493b5fa Peter Zijlstra 2021-09-23 482 * Updates topwaiter::requeue_state if a top waiter exists.
e5c6828493b5fa Peter Zijlstra 2021-09-23 483 */
e5c6828493b5fa Peter Zijlstra 2021-09-23 484 ret = futex_proxy_trylock_atomic(uaddr2, hb1, hb2, &key1,
e5c6828493b5fa Peter Zijlstra 2021-09-23 485 &key2, &pi_state,
e5c6828493b5fa Peter Zijlstra 2021-09-23 486 &exiting, nr_requeue);
e5c6828493b5fa Peter Zijlstra 2021-09-23 487
e5c6828493b5fa Peter Zijlstra 2021-09-23 488 /*
e5c6828493b5fa Peter Zijlstra 2021-09-23 489 * At this point the top_waiter has either taken uaddr2 or
e5c6828493b5fa Peter Zijlstra 2021-09-23 490 * is waiting on it. In both cases pi_state has been
e5c6828493b5fa Peter Zijlstra 2021-09-23 491 * established and an initial refcount on it. In case of an
e5c6828493b5fa Peter Zijlstra 2021-09-23 492 * error there's nothing.
e5c6828493b5fa Peter Zijlstra 2021-09-23 493 *
e5c6828493b5fa Peter Zijlstra 2021-09-23 494 * The top waiter's requeue_state is up to date:
e5c6828493b5fa Peter Zijlstra 2021-09-23 495 *
e5c6828493b5fa Peter Zijlstra 2021-09-23 496 * - If the lock was acquired atomically (ret == 1), then
e5c6828493b5fa Peter Zijlstra 2021-09-23 497 * the state is Q_REQUEUE_PI_LOCKED.
e5c6828493b5fa Peter Zijlstra 2021-09-23 498 *
e5c6828493b5fa Peter Zijlstra 2021-09-23 499 * The top waiter has been dequeued and woken up and can
e5c6828493b5fa Peter Zijlstra 2021-09-23 500 * return to user space immediately. The kernel/user
e5c6828493b5fa Peter Zijlstra 2021-09-23 501 * space state is consistent. In case that there must be
e5c6828493b5fa Peter Zijlstra 2021-09-23 502 * more waiters requeued the WAITERS bit in the user
e5c6828493b5fa Peter Zijlstra 2021-09-23 503 * space futex is set so the top waiter task has to go
e5c6828493b5fa Peter Zijlstra 2021-09-23 504 * into the syscall slowpath to unlock the futex. This
e5c6828493b5fa Peter Zijlstra 2021-09-23 505 * will block until this requeue operation has been
e5c6828493b5fa Peter Zijlstra 2021-09-23 506 * completed and the hash bucket locks have been
e5c6828493b5fa Peter Zijlstra 2021-09-23 507 * dropped.
e5c6828493b5fa Peter Zijlstra 2021-09-23 508 *
e5c6828493b5fa Peter Zijlstra 2021-09-23 509 * - If the trylock failed with an error (ret < 0) then
e5c6828493b5fa Peter Zijlstra 2021-09-23 510 * the state is either Q_REQUEUE_PI_NONE, i.e. "nothing
e5c6828493b5fa Peter Zijlstra 2021-09-23 511 * happened", or Q_REQUEUE_PI_IGNORE when there was an
e5c6828493b5fa Peter Zijlstra 2021-09-23 512 * interleaved early wakeup.
e5c6828493b5fa Peter Zijlstra 2021-09-23 513 *
e5c6828493b5fa Peter Zijlstra 2021-09-23 514 * - If the trylock did not succeed (ret == 0) then the
e5c6828493b5fa Peter Zijlstra 2021-09-23 515 * state is either Q_REQUEUE_PI_IN_PROGRESS or
e5c6828493b5fa Peter Zijlstra 2021-09-23 516 * Q_REQUEUE_PI_WAIT if an early wakeup interleaved.
e5c6828493b5fa Peter Zijlstra 2021-09-23 517 * This will be cleaned up in the loop below, which
e5c6828493b5fa Peter Zijlstra 2021-09-23 518 * cannot fail because futex_proxy_trylock_atomic() did
e5c6828493b5fa Peter Zijlstra 2021-09-23 519 * the same sanity checks for requeue_pi as the loop
e5c6828493b5fa Peter Zijlstra 2021-09-23 520 * below does.
e5c6828493b5fa Peter Zijlstra 2021-09-23 521 */
e5c6828493b5fa Peter Zijlstra 2021-09-23 522 switch (ret) {
e5c6828493b5fa Peter Zijlstra 2021-09-23 523 case 0:
e5c6828493b5fa Peter Zijlstra 2021-09-23 524 /* We hold a reference on the pi state. */
e5c6828493b5fa Peter Zijlstra 2021-09-23 525 break;
e5c6828493b5fa Peter Zijlstra 2021-09-23 526
e5c6828493b5fa Peter Zijlstra 2021-09-23 527 case 1:
e5c6828493b5fa Peter Zijlstra 2021-09-23 528 /*
e5c6828493b5fa Peter Zijlstra 2021-09-23 529 * futex_proxy_trylock_atomic() acquired the user space
e5c6828493b5fa Peter Zijlstra 2021-09-23 530 * futex. Adjust task_count.
e5c6828493b5fa Peter Zijlstra 2021-09-23 531 */
e5c6828493b5fa Peter Zijlstra 2021-09-23 532 task_count++;
e5c6828493b5fa Peter Zijlstra 2021-09-23 533 ret = 0;
e5c6828493b5fa Peter Zijlstra 2021-09-23 534 break;
e5c6828493b5fa Peter Zijlstra 2021-09-23 535
e5c6828493b5fa Peter Zijlstra 2021-09-23 536 /*
e5c6828493b5fa Peter Zijlstra 2021-09-23 537 * If the above failed, then pi_state is NULL and
e5c6828493b5fa Peter Zijlstra 2021-09-23 538 * waiter::requeue_state is correct.
e5c6828493b5fa Peter Zijlstra 2021-09-23 539 */
e5c6828493b5fa Peter Zijlstra 2021-09-23 540 case -EFAULT:
e5c6828493b5fa Peter Zijlstra 2021-09-23 541 double_unlock_hb(hb1, hb2);
e5c6828493b5fa Peter Zijlstra 2021-09-23 542 futex_hb_waiters_dec(hb2);
e5c6828493b5fa Peter Zijlstra 2021-09-23 543 ret = fault_in_user_writeable(uaddr2);
e5c6828493b5fa Peter Zijlstra 2021-09-23 544 if (!ret)
e5c6828493b5fa Peter Zijlstra 2021-09-23 545 goto retry;
e5c6828493b5fa Peter Zijlstra 2021-09-23 546 return ret;
e5c6828493b5fa Peter Zijlstra 2021-09-23 547 case -EBUSY:
e5c6828493b5fa Peter Zijlstra 2021-09-23 548 case -EAGAIN:
e5c6828493b5fa Peter Zijlstra 2021-09-23 549 /*
e5c6828493b5fa Peter Zijlstra 2021-09-23 550 * Two reasons for this:
e5c6828493b5fa Peter Zijlstra 2021-09-23 551 * - EBUSY: Owner is exiting and we just wait for the
e5c6828493b5fa Peter Zijlstra 2021-09-23 552 * exit to complete.
e5c6828493b5fa Peter Zijlstra 2021-09-23 553 * - EAGAIN: The user space value changed.
e5c6828493b5fa Peter Zijlstra 2021-09-23 554 */
e5c6828493b5fa Peter Zijlstra 2021-09-23 555 double_unlock_hb(hb1, hb2);
e5c6828493b5fa Peter Zijlstra 2021-09-23 556 futex_hb_waiters_dec(hb2);
e5c6828493b5fa Peter Zijlstra 2021-09-23 557 /*
e5c6828493b5fa Peter Zijlstra 2021-09-23 558 * Handle the case where the owner is in the middle of
e5c6828493b5fa Peter Zijlstra 2021-09-23 559 * exiting. Wait for the exit to complete otherwise
e5c6828493b5fa Peter Zijlstra 2021-09-23 560 * this task might loop forever, aka. live lock.
e5c6828493b5fa Peter Zijlstra 2021-09-23 561 */
e5c6828493b5fa Peter Zijlstra 2021-09-23 562 wait_for_owner_exiting(ret, exiting);
e5c6828493b5fa Peter Zijlstra 2021-09-23 563 cond_resched();
e5c6828493b5fa Peter Zijlstra 2021-09-23 564 goto retry;
e5c6828493b5fa Peter Zijlstra 2021-09-23 565 default:
e5c6828493b5fa Peter Zijlstra 2021-09-23 566 goto out_unlock;
e5c6828493b5fa Peter Zijlstra 2021-09-23 567 }
e5c6828493b5fa Peter Zijlstra 2021-09-23 568 }
e5c6828493b5fa Peter Zijlstra 2021-09-23 569
e5c6828493b5fa Peter Zijlstra 2021-09-23 570 plist_for_each_entry_safe(this, next, &hb1->chain, list) {
e5c6828493b5fa Peter Zijlstra 2021-09-23 571 if (task_count - nr_wake >= nr_requeue)
e5c6828493b5fa Peter Zijlstra 2021-09-23 572 break;
e5c6828493b5fa Peter Zijlstra 2021-09-23 573
e5c6828493b5fa Peter Zijlstra 2021-09-23 574 if (!futex_match(&this->key, &key1))
e5c6828493b5fa Peter Zijlstra 2021-09-23 575 continue;
e5c6828493b5fa Peter Zijlstra 2021-09-23 576
e5c6828493b5fa Peter Zijlstra 2021-09-23 577 /*
e5c6828493b5fa Peter Zijlstra 2021-09-23 578 * FUTEX_WAIT_REQUEUE_PI and FUTEX_CMP_REQUEUE_PI should always
e5c6828493b5fa Peter Zijlstra 2021-09-23 579 * be paired with each other and no other futex ops.
e5c6828493b5fa Peter Zijlstra 2021-09-23 580 *
e5c6828493b5fa Peter Zijlstra 2021-09-23 581 * We should never be requeueing a futex_q with a pi_state,
e5c6828493b5fa Peter Zijlstra 2021-09-23 582 * which is awaiting a futex_unlock_pi().
e5c6828493b5fa Peter Zijlstra 2021-09-23 583 */
e5c6828493b5fa Peter Zijlstra 2021-09-23 584 if ((requeue_pi && !this->rt_waiter) ||
e5c6828493b5fa Peter Zijlstra 2021-09-23 585 (!requeue_pi && this->rt_waiter) ||
e5c6828493b5fa Peter Zijlstra 2021-09-23 586 this->pi_state) {
e5c6828493b5fa Peter Zijlstra 2021-09-23 587 ret = -EINVAL;
e5c6828493b5fa Peter Zijlstra 2021-09-23 588 break;
e5c6828493b5fa Peter Zijlstra 2021-09-23 589 }
e5c6828493b5fa Peter Zijlstra 2021-09-23 590
e5c6828493b5fa Peter Zijlstra 2021-09-23 591 /* Plain futexes just wake or requeue and are done */
e5c6828493b5fa Peter Zijlstra 2021-09-23 592 if (!requeue_pi) {
e5c6828493b5fa Peter Zijlstra 2021-09-23 593 if (++task_count <= nr_wake)
e5c6828493b5fa Peter Zijlstra 2021-09-23 594 futex_wake_mark(&wake_q, this);
e5c6828493b5fa Peter Zijlstra 2021-09-23 595 else
e5c6828493b5fa Peter Zijlstra 2021-09-23 596 requeue_futex(this, hb1, hb2, &key2);
e5c6828493b5fa Peter Zijlstra 2021-09-23 597 continue;
e5c6828493b5fa Peter Zijlstra 2021-09-23 598 }
e5c6828493b5fa Peter Zijlstra 2021-09-23 599
e5c6828493b5fa Peter Zijlstra 2021-09-23 600 /* Ensure we requeue to the expected futex for requeue_pi. */
e5c6828493b5fa Peter Zijlstra 2021-09-23 601 if (!futex_match(this->requeue_pi_key, &key2)) {
e5c6828493b5fa Peter Zijlstra 2021-09-23 602 ret = -EINVAL;
e5c6828493b5fa Peter Zijlstra 2021-09-23 603 break;
e5c6828493b5fa Peter Zijlstra 2021-09-23 604 }
e5c6828493b5fa Peter Zijlstra 2021-09-23 605
e5c6828493b5fa Peter Zijlstra 2021-09-23 606 /*
e5c6828493b5fa Peter Zijlstra 2021-09-23 607 * Requeue nr_requeue waiters and possibly one more in the case
e5c6828493b5fa Peter Zijlstra 2021-09-23 608 * of requeue_pi if we couldn't acquire the lock atomically.
e5c6828493b5fa Peter Zijlstra 2021-09-23 609 *
e5c6828493b5fa Peter Zijlstra 2021-09-23 610 * Prepare the waiter to take the rt_mutex. Take a refcount
e5c6828493b5fa Peter Zijlstra 2021-09-23 611 * on the pi_state and store the pointer in the futex_q
e5c6828493b5fa Peter Zijlstra 2021-09-23 612 * object of the waiter.
e5c6828493b5fa Peter Zijlstra 2021-09-23 613 */
e5c6828493b5fa Peter Zijlstra 2021-09-23 614 get_pi_state(pi_state);
e5c6828493b5fa Peter Zijlstra 2021-09-23 615
e5c6828493b5fa Peter Zijlstra 2021-09-23 616 /* Don't requeue when the waiter is already on the way out. */
e5c6828493b5fa Peter Zijlstra 2021-09-23 617 if (!futex_requeue_pi_prepare(this, pi_state)) {
e5c6828493b5fa Peter Zijlstra 2021-09-23 618 /*
e5c6828493b5fa Peter Zijlstra 2021-09-23 619 * Early woken waiter signaled that it is on the
e5c6828493b5fa Peter Zijlstra 2021-09-23 620 * way out. Drop the pi_state reference and try the
e5c6828493b5fa Peter Zijlstra 2021-09-23 621 * next waiter. @this->pi_state is still NULL.
e5c6828493b5fa Peter Zijlstra 2021-09-23 622 */
e5c6828493b5fa Peter Zijlstra 2021-09-23 623 put_pi_state(pi_state);
e5c6828493b5fa Peter Zijlstra 2021-09-23 624 continue;
e5c6828493b5fa Peter Zijlstra 2021-09-23 625 }
e5c6828493b5fa Peter Zijlstra 2021-09-23 626
e5c6828493b5fa Peter Zijlstra 2021-09-23 627 ret = rt_mutex_start_proxy_lock(&pi_state->pi_mutex,
e5c6828493b5fa Peter Zijlstra 2021-09-23 628 this->rt_waiter,
e5c6828493b5fa Peter Zijlstra 2021-09-23 629 this->task);
e5c6828493b5fa Peter Zijlstra 2021-09-23 630
e5c6828493b5fa Peter Zijlstra 2021-09-23 631 if (ret == 1) {
e5c6828493b5fa Peter Zijlstra 2021-09-23 632 /*
e5c6828493b5fa Peter Zijlstra 2021-09-23 633 * We got the lock. We do neither drop the refcount
e5c6828493b5fa Peter Zijlstra 2021-09-23 634 * on pi_state nor clear this->pi_state because the
e5c6828493b5fa Peter Zijlstra 2021-09-23 635 * waiter needs the pi_state for cleaning up the
e5c6828493b5fa Peter Zijlstra 2021-09-23 636 * user space value. It will drop the refcount
e5c6828493b5fa Peter Zijlstra 2021-09-23 637 * after doing so. this::requeue_state is updated
e5c6828493b5fa Peter Zijlstra 2021-09-23 638 * in the wakeup as well.
e5c6828493b5fa Peter Zijlstra 2021-09-23 639 */
e5c6828493b5fa Peter Zijlstra 2021-09-23 640 requeue_pi_wake_futex(this, &key2, hb2);
e5c6828493b5fa Peter Zijlstra 2021-09-23 641 task_count++;
e5c6828493b5fa Peter Zijlstra 2021-09-23 642 } else if (!ret) {
e5c6828493b5fa Peter Zijlstra 2021-09-23 643 /* Waiter is queued, move it to hb2 */
e5c6828493b5fa Peter Zijlstra 2021-09-23 644 requeue_futex(this, hb1, hb2, &key2);
e5c6828493b5fa Peter Zijlstra 2021-09-23 645 futex_requeue_pi_complete(this, 0);
e5c6828493b5fa Peter Zijlstra 2021-09-23 646 task_count++;
e5c6828493b5fa Peter Zijlstra 2021-09-23 647 } else {
e5c6828493b5fa Peter Zijlstra 2021-09-23 648 /*
e5c6828493b5fa Peter Zijlstra 2021-09-23 649 * rt_mutex_start_proxy_lock() detected a potential
e5c6828493b5fa Peter Zijlstra 2021-09-23 650 * deadlock when we tried to queue that waiter.
e5c6828493b5fa Peter Zijlstra 2021-09-23 651 * Drop the pi_state reference which we took above
e5c6828493b5fa Peter Zijlstra 2021-09-23 652 * and remove the pointer to the state from the
e5c6828493b5fa Peter Zijlstra 2021-09-23 653 * waiters futex_q object.
e5c6828493b5fa Peter Zijlstra 2021-09-23 654 */
e5c6828493b5fa Peter Zijlstra 2021-09-23 655 this->pi_state = NULL;
e5c6828493b5fa Peter Zijlstra 2021-09-23 656 put_pi_state(pi_state);
e5c6828493b5fa Peter Zijlstra 2021-09-23 657 futex_requeue_pi_complete(this, ret);
e5c6828493b5fa Peter Zijlstra 2021-09-23 658 /*
e5c6828493b5fa Peter Zijlstra 2021-09-23 659 * We stop queueing more waiters and let user space
e5c6828493b5fa Peter Zijlstra 2021-09-23 660 * deal with the mess.
e5c6828493b5fa Peter Zijlstra 2021-09-23 661 */
e5c6828493b5fa Peter Zijlstra 2021-09-23 662 break;
e5c6828493b5fa Peter Zijlstra 2021-09-23 663 }
e5c6828493b5fa Peter Zijlstra 2021-09-23 664 }
e5c6828493b5fa Peter Zijlstra 2021-09-23 665
e5c6828493b5fa Peter Zijlstra 2021-09-23 666 /*
e5c6828493b5fa Peter Zijlstra 2021-09-23 667 * We took an extra initial reference to the pi_state in
e5c6828493b5fa Peter Zijlstra 2021-09-23 668 * futex_proxy_trylock_atomic(). We need to drop it here again.
e5c6828493b5fa Peter Zijlstra 2021-09-23 669 */
e5c6828493b5fa Peter Zijlstra 2021-09-23 670 put_pi_state(pi_state);
e5c6828493b5fa Peter Zijlstra 2021-09-23 671
e5c6828493b5fa Peter Zijlstra 2021-09-23 672 out_unlock:
e5c6828493b5fa Peter Zijlstra 2021-09-23 673 double_unlock_hb(hb1, hb2);
e5c6828493b5fa Peter Zijlstra 2021-09-23 674 wake_up_q(&wake_q);
e5c6828493b5fa Peter Zijlstra 2021-09-23 675 futex_hb_waiters_dec(hb2);
e5c6828493b5fa Peter Zijlstra 2021-09-23 676 return ret ? ret : task_count;
e5c6828493b5fa Peter Zijlstra 2021-09-23 677 }
e5c6828493b5fa Peter Zijlstra 2021-09-23 678
---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org
[-- Attachment #2: config.gz --]
[-- Type: application/gzip, Size: 19600 bytes --]
^ permalink raw reply [flat|nested] 2+ messages in thread
* kernel/futex/requeue.c:462 futex_requeue() warn: bitwise AND condition is false here
@ 2021-11-18 22:08 kernel test robot
0 siblings, 0 replies; 2+ messages in thread
From: kernel test robot @ 2021-11-18 22:08 UTC (permalink / raw)
To: kbuild
[-- Attachment #1: Type: text/plain, Size: 26812 bytes --]
CC: kbuild-all(a)lists.01.org
CC: linux-kernel(a)vger.kernel.org
TO: Peter Zijlstra <peterz@infradead.org>
CC: "André Almeida" <andrealmeid@collabora.com>
tree: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git master
head: 42eb8fdac2fc5d62392dcfcf0253753e821a97b0
commit: e5c6828493b5fa6a3c4606b43e80ab6c5ec1111f futex: Split out requeue
date: 6 weeks ago
:::::: branch date: 22 hours ago
:::::: commit date: 6 weeks ago
config: m68k-randconfig-m031-20211117 (attached as .config)
compiler: m68k-linux-gcc (GCC) 11.2.0
If you fix the issue, kindly add following tag as appropriate
Reported-by: kernel test robot <lkp@intel.com>
Reported-by: Dan Carpenter <dan.carpenter@oracle.com>
smatch warnings:
kernel/futex/requeue.c:462 futex_requeue() warn: bitwise AND condition is false here
vim +462 kernel/futex/requeue.c
e5c6828493b5fa Peter Zijlstra 2021-09-23 345
e5c6828493b5fa Peter Zijlstra 2021-09-23 346 /**
e5c6828493b5fa Peter Zijlstra 2021-09-23 347 * futex_requeue() - Requeue waiters from uaddr1 to uaddr2
e5c6828493b5fa Peter Zijlstra 2021-09-23 348 * @uaddr1: source futex user address
e5c6828493b5fa Peter Zijlstra 2021-09-23 349 * @flags: futex flags (FLAGS_SHARED, etc.)
e5c6828493b5fa Peter Zijlstra 2021-09-23 350 * @uaddr2: target futex user address
e5c6828493b5fa Peter Zijlstra 2021-09-23 351 * @nr_wake: number of waiters to wake (must be 1 for requeue_pi)
e5c6828493b5fa Peter Zijlstra 2021-09-23 352 * @nr_requeue: number of waiters to requeue (0-INT_MAX)
e5c6828493b5fa Peter Zijlstra 2021-09-23 353 * @cmpval: @uaddr1 expected value (or %NULL)
e5c6828493b5fa Peter Zijlstra 2021-09-23 354 * @requeue_pi: if we are attempting to requeue from a non-pi futex to a
e5c6828493b5fa Peter Zijlstra 2021-09-23 355 * pi futex (pi to pi requeue is not supported)
e5c6828493b5fa Peter Zijlstra 2021-09-23 356 *
e5c6828493b5fa Peter Zijlstra 2021-09-23 357 * Requeue waiters on uaddr1 to uaddr2. In the requeue_pi case, try to acquire
e5c6828493b5fa Peter Zijlstra 2021-09-23 358 * uaddr2 atomically on behalf of the top waiter.
e5c6828493b5fa Peter Zijlstra 2021-09-23 359 *
e5c6828493b5fa Peter Zijlstra 2021-09-23 360 * Return:
e5c6828493b5fa Peter Zijlstra 2021-09-23 361 * - >=0 - on success, the number of tasks requeued or woken;
e5c6828493b5fa Peter Zijlstra 2021-09-23 362 * - <0 - on error
e5c6828493b5fa Peter Zijlstra 2021-09-23 363 */
e5c6828493b5fa Peter Zijlstra 2021-09-23 364 int futex_requeue(u32 __user *uaddr1, unsigned int flags, u32 __user *uaddr2,
e5c6828493b5fa Peter Zijlstra 2021-09-23 365 int nr_wake, int nr_requeue, u32 *cmpval, int requeue_pi)
e5c6828493b5fa Peter Zijlstra 2021-09-23 366 {
e5c6828493b5fa Peter Zijlstra 2021-09-23 367 union futex_key key1 = FUTEX_KEY_INIT, key2 = FUTEX_KEY_INIT;
e5c6828493b5fa Peter Zijlstra 2021-09-23 368 int task_count = 0, ret;
e5c6828493b5fa Peter Zijlstra 2021-09-23 369 struct futex_pi_state *pi_state = NULL;
e5c6828493b5fa Peter Zijlstra 2021-09-23 370 struct futex_hash_bucket *hb1, *hb2;
e5c6828493b5fa Peter Zijlstra 2021-09-23 371 struct futex_q *this, *next;
e5c6828493b5fa Peter Zijlstra 2021-09-23 372 DEFINE_WAKE_Q(wake_q);
e5c6828493b5fa Peter Zijlstra 2021-09-23 373
e5c6828493b5fa Peter Zijlstra 2021-09-23 374 if (nr_wake < 0 || nr_requeue < 0)
e5c6828493b5fa Peter Zijlstra 2021-09-23 375 return -EINVAL;
e5c6828493b5fa Peter Zijlstra 2021-09-23 376
e5c6828493b5fa Peter Zijlstra 2021-09-23 377 /*
e5c6828493b5fa Peter Zijlstra 2021-09-23 378 * When PI not supported: return -ENOSYS if requeue_pi is true,
e5c6828493b5fa Peter Zijlstra 2021-09-23 379 * consequently the compiler knows requeue_pi is always false past
e5c6828493b5fa Peter Zijlstra 2021-09-23 380 * this point which will optimize away all the conditional code
e5c6828493b5fa Peter Zijlstra 2021-09-23 381 * further down.
e5c6828493b5fa Peter Zijlstra 2021-09-23 382 */
e5c6828493b5fa Peter Zijlstra 2021-09-23 383 if (!IS_ENABLED(CONFIG_FUTEX_PI) && requeue_pi)
e5c6828493b5fa Peter Zijlstra 2021-09-23 384 return -ENOSYS;
e5c6828493b5fa Peter Zijlstra 2021-09-23 385
e5c6828493b5fa Peter Zijlstra 2021-09-23 386 if (requeue_pi) {
e5c6828493b5fa Peter Zijlstra 2021-09-23 387 /*
e5c6828493b5fa Peter Zijlstra 2021-09-23 388 * Requeue PI only works on two distinct uaddrs. This
e5c6828493b5fa Peter Zijlstra 2021-09-23 389 * check is only valid for private futexes. See below.
e5c6828493b5fa Peter Zijlstra 2021-09-23 390 */
e5c6828493b5fa Peter Zijlstra 2021-09-23 391 if (uaddr1 == uaddr2)
e5c6828493b5fa Peter Zijlstra 2021-09-23 392 return -EINVAL;
e5c6828493b5fa Peter Zijlstra 2021-09-23 393
e5c6828493b5fa Peter Zijlstra 2021-09-23 394 /*
e5c6828493b5fa Peter Zijlstra 2021-09-23 395 * futex_requeue() allows the caller to define the number
e5c6828493b5fa Peter Zijlstra 2021-09-23 396 * of waiters to wake up via the @nr_wake argument. With
e5c6828493b5fa Peter Zijlstra 2021-09-23 397 * REQUEUE_PI, waking up more than one waiter is creating
e5c6828493b5fa Peter Zijlstra 2021-09-23 398 * more problems than it solves. Waking up a waiter makes
e5c6828493b5fa Peter Zijlstra 2021-09-23 399 * only sense if the PI futex @uaddr2 is uncontended as
e5c6828493b5fa Peter Zijlstra 2021-09-23 400 * this allows the requeue code to acquire the futex
e5c6828493b5fa Peter Zijlstra 2021-09-23 401 * @uaddr2 before waking the waiter. The waiter can then
e5c6828493b5fa Peter Zijlstra 2021-09-23 402 * return to user space without further action. A secondary
e5c6828493b5fa Peter Zijlstra 2021-09-23 403 * wakeup would just make the futex_wait_requeue_pi()
e5c6828493b5fa Peter Zijlstra 2021-09-23 404 * handling more complex, because that code would have to
e5c6828493b5fa Peter Zijlstra 2021-09-23 405 * look up pi_state and do more or less all the handling
e5c6828493b5fa Peter Zijlstra 2021-09-23 406 * which the requeue code has to do for the to be requeued
e5c6828493b5fa Peter Zijlstra 2021-09-23 407 * waiters. So restrict the number of waiters to wake to
e5c6828493b5fa Peter Zijlstra 2021-09-23 408 * one, and only wake it up when the PI futex is
e5c6828493b5fa Peter Zijlstra 2021-09-23 409 * uncontended. Otherwise requeue it and let the unlock of
e5c6828493b5fa Peter Zijlstra 2021-09-23 410 * the PI futex handle the wakeup.
e5c6828493b5fa Peter Zijlstra 2021-09-23 411 *
e5c6828493b5fa Peter Zijlstra 2021-09-23 412 * All REQUEUE_PI users, e.g. pthread_cond_signal() and
e5c6828493b5fa Peter Zijlstra 2021-09-23 413 * pthread_cond_broadcast() must use nr_wake=1.
e5c6828493b5fa Peter Zijlstra 2021-09-23 414 */
e5c6828493b5fa Peter Zijlstra 2021-09-23 415 if (nr_wake != 1)
e5c6828493b5fa Peter Zijlstra 2021-09-23 416 return -EINVAL;
e5c6828493b5fa Peter Zijlstra 2021-09-23 417
e5c6828493b5fa Peter Zijlstra 2021-09-23 418 /*
e5c6828493b5fa Peter Zijlstra 2021-09-23 419 * requeue_pi requires a pi_state, try to allocate it now
e5c6828493b5fa Peter Zijlstra 2021-09-23 420 * without any locks in case it fails.
e5c6828493b5fa Peter Zijlstra 2021-09-23 421 */
e5c6828493b5fa Peter Zijlstra 2021-09-23 422 if (refill_pi_state_cache())
e5c6828493b5fa Peter Zijlstra 2021-09-23 423 return -ENOMEM;
e5c6828493b5fa Peter Zijlstra 2021-09-23 424 }
e5c6828493b5fa Peter Zijlstra 2021-09-23 425
e5c6828493b5fa Peter Zijlstra 2021-09-23 426 retry:
e5c6828493b5fa Peter Zijlstra 2021-09-23 427 ret = get_futex_key(uaddr1, flags & FLAGS_SHARED, &key1, FUTEX_READ);
e5c6828493b5fa Peter Zijlstra 2021-09-23 428 if (unlikely(ret != 0))
e5c6828493b5fa Peter Zijlstra 2021-09-23 429 return ret;
e5c6828493b5fa Peter Zijlstra 2021-09-23 430 ret = get_futex_key(uaddr2, flags & FLAGS_SHARED, &key2,
e5c6828493b5fa Peter Zijlstra 2021-09-23 431 requeue_pi ? FUTEX_WRITE : FUTEX_READ);
e5c6828493b5fa Peter Zijlstra 2021-09-23 432 if (unlikely(ret != 0))
e5c6828493b5fa Peter Zijlstra 2021-09-23 433 return ret;
e5c6828493b5fa Peter Zijlstra 2021-09-23 434
e5c6828493b5fa Peter Zijlstra 2021-09-23 435 /*
e5c6828493b5fa Peter Zijlstra 2021-09-23 436 * The check above which compares uaddrs is not sufficient for
e5c6828493b5fa Peter Zijlstra 2021-09-23 437 * shared futexes. We need to compare the keys:
e5c6828493b5fa Peter Zijlstra 2021-09-23 438 */
e5c6828493b5fa Peter Zijlstra 2021-09-23 439 if (requeue_pi && futex_match(&key1, &key2))
e5c6828493b5fa Peter Zijlstra 2021-09-23 440 return -EINVAL;
e5c6828493b5fa Peter Zijlstra 2021-09-23 441
e5c6828493b5fa Peter Zijlstra 2021-09-23 442 hb1 = futex_hash(&key1);
e5c6828493b5fa Peter Zijlstra 2021-09-23 443 hb2 = futex_hash(&key2);
e5c6828493b5fa Peter Zijlstra 2021-09-23 444
e5c6828493b5fa Peter Zijlstra 2021-09-23 445 retry_private:
e5c6828493b5fa Peter Zijlstra 2021-09-23 446 futex_hb_waiters_inc(hb2);
e5c6828493b5fa Peter Zijlstra 2021-09-23 447 double_lock_hb(hb1, hb2);
e5c6828493b5fa Peter Zijlstra 2021-09-23 448
e5c6828493b5fa Peter Zijlstra 2021-09-23 449 if (likely(cmpval != NULL)) {
e5c6828493b5fa Peter Zijlstra 2021-09-23 450 u32 curval;
e5c6828493b5fa Peter Zijlstra 2021-09-23 451
e5c6828493b5fa Peter Zijlstra 2021-09-23 452 ret = futex_get_value_locked(&curval, uaddr1);
e5c6828493b5fa Peter Zijlstra 2021-09-23 453
e5c6828493b5fa Peter Zijlstra 2021-09-23 454 if (unlikely(ret)) {
e5c6828493b5fa Peter Zijlstra 2021-09-23 455 double_unlock_hb(hb1, hb2);
e5c6828493b5fa Peter Zijlstra 2021-09-23 456 futex_hb_waiters_dec(hb2);
e5c6828493b5fa Peter Zijlstra 2021-09-23 457
e5c6828493b5fa Peter Zijlstra 2021-09-23 458 ret = get_user(curval, uaddr1);
e5c6828493b5fa Peter Zijlstra 2021-09-23 459 if (ret)
e5c6828493b5fa Peter Zijlstra 2021-09-23 460 return ret;
e5c6828493b5fa Peter Zijlstra 2021-09-23 461
e5c6828493b5fa Peter Zijlstra 2021-09-23 @462 if (!(flags & FLAGS_SHARED))
e5c6828493b5fa Peter Zijlstra 2021-09-23 463 goto retry_private;
e5c6828493b5fa Peter Zijlstra 2021-09-23 464
e5c6828493b5fa Peter Zijlstra 2021-09-23 465 goto retry;
e5c6828493b5fa Peter Zijlstra 2021-09-23 466 }
e5c6828493b5fa Peter Zijlstra 2021-09-23 467 if (curval != *cmpval) {
e5c6828493b5fa Peter Zijlstra 2021-09-23 468 ret = -EAGAIN;
e5c6828493b5fa Peter Zijlstra 2021-09-23 469 goto out_unlock;
e5c6828493b5fa Peter Zijlstra 2021-09-23 470 }
e5c6828493b5fa Peter Zijlstra 2021-09-23 471 }
e5c6828493b5fa Peter Zijlstra 2021-09-23 472
e5c6828493b5fa Peter Zijlstra 2021-09-23 473 if (requeue_pi) {
e5c6828493b5fa Peter Zijlstra 2021-09-23 474 struct task_struct *exiting = NULL;
e5c6828493b5fa Peter Zijlstra 2021-09-23 475
e5c6828493b5fa Peter Zijlstra 2021-09-23 476 /*
e5c6828493b5fa Peter Zijlstra 2021-09-23 477 * Attempt to acquire uaddr2 and wake the top waiter. If we
e5c6828493b5fa Peter Zijlstra 2021-09-23 478 * intend to requeue waiters, force setting the FUTEX_WAITERS
e5c6828493b5fa Peter Zijlstra 2021-09-23 479 * bit. We force this here where we are able to easily handle
e5c6828493b5fa Peter Zijlstra 2021-09-23 480 * faults rather in the requeue loop below.
e5c6828493b5fa Peter Zijlstra 2021-09-23 481 *
e5c6828493b5fa Peter Zijlstra 2021-09-23 482 * Updates topwaiter::requeue_state if a top waiter exists.
e5c6828493b5fa Peter Zijlstra 2021-09-23 483 */
e5c6828493b5fa Peter Zijlstra 2021-09-23 484 ret = futex_proxy_trylock_atomic(uaddr2, hb1, hb2, &key1,
e5c6828493b5fa Peter Zijlstra 2021-09-23 485 &key2, &pi_state,
e5c6828493b5fa Peter Zijlstra 2021-09-23 486 &exiting, nr_requeue);
e5c6828493b5fa Peter Zijlstra 2021-09-23 487
e5c6828493b5fa Peter Zijlstra 2021-09-23 488 /*
e5c6828493b5fa Peter Zijlstra 2021-09-23 489 * At this point the top_waiter has either taken uaddr2 or
e5c6828493b5fa Peter Zijlstra 2021-09-23 490 * is waiting on it. In both cases pi_state has been
e5c6828493b5fa Peter Zijlstra 2021-09-23 491 * established and an initial refcount on it. In case of an
e5c6828493b5fa Peter Zijlstra 2021-09-23 492 * error there's nothing.
e5c6828493b5fa Peter Zijlstra 2021-09-23 493 *
e5c6828493b5fa Peter Zijlstra 2021-09-23 494 * The top waiter's requeue_state is up to date:
e5c6828493b5fa Peter Zijlstra 2021-09-23 495 *
e5c6828493b5fa Peter Zijlstra 2021-09-23 496 * - If the lock was acquired atomically (ret == 1), then
e5c6828493b5fa Peter Zijlstra 2021-09-23 497 * the state is Q_REQUEUE_PI_LOCKED.
e5c6828493b5fa Peter Zijlstra 2021-09-23 498 *
e5c6828493b5fa Peter Zijlstra 2021-09-23 499 * The top waiter has been dequeued and woken up and can
e5c6828493b5fa Peter Zijlstra 2021-09-23 500 * return to user space immediately. The kernel/user
e5c6828493b5fa Peter Zijlstra 2021-09-23 501 * space state is consistent. In case that there must be
e5c6828493b5fa Peter Zijlstra 2021-09-23 502 * more waiters requeued the WAITERS bit in the user
e5c6828493b5fa Peter Zijlstra 2021-09-23 503 * space futex is set so the top waiter task has to go
e5c6828493b5fa Peter Zijlstra 2021-09-23 504 * into the syscall slowpath to unlock the futex. This
e5c6828493b5fa Peter Zijlstra 2021-09-23 505 * will block until this requeue operation has been
e5c6828493b5fa Peter Zijlstra 2021-09-23 506 * completed and the hash bucket locks have been
e5c6828493b5fa Peter Zijlstra 2021-09-23 507 * dropped.
e5c6828493b5fa Peter Zijlstra 2021-09-23 508 *
e5c6828493b5fa Peter Zijlstra 2021-09-23 509 * - If the trylock failed with an error (ret < 0) then
e5c6828493b5fa Peter Zijlstra 2021-09-23 510 * the state is either Q_REQUEUE_PI_NONE, i.e. "nothing
e5c6828493b5fa Peter Zijlstra 2021-09-23 511 * happened", or Q_REQUEUE_PI_IGNORE when there was an
e5c6828493b5fa Peter Zijlstra 2021-09-23 512 * interleaved early wakeup.
e5c6828493b5fa Peter Zijlstra 2021-09-23 513 *
e5c6828493b5fa Peter Zijlstra 2021-09-23 514 * - If the trylock did not succeed (ret == 0) then the
e5c6828493b5fa Peter Zijlstra 2021-09-23 515 * state is either Q_REQUEUE_PI_IN_PROGRESS or
e5c6828493b5fa Peter Zijlstra 2021-09-23 516 * Q_REQUEUE_PI_WAIT if an early wakeup interleaved.
e5c6828493b5fa Peter Zijlstra 2021-09-23 517 * This will be cleaned up in the loop below, which
e5c6828493b5fa Peter Zijlstra 2021-09-23 518 * cannot fail because futex_proxy_trylock_atomic() did
e5c6828493b5fa Peter Zijlstra 2021-09-23 519 * the same sanity checks for requeue_pi as the loop
e5c6828493b5fa Peter Zijlstra 2021-09-23 520 * below does.
e5c6828493b5fa Peter Zijlstra 2021-09-23 521 */
e5c6828493b5fa Peter Zijlstra 2021-09-23 522 switch (ret) {
e5c6828493b5fa Peter Zijlstra 2021-09-23 523 case 0:
e5c6828493b5fa Peter Zijlstra 2021-09-23 524 /* We hold a reference on the pi state. */
e5c6828493b5fa Peter Zijlstra 2021-09-23 525 break;
e5c6828493b5fa Peter Zijlstra 2021-09-23 526
e5c6828493b5fa Peter Zijlstra 2021-09-23 527 case 1:
e5c6828493b5fa Peter Zijlstra 2021-09-23 528 /*
e5c6828493b5fa Peter Zijlstra 2021-09-23 529 * futex_proxy_trylock_atomic() acquired the user space
e5c6828493b5fa Peter Zijlstra 2021-09-23 530 * futex. Adjust task_count.
e5c6828493b5fa Peter Zijlstra 2021-09-23 531 */
e5c6828493b5fa Peter Zijlstra 2021-09-23 532 task_count++;
e5c6828493b5fa Peter Zijlstra 2021-09-23 533 ret = 0;
e5c6828493b5fa Peter Zijlstra 2021-09-23 534 break;
e5c6828493b5fa Peter Zijlstra 2021-09-23 535
e5c6828493b5fa Peter Zijlstra 2021-09-23 536 /*
e5c6828493b5fa Peter Zijlstra 2021-09-23 537 * If the above failed, then pi_state is NULL and
e5c6828493b5fa Peter Zijlstra 2021-09-23 538 * waiter::requeue_state is correct.
e5c6828493b5fa Peter Zijlstra 2021-09-23 539 */
e5c6828493b5fa Peter Zijlstra 2021-09-23 540 case -EFAULT:
e5c6828493b5fa Peter Zijlstra 2021-09-23 541 double_unlock_hb(hb1, hb2);
e5c6828493b5fa Peter Zijlstra 2021-09-23 542 futex_hb_waiters_dec(hb2);
e5c6828493b5fa Peter Zijlstra 2021-09-23 543 ret = fault_in_user_writeable(uaddr2);
e5c6828493b5fa Peter Zijlstra 2021-09-23 544 if (!ret)
e5c6828493b5fa Peter Zijlstra 2021-09-23 545 goto retry;
e5c6828493b5fa Peter Zijlstra 2021-09-23 546 return ret;
e5c6828493b5fa Peter Zijlstra 2021-09-23 547 case -EBUSY:
e5c6828493b5fa Peter Zijlstra 2021-09-23 548 case -EAGAIN:
e5c6828493b5fa Peter Zijlstra 2021-09-23 549 /*
e5c6828493b5fa Peter Zijlstra 2021-09-23 550 * Two reasons for this:
e5c6828493b5fa Peter Zijlstra 2021-09-23 551 * - EBUSY: Owner is exiting and we just wait for the
e5c6828493b5fa Peter Zijlstra 2021-09-23 552 * exit to complete.
e5c6828493b5fa Peter Zijlstra 2021-09-23 553 * - EAGAIN: The user space value changed.
e5c6828493b5fa Peter Zijlstra 2021-09-23 554 */
e5c6828493b5fa Peter Zijlstra 2021-09-23 555 double_unlock_hb(hb1, hb2);
e5c6828493b5fa Peter Zijlstra 2021-09-23 556 futex_hb_waiters_dec(hb2);
e5c6828493b5fa Peter Zijlstra 2021-09-23 557 /*
e5c6828493b5fa Peter Zijlstra 2021-09-23 558 * Handle the case where the owner is in the middle of
e5c6828493b5fa Peter Zijlstra 2021-09-23 559 * exiting. Wait for the exit to complete otherwise
e5c6828493b5fa Peter Zijlstra 2021-09-23 560 * this task might loop forever, aka. live lock.
e5c6828493b5fa Peter Zijlstra 2021-09-23 561 */
e5c6828493b5fa Peter Zijlstra 2021-09-23 562 wait_for_owner_exiting(ret, exiting);
e5c6828493b5fa Peter Zijlstra 2021-09-23 563 cond_resched();
e5c6828493b5fa Peter Zijlstra 2021-09-23 564 goto retry;
e5c6828493b5fa Peter Zijlstra 2021-09-23 565 default:
e5c6828493b5fa Peter Zijlstra 2021-09-23 566 goto out_unlock;
e5c6828493b5fa Peter Zijlstra 2021-09-23 567 }
e5c6828493b5fa Peter Zijlstra 2021-09-23 568 }
e5c6828493b5fa Peter Zijlstra 2021-09-23 569
e5c6828493b5fa Peter Zijlstra 2021-09-23 570 plist_for_each_entry_safe(this, next, &hb1->chain, list) {
e5c6828493b5fa Peter Zijlstra 2021-09-23 571 if (task_count - nr_wake >= nr_requeue)
e5c6828493b5fa Peter Zijlstra 2021-09-23 572 break;
e5c6828493b5fa Peter Zijlstra 2021-09-23 573
e5c6828493b5fa Peter Zijlstra 2021-09-23 574 if (!futex_match(&this->key, &key1))
e5c6828493b5fa Peter Zijlstra 2021-09-23 575 continue;
e5c6828493b5fa Peter Zijlstra 2021-09-23 576
e5c6828493b5fa Peter Zijlstra 2021-09-23 577 /*
e5c6828493b5fa Peter Zijlstra 2021-09-23 578 * FUTEX_WAIT_REQUEUE_PI and FUTEX_CMP_REQUEUE_PI should always
e5c6828493b5fa Peter Zijlstra 2021-09-23 579 * be paired with each other and no other futex ops.
e5c6828493b5fa Peter Zijlstra 2021-09-23 580 *
e5c6828493b5fa Peter Zijlstra 2021-09-23 581 * We should never be requeueing a futex_q with a pi_state,
e5c6828493b5fa Peter Zijlstra 2021-09-23 582 * which is awaiting a futex_unlock_pi().
e5c6828493b5fa Peter Zijlstra 2021-09-23 583 */
e5c6828493b5fa Peter Zijlstra 2021-09-23 584 if ((requeue_pi && !this->rt_waiter) ||
e5c6828493b5fa Peter Zijlstra 2021-09-23 585 (!requeue_pi && this->rt_waiter) ||
e5c6828493b5fa Peter Zijlstra 2021-09-23 586 this->pi_state) {
e5c6828493b5fa Peter Zijlstra 2021-09-23 587 ret = -EINVAL;
e5c6828493b5fa Peter Zijlstra 2021-09-23 588 break;
e5c6828493b5fa Peter Zijlstra 2021-09-23 589 }
e5c6828493b5fa Peter Zijlstra 2021-09-23 590
e5c6828493b5fa Peter Zijlstra 2021-09-23 591 /* Plain futexes just wake or requeue and are done */
e5c6828493b5fa Peter Zijlstra 2021-09-23 592 if (!requeue_pi) {
e5c6828493b5fa Peter Zijlstra 2021-09-23 593 if (++task_count <= nr_wake)
e5c6828493b5fa Peter Zijlstra 2021-09-23 594 futex_wake_mark(&wake_q, this);
e5c6828493b5fa Peter Zijlstra 2021-09-23 595 else
e5c6828493b5fa Peter Zijlstra 2021-09-23 596 requeue_futex(this, hb1, hb2, &key2);
e5c6828493b5fa Peter Zijlstra 2021-09-23 597 continue;
e5c6828493b5fa Peter Zijlstra 2021-09-23 598 }
e5c6828493b5fa Peter Zijlstra 2021-09-23 599
e5c6828493b5fa Peter Zijlstra 2021-09-23 600 /* Ensure we requeue to the expected futex for requeue_pi. */
e5c6828493b5fa Peter Zijlstra 2021-09-23 601 if (!futex_match(this->requeue_pi_key, &key2)) {
e5c6828493b5fa Peter Zijlstra 2021-09-23 602 ret = -EINVAL;
e5c6828493b5fa Peter Zijlstra 2021-09-23 603 break;
e5c6828493b5fa Peter Zijlstra 2021-09-23 604 }
e5c6828493b5fa Peter Zijlstra 2021-09-23 605
e5c6828493b5fa Peter Zijlstra 2021-09-23 606 /*
e5c6828493b5fa Peter Zijlstra 2021-09-23 607 * Requeue nr_requeue waiters and possibly one more in the case
e5c6828493b5fa Peter Zijlstra 2021-09-23 608 * of requeue_pi if we couldn't acquire the lock atomically.
e5c6828493b5fa Peter Zijlstra 2021-09-23 609 *
e5c6828493b5fa Peter Zijlstra 2021-09-23 610 * Prepare the waiter to take the rt_mutex. Take a refcount
e5c6828493b5fa Peter Zijlstra 2021-09-23 611 * on the pi_state and store the pointer in the futex_q
e5c6828493b5fa Peter Zijlstra 2021-09-23 612 * object of the waiter.
e5c6828493b5fa Peter Zijlstra 2021-09-23 613 */
e5c6828493b5fa Peter Zijlstra 2021-09-23 614 get_pi_state(pi_state);
e5c6828493b5fa Peter Zijlstra 2021-09-23 615
e5c6828493b5fa Peter Zijlstra 2021-09-23 616 /* Don't requeue when the waiter is already on the way out. */
e5c6828493b5fa Peter Zijlstra 2021-09-23 617 if (!futex_requeue_pi_prepare(this, pi_state)) {
e5c6828493b5fa Peter Zijlstra 2021-09-23 618 /*
e5c6828493b5fa Peter Zijlstra 2021-09-23 619 * Early woken waiter signaled that it is on the
e5c6828493b5fa Peter Zijlstra 2021-09-23 620 * way out. Drop the pi_state reference and try the
e5c6828493b5fa Peter Zijlstra 2021-09-23 621 * next waiter. @this->pi_state is still NULL.
e5c6828493b5fa Peter Zijlstra 2021-09-23 622 */
e5c6828493b5fa Peter Zijlstra 2021-09-23 623 put_pi_state(pi_state);
e5c6828493b5fa Peter Zijlstra 2021-09-23 624 continue;
e5c6828493b5fa Peter Zijlstra 2021-09-23 625 }
e5c6828493b5fa Peter Zijlstra 2021-09-23 626
e5c6828493b5fa Peter Zijlstra 2021-09-23 627 ret = rt_mutex_start_proxy_lock(&pi_state->pi_mutex,
e5c6828493b5fa Peter Zijlstra 2021-09-23 628 this->rt_waiter,
e5c6828493b5fa Peter Zijlstra 2021-09-23 629 this->task);
e5c6828493b5fa Peter Zijlstra 2021-09-23 630
e5c6828493b5fa Peter Zijlstra 2021-09-23 631 if (ret == 1) {
e5c6828493b5fa Peter Zijlstra 2021-09-23 632 /*
e5c6828493b5fa Peter Zijlstra 2021-09-23 633 * We got the lock. We do neither drop the refcount
e5c6828493b5fa Peter Zijlstra 2021-09-23 634 * on pi_state nor clear this->pi_state because the
e5c6828493b5fa Peter Zijlstra 2021-09-23 635 * waiter needs the pi_state for cleaning up the
e5c6828493b5fa Peter Zijlstra 2021-09-23 636 * user space value. It will drop the refcount
e5c6828493b5fa Peter Zijlstra 2021-09-23 637 * after doing so. this::requeue_state is updated
e5c6828493b5fa Peter Zijlstra 2021-09-23 638 * in the wakeup as well.
e5c6828493b5fa Peter Zijlstra 2021-09-23 639 */
e5c6828493b5fa Peter Zijlstra 2021-09-23 640 requeue_pi_wake_futex(this, &key2, hb2);
e5c6828493b5fa Peter Zijlstra 2021-09-23 641 task_count++;
e5c6828493b5fa Peter Zijlstra 2021-09-23 642 } else if (!ret) {
e5c6828493b5fa Peter Zijlstra 2021-09-23 643 /* Waiter is queued, move it to hb2 */
e5c6828493b5fa Peter Zijlstra 2021-09-23 644 requeue_futex(this, hb1, hb2, &key2);
e5c6828493b5fa Peter Zijlstra 2021-09-23 645 futex_requeue_pi_complete(this, 0);
e5c6828493b5fa Peter Zijlstra 2021-09-23 646 task_count++;
e5c6828493b5fa Peter Zijlstra 2021-09-23 647 } else {
e5c6828493b5fa Peter Zijlstra 2021-09-23 648 /*
e5c6828493b5fa Peter Zijlstra 2021-09-23 649 * rt_mutex_start_proxy_lock() detected a potential
e5c6828493b5fa Peter Zijlstra 2021-09-23 650 * deadlock when we tried to queue that waiter.
e5c6828493b5fa Peter Zijlstra 2021-09-23 651 * Drop the pi_state reference which we took above
e5c6828493b5fa Peter Zijlstra 2021-09-23 652 * and remove the pointer to the state from the
e5c6828493b5fa Peter Zijlstra 2021-09-23 653 * waiters futex_q object.
e5c6828493b5fa Peter Zijlstra 2021-09-23 654 */
e5c6828493b5fa Peter Zijlstra 2021-09-23 655 this->pi_state = NULL;
e5c6828493b5fa Peter Zijlstra 2021-09-23 656 put_pi_state(pi_state);
e5c6828493b5fa Peter Zijlstra 2021-09-23 657 futex_requeue_pi_complete(this, ret);
e5c6828493b5fa Peter Zijlstra 2021-09-23 658 /*
e5c6828493b5fa Peter Zijlstra 2021-09-23 659 * We stop queueing more waiters and let user space
e5c6828493b5fa Peter Zijlstra 2021-09-23 660 * deal with the mess.
e5c6828493b5fa Peter Zijlstra 2021-09-23 661 */
e5c6828493b5fa Peter Zijlstra 2021-09-23 662 break;
e5c6828493b5fa Peter Zijlstra 2021-09-23 663 }
e5c6828493b5fa Peter Zijlstra 2021-09-23 664 }
e5c6828493b5fa Peter Zijlstra 2021-09-23 665
e5c6828493b5fa Peter Zijlstra 2021-09-23 666 /*
e5c6828493b5fa Peter Zijlstra 2021-09-23 667 * We took an extra initial reference to the pi_state in
e5c6828493b5fa Peter Zijlstra 2021-09-23 668 * futex_proxy_trylock_atomic(). We need to drop it here again.
e5c6828493b5fa Peter Zijlstra 2021-09-23 669 */
e5c6828493b5fa Peter Zijlstra 2021-09-23 670 put_pi_state(pi_state);
e5c6828493b5fa Peter Zijlstra 2021-09-23 671
e5c6828493b5fa Peter Zijlstra 2021-09-23 672 out_unlock:
e5c6828493b5fa Peter Zijlstra 2021-09-23 673 double_unlock_hb(hb1, hb2);
e5c6828493b5fa Peter Zijlstra 2021-09-23 674 wake_up_q(&wake_q);
e5c6828493b5fa Peter Zijlstra 2021-09-23 675 futex_hb_waiters_dec(hb2);
e5c6828493b5fa Peter Zijlstra 2021-09-23 676 return ret ? ret : task_count;
e5c6828493b5fa Peter Zijlstra 2021-09-23 677 }
e5c6828493b5fa Peter Zijlstra 2021-09-23 678
---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org
[-- Attachment #2: config.gz --]
[-- Type: application/gzip, Size: 19600 bytes --]
^ permalink raw reply [flat|nested] 2+ messages in thread
end of thread, other threads:[~2021-11-18 22:08 UTC | newest]
Thread overview: 2+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-11-17 17:12 kernel/futex/requeue.c:462 futex_requeue() warn: bitwise AND condition is false here kernel test robot
2021-11-18 22:08 kernel test robot
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.