From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <linux-kernel-owner@vger.kernel.org>
Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
	id S1752739AbbBSAXo (ORCPT <rfc822;w@1wt.eu>);
	Wed, 18 Feb 2015 19:23:44 -0500
Received: from mail-qg0-f53.google.com ([209.85.192.53]:62795 "EHLO
	mail-qg0-f53.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org
	with ESMTP id S1751982AbbBSAXn (ORCPT
	<rfc822;linux-kernel@vger.kernel.org>);
	Wed, 18 Feb 2015 19:23:43 -0500
MIME-Version: 1.0
In-Reply-To: <1424305436-6716-1-git-send-email-brian@peloton-tech.com>
References: <1424305436-6716-1-git-send-email-brian@peloton-tech.com>
From: Brian Silverman <brian@peloton-tech.com>
Date: Wed, 18 Feb 2015 19:23:12 -0500
Message-ID: <CAGt3f4=HmWU0ng=PNNyzA_5-FefMmWg-pOKzSL3Sc0ow=xHVug@mail.gmail.com>
Subject: Re: [PATCH] sched: fix RLIMIT_RTTIME when PI-boosting to RT
To: mingo <mingo@redhat.com>, peterz <peterz@infradead.org>
Cc: Austin Schuh <austin@peloton-tech.com>,
        linux-kernel <linux-kernel@vger.kernel.org>,
        Brian Silverman <brian@peloton-tech.com>
Content-Type: text/plain; charset=UTF-8
Sender: linux-kernel-owner@vger.kernel.org
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

Here's my test code. Compile with `gcc -pthread -lrt test_pi.c`. It
requires permission to set a realtime scheduling policy of 2 when
running.

#define _GNU_SOURCE

#include <pthread.h>
#include <unistd.h>
#include <stdio.h>
#include <time.h>
#include <sched.h>
#include <assert.h>
#include <sys/resource.h>
#include <string.h>
#include <signal.h>
#include <stdlib.h>

static const struct timespec kSleepTime = {0, 10000};
static pthread_mutex_t mutex;

extern void nop() {}

void *nonrealtime(void *ignored_param) {
  while (1) {
    assert(pthread_mutex_lock(&mutex) == 0);
    assert(pthread_mutex_unlock(&mutex) == 0);
    assert(clock_nanosleep(CLOCK_MONOTONIC, 0, &kSleepTime, NULL) == 0);
  }
}

void *realtime(void *ignored_param) {
  struct sched_param param;
  memset(&param, 0, sizeof(param));
  param.sched_priority = 2;
  assert(sched_setscheduler(0, SCHED_FIFO, &param) == 0);
  while (1) {
    assert(pthread_mutex_lock(&mutex) == 0);
    assert(clock_nanosleep(CLOCK_MONOTONIC, 0, &kSleepTime, NULL) == 0);
    assert(pthread_mutex_unlock(&mutex) == 0);
  }
}

void signal_handler(int number) {
  printf("got signal %d, SIGXCPU=%d\n", number, SIGXCPU);
  exit(0);
}

int main() {
  struct sigaction action;
  memset(&action, 0, sizeof(action));
  action.sa_handler = signal_handler;
  assert(sigaction(SIGXCPU, &action, NULL) == 0);

  struct rlimit rlim;
  rlim.rlim_cur = 500;
  rlim.rlim_max = 5000;
  assert(prlimit(0, RLIMIT_RTTIME, &rlim, NULL) == 0);

  pthread_mutexattr_t mutexattr;
  assert(pthread_mutexattr_init(&mutexattr) == 0);
  assert(pthread_mutexattr_setprotocol(&mutexattr, PTHREAD_PRIO_INHERIT) == 0);
  assert(pthread_mutex_init(&mutex, &mutexattr) == 0);
  assert(pthread_mutexattr_destroy(&mutexattr) == 0);

  pthread_t nrt, rt;
  assert(pthread_create(&nrt, NULL, nonrealtime, NULL) == 0);
  assert(pthread_create(&rt, NULL, realtime, NULL) == 0);
  assert(pthread_join(nrt, NULL) == 0);
  assert(pthread_join(rt, NULL) == 0);
  return 0;
}


On Wed, Feb 18, 2015 at 7:23 PM,  <brian@peloton-tech.com> wrote:
> From: Brian Silverman <brian@peloton-tech.com>
>
> When non-realtime tasks get priority-inheritance boosted to a realtime
> scheduling class, RLIMIT_RTTIME starts to apply to them. However, the
> counter used for checking this (the same one used for SCHED_RR
> timeslices) was not getting reset. This meant that tasks running with a
> non-realtime scheduling class which are repeatedly boosted to a realtime
> one, but never block while they are running realtime, eventually hit the
> timeout without ever running for a time over the limit. This patch
> resets the realtime timeslice counter when un-PI-boosting from an RT to
> a non-RT scheduling class.
>
> I have some test code with two threads and a shared PTHREAD_PRIO_INHERIT
> mutex which induces priority boosting and spins while boosted that gets
> killed by a SIGXCPU on non-fixed kernels but doesn't with this patch
> applied. It happens much faster with a CONFIG_PREEMPT_RT kernel, and
> does happen eventually with PREEMPT_VOLUNTARY kernels.
>
> Signed-off-by: Brian Silverman <brian@peloton-tech.com>
> ---
> I am not subscribed to the list so please CC me on any responses.
>
>  kernel/sched/core.c |    2 ++
>  1 file changed, 2 insertions(+)
>
> diff --git a/kernel/sched/core.c b/kernel/sched/core.c
> index 87b9814..16ad0ed 100644
> --- a/kernel/sched/core.c
> +++ b/kernel/sched/core.c
> @@ -3192,6 +3192,8 @@ void rt_mutex_setprio(struct task_struct *p, int prio)
>         } else {
>                 if (dl_prio(oldprio))
>                         p->dl.dl_boosted = 0;
> +               if (rt_prio(oldprio))
> +                       p->rt.timeout = 0;
>                 p->sched_class = &fair_sched_class;
>         }
>
> --
> 1.7.10.4
>