* [PATCH 0/2] add thread priority accessors
@ 2022-05-23 10:46 Tyler Retzlaff
2022-05-23 10:46 ` [PATCH 1/2] eal: get/set thread priority per thread identifier Tyler Retzlaff
` (3 more replies)
0 siblings, 4 replies; 15+ messages in thread
From: Tyler Retzlaff @ 2022-05-23 10:46 UTC (permalink / raw)
To: dev; +Cc: thomas, dmitry.kozliuk, anatoly.burakov, Tyler Retzlaff
this series introduces accessors for get and set of thread
priority based on original patches from Narcisa Vasile.
Tyler Retzlaff (2):
eal: get/set thread priority per thread identifier
test/threads: add unit test for get set priority
app/test/test_threads.c | 52 +++++++++++++++++
lib/eal/include/rte_thread.h | 50 +++++++++++++++++
lib/eal/unix/rte_thread.c | 101 +++++++++++++++++++++++++++++++++
lib/eal/windows/rte_thread.c | 130 +++++++++++++++++++++++++++++++++++++++++++
4 files changed, 333 insertions(+)
--
1.8.3.1
^ permalink raw reply [flat|nested] 15+ messages in thread* [PATCH 1/2] eal: get/set thread priority per thread identifier 2022-05-23 10:46 [PATCH 0/2] add thread priority accessors Tyler Retzlaff @ 2022-05-23 10:46 ` Tyler Retzlaff 2022-05-23 10:46 ` [PATCH 2/2] test/threads: add unit test for get set priority Tyler Retzlaff ` (2 subsequent siblings) 3 siblings, 0 replies; 15+ messages in thread From: Tyler Retzlaff @ 2022-05-23 10:46 UTC (permalink / raw) To: dev Cc: thomas, dmitry.kozliuk, anatoly.burakov, Tyler Retzlaff, Narcisa Vasile Add functions for setting and getting the priority of a thread. Priorities on multiple platforms are similarly determined by a priority value and a priority class/policy. Currently in DPDK most threads operate at the OS-default priority level but there are cases when increasing the priority is useful. For example, high performance applications may require elevated priority levels. For these reasons, EAL will expose two priority levels which are named suggestively "normal" and "realtime_critical" and are computed as follows: On Linux, the following mapping is created: RTE_THREAD_PRIORITY_NORMAL corresponds to * policy SCHED_OTHER * priority value: (sched_get_priority_min(SCHED_OTHER) + sched_get_priority_max(SCHED_OTHER))/2; RTE_THREAD_PRIORITY_REALTIME_CRITICAL corresponds to * policy SCHED_RR * priority value: sched_get_priority_max(SCHED_RR); On Windows, the following mapping is created: RTE_THREAD_PRIORITY_NORMAL corresponds to * class NORMAL_PRIORITY_CLASS * priority THREAD_PRIORITY_NORMAL RTE_THREAD_PRIORITY_REALTIME_CRITICAL corresponds to * class REALTIME_PRIORITY_CLASS (when running with privileges) * class HIGH_PRIORITY_CLASS (when running without privileges) * priority THREAD_PRIORITY_TIME_CRITICAL Note that on Linux the resulting priority value will be 0, in accordance to the documentation that mention the value should be 0 for SCHED_OTHER policy. Signed-off-by: Narcisa Vasile <navasile@linux.microsoft.com> Signed-off-by: Tyler Retzlaff <roretzla@linux.microsoft.com> --- lib/eal/include/rte_thread.h | 50 +++++++++++++++++ lib/eal/unix/rte_thread.c | 101 +++++++++++++++++++++++++++++++++ lib/eal/windows/rte_thread.c | 130 +++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 281 insertions(+) diff --git a/lib/eal/include/rte_thread.h b/lib/eal/include/rte_thread.h index 7888f7a..9e261bf 100644 --- a/lib/eal/include/rte_thread.h +++ b/lib/eal/include/rte_thread.h @@ -31,6 +31,16 @@ } rte_thread_t; /** + * Thread priority values. + */ +enum rte_thread_priority { + RTE_THREAD_PRIORITY_NORMAL = 0, + /**< normal thread priority, the default */ + RTE_THREAD_PRIORITY_REALTIME_CRITICAL = 1, + /**< highest thread priority allowed */ +}; + +/** * TLS key type, an opaque pointer. */ typedef struct eal_tls_key *rte_thread_key; @@ -115,6 +125,46 @@ int rte_thread_get_affinity_by_id(rte_thread_t thread_id, #endif /* RTE_HAS_CPUSET */ /** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Get the priority of a thread. + * + * @param thread_id + * Id of the thread for which to get priority. + * + * @param priority + * Location to store the retrieved priority. + * + * @return + * On success, return 0. + * On failure, return a positive errno-style error number. + */ +__rte_experimental +int rte_thread_get_priority(rte_thread_t thread_id, + enum rte_thread_priority *priority); + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Set the priority of a thread. + * + * @param thread_id + * Id of the thread for which to set priority. + * + * @param priority + * Priority value to be set. + * + * @return + * On success, return 0. + * On failure, return a positive errno-style error number. + */ +__rte_experimental +int rte_thread_set_priority(rte_thread_t thread_id, + enum rte_thread_priority priority); + +/** * Create a TLS data key visible to all threads in the process. * the created key is later used to get/set a value. * and optional destructor can be set to be called when a thread exits. diff --git a/lib/eal/unix/rte_thread.c b/lib/eal/unix/rte_thread.c index 9e5fa47..2b19f4a 100644 --- a/lib/eal/unix/rte_thread.c +++ b/lib/eal/unix/rte_thread.c @@ -16,6 +16,64 @@ struct eal_tls_key { pthread_key_t thread_index; }; +static int +thread_map_priority_to_os_value(enum rte_thread_priority eal_pri, + int *os_pri, int *pol) +{ + /* Clear the output parameters */ + *os_pri = sched_get_priority_min(SCHED_OTHER) - 1; + *pol = -1; + + switch (eal_pri) { + case RTE_THREAD_PRIORITY_NORMAL: + *pol = SCHED_OTHER; + + /* + * Choose the middle of the range to represent + * the priority 'normal'. + * On Linux, this should be 0, since both + * sched_get_priority_min/_max return 0 for SCHED_OTHER. + */ + *os_pri = (sched_get_priority_min(SCHED_OTHER) + + sched_get_priority_max(SCHED_OTHER))/2; + break; + case RTE_THREAD_PRIORITY_REALTIME_CRITICAL: + *pol = SCHED_RR; + *os_pri = sched_get_priority_max(SCHED_RR); + break; + default: + RTE_LOG(DEBUG, EAL, "The requested priority value is invalid.\n"); + return EINVAL; + } + return 0; +} + +static int +thread_map_os_priority_to_eal_priority(int policy, int os_pri, + enum rte_thread_priority *eal_pri) +{ + switch (policy) { + case SCHED_OTHER: + if (os_pri == (sched_get_priority_min(SCHED_OTHER) + + sched_get_priority_max(SCHED_OTHER))/2) { + *eal_pri = RTE_THREAD_PRIORITY_NORMAL; + return 0; + } + break; + case SCHED_RR: + if (os_pri == sched_get_priority_max(SCHED_RR)) { + *eal_pri = RTE_THREAD_PRIORITY_REALTIME_CRITICAL; + return 0; + } + break; + default: + RTE_LOG(DEBUG, EAL, "The OS priority value does not map to an EAL-defined priority.\n"); + return EINVAL; + } + + return 0; +} + rte_thread_t rte_thread_self(void) { @@ -29,6 +87,49 @@ struct eal_tls_key { } int +rte_thread_get_priority(rte_thread_t thread_id, + enum rte_thread_priority *priority) +{ + int ret; + int policy; + struct sched_param param; + + ret = pthread_getschedparam((pthread_t)thread_id.opaque_id, &policy, + ¶m); + if (ret != 0) { + RTE_LOG(DEBUG, EAL, "pthread_getschedparam failed\n"); + goto cleanup; + } + + return thread_map_os_priority_to_eal_priority(policy, + param.sched_priority, priority); + +cleanup: + return ret; +} + +int +rte_thread_set_priority(rte_thread_t thread_id, + enum rte_thread_priority priority) +{ + int ret; + int policy; + struct sched_param param; + +/* Realtime priority can cause crashes on non-Windows platforms. */ + if (priority == RTE_THREAD_PRIORITY_REALTIME_CRITICAL) + return ENOTSUP; + + ret = thread_map_priority_to_os_value(priority, ¶m.sched_priority, + &policy); + if (ret != 0) + return ret; + + return pthread_setschedparam((pthread_t)thread_id.opaque_id, + policy, ¶m); +} + +int rte_thread_key_create(rte_thread_key *key, void (*destructor)(void *)) { int err; diff --git a/lib/eal/windows/rte_thread.c b/lib/eal/windows/rte_thread.c index a616703..495225e 100644 --- a/lib/eal/windows/rte_thread.c +++ b/lib/eal/windows/rte_thread.c @@ -61,6 +61,58 @@ struct eal_tls_key { return thread_translate_win32_error(error); } +static int +thread_map_priority_to_os_value(enum rte_thread_priority eal_pri, + int *os_pri, DWORD *pri_class) +{ + /* Clear the output parameters */ + *os_pri = -1; + *pri_class = -1; + + switch (eal_pri) { + case RTE_THREAD_PRIORITY_NORMAL: + *pri_class = NORMAL_PRIORITY_CLASS; + *os_pri = THREAD_PRIORITY_NORMAL; + break; + case RTE_THREAD_PRIORITY_REALTIME_CRITICAL: + *pri_class = REALTIME_PRIORITY_CLASS; + *os_pri = THREAD_PRIORITY_TIME_CRITICAL; + break; + default: + RTE_LOG(DEBUG, EAL, "The requested priority value is invalid.\n"); + return EINVAL; + } + + return 0; +} + +static int +thread_map_os_priority_to_eal_value(int os_pri, DWORD pri_class, + enum rte_thread_priority *eal_pri) +{ + switch (pri_class) { + case NORMAL_PRIORITY_CLASS: + if (os_pri == THREAD_PRIORITY_NORMAL) { + *eal_pri = RTE_THREAD_PRIORITY_NORMAL; + return 0; + } + break; + case HIGH_PRIORITY_CLASS: + RTE_LOG(WARNING, EAL, "The OS priority class is high not real-time.\n"); + /* FALLTHROUGH */ + case REALTIME_PRIORITY_CLASS: + if (os_pri == THREAD_PRIORITY_TIME_CRITICAL) { + *eal_pri = RTE_THREAD_PRIORITY_REALTIME_CRITICAL; + return 0; + } + break; + default: + RTE_LOG(DEBUG, EAL, "The OS priority value does not map to an EAL-defined priority.\n"); + return EINVAL; + } + return 0; +} + rte_thread_t rte_thread_self(void) { @@ -72,6 +124,84 @@ struct eal_tls_key { } int +rte_thread_get_priority(rte_thread_t thread_id, + enum rte_thread_priority *priority) +{ + HANDLE thread_handle = NULL; + DWORD pri_class; + int os_pri; + int ret; + + pri_class = GetPriorityClass(GetCurrentProcess()); + if (pri_class == 0) { + ret = thread_log_last_error("GetPriorityClass()"); + goto cleanup; + } + + thread_handle = OpenThread(THREAD_SET_INFORMATION | + THREAD_QUERY_INFORMATION, FALSE, thread_id.opaque_id); + if (thread_handle == NULL) { + ret = thread_log_last_error("OpenThread()"); + goto cleanup; + } + + os_pri = GetThreadPriority(thread_handle); + if (os_pri == THREAD_PRIORITY_ERROR_RETURN) { + ret = thread_log_last_error("GetThreadPriority()"); + goto cleanup; + } + + ret = thread_map_os_priority_to_eal_value(os_pri, pri_class, priority); + if (ret != 0) + goto cleanup; + +cleanup: + if (thread_handle != NULL) + CloseHandle(thread_handle); + + return ret; +} + +int +rte_thread_set_priority(rte_thread_t thread_id, + enum rte_thread_priority priority) +{ + HANDLE thread_handle; + DWORD priority_class; + int os_priority; + int ret = 0; + + thread_handle = OpenThread(THREAD_SET_INFORMATION | + THREAD_QUERY_INFORMATION, FALSE, + thread_id.opaque_id); + if (thread_handle == NULL) { + ret = thread_log_last_error("OpenThread()"); + goto cleanup; + } + + ret = thread_map_priority_to_os_value(priority, &os_priority, + &priority_class); + if (ret != 0) + goto cleanup; + + if (!SetPriorityClass(GetCurrentProcess(), priority_class)) { + ret = thread_log_last_error("SetPriorityClass()"); + goto cleanup; + } + + if (!SetThreadPriority(thread_handle, os_priority)) { + ret = thread_log_last_error("SetThreadPriority()"); + goto cleanup; + } + +cleanup: + if (thread_handle != NULL) + CloseHandle(thread_handle); + + return ret; +} + +int rte_thread_key_create(rte_thread_key *key, __rte_unused void (*destructor)(void *)) { -- 1.8.3.1 ^ permalink raw reply related [flat|nested] 15+ messages in thread
* [PATCH 2/2] test/threads: add unit test for get set priority 2022-05-23 10:46 [PATCH 0/2] add thread priority accessors Tyler Retzlaff 2022-05-23 10:46 ` [PATCH 1/2] eal: get/set thread priority per thread identifier Tyler Retzlaff @ 2022-05-23 10:46 ` Tyler Retzlaff 2022-05-23 10:52 ` Tyler Retzlaff 2022-05-23 13:07 ` [PATCH v2 0/2] add thread priority accessors Tyler Retzlaff 2022-05-24 11:08 ` [PATCH v3 0/2] add thread priority accessors Tyler Retzlaff 3 siblings, 1 reply; 15+ messages in thread From: Tyler Retzlaff @ 2022-05-23 10:46 UTC (permalink / raw) To: dev; +Cc: thomas, dmitry.kozliuk, anatoly.burakov, Tyler Retzlaff Add unit tests to exercise and demonstrate rte_thread_{get,set}_priority(). Signed-off-by: Tyler Retzlaff <roretzla@linux.microsoft.com> --- app/test/test_threads.c | 52 +++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 52 insertions(+) diff --git a/app/test/test_threads.c b/app/test/test_threads.c index e1a2ea5..d369418 100644 --- a/app/test/test_threads.c +++ b/app/test/test_threads.c @@ -24,6 +24,56 @@ } static int +test_thread_priority(void) +{ + pthread_t id; + rte_thread_t thread_id; + enum rte_thread_priority priority; + + thread_id_ready = 0; + RTE_TEST_ASSERT(pthread_create(&id, NULL, thread_main, &thread_id) == 0, + "Failed to create thread"); + + while (__atomic_load_n(&thread_id_ready, __ATOMIC_ACQUIRE) == 0) + ; + + priority = RTE_THREAD_PRIORITY_NORMAL; + RTE_TEST_ASSERT(rte_thread_set_priority(thread_id, priority) == 0, + "Failed to set thread priority"); + RTE_TEST_ASSERT(rte_thread_get_priority(thread_id, &priority) == 0, + "Failed to get thread priority"); + RTE_TEST_ASSERT(priority == RTE_THREAD_PRIORITY_NORMAL, + "Priority set mismatches priority get"); + + priority = RTE_THREAD_PRIORITY_REALTIME_CRITICAL; +#ifndef RTE_EXEC_ENV_WINDOWS + RTE_TEST_ASSERT(rte_thread_set_priority(thread_id, priority) == -ENOTSUP, + "Priority set to critical should fail"); + RTE_TEST_ASSERT(rte_thread_get_priority(thread_id, &priority) == 0, + "Failed to get thread priority"); + RTE_TEST_ASSERT(priority == RTE_THREAD_PRIORITY_NORMAL, + "Failed set to critical should have retained normal"); +#else + RTE_TEST_ASSERT(rte_thread_set_priority(thread_id, priority) == 0, + "Priority set to critical should succeed"); + RTE_TEST_ASSERT(rte_thread_get_priority(thread_id, &priority) == 0, + "Failed to get thread priority"); + RTE_TEST_ASSERT(priority == RTE_THREAD_PRIORITY_REALTIME_CRITICAL, + "Priority get set mistmatches priority get"); +#endif + + priority = RTE_THREAD_PRIORITY_NORMAL; + RTE_TEST_ASSERT(rte_thread_set_priority(thread_id, priority) == 0, + "Failed to set thread priority"); + RTE_TEST_ASSERT(rte_thread_get_priority(thread_id, &priority) == 0, + "Failed to get thread priority"); + RTE_TEST_ASSERT(priority == RTE_THREAD_PRIORITY_NORMAL, + "Priority set mismatches priority get"); + + return 0; +} + +static int test_thread_affinity(void) { pthread_t id; @@ -31,6 +81,7 @@ rte_cpuset_t cpuset0; rte_cpuset_t cpuset1; + thread_id_ready = 0; RTE_TEST_ASSERT(pthread_create(&id, NULL, thread_main, &thread_id) == 0, "Failed to create thread"); @@ -68,6 +119,7 @@ .teardown = NULL, .unit_test_cases = { TEST_CASE(test_thread_affinity), + TEST_CASE(test_thread_priority), TEST_CASES_END() } }; -- 1.8.3.1 ^ permalink raw reply related [flat|nested] 15+ messages in thread
* Re: [PATCH 2/2] test/threads: add unit test for get set priority 2022-05-23 10:46 ` [PATCH 2/2] test/threads: add unit test for get set priority Tyler Retzlaff @ 2022-05-23 10:52 ` Tyler Retzlaff 0 siblings, 0 replies; 15+ messages in thread From: Tyler Retzlaff @ 2022-05-23 10:52 UTC (permalink / raw) To: dev; +Cc: thomas, dmitry.kozliuk, anatoly.burakov On Mon, May 23, 2022 at 03:46:13AM -0700, Tyler Retzlaff wrote: > Add unit tests to exercise and demonstrate rte_thread_{get,set}_priority(). > > Signed-off-by: Tyler Retzlaff <roretzla@linux.microsoft.com> > --- > app/test/test_threads.c | 52 +++++++++++++++++++++++++++++++++++++++++++++++++ > 1 file changed, 52 insertions(+) > > diff --git a/app/test/test_threads.c b/app/test/test_threads.c > index e1a2ea5..d369418 100644 > --- a/app/test/test_threads.c > +++ b/app/test/test_threads.c > @@ -24,6 +24,56 @@ > } > > static int > +test_thread_priority(void) > +{ > + pthread_t id; > + rte_thread_t thread_id; > + enum rte_thread_priority priority; > + > + thread_id_ready = 0; > + RTE_TEST_ASSERT(pthread_create(&id, NULL, thread_main, &thread_id) == 0, > + "Failed to create thread"); > + > + while (__atomic_load_n(&thread_id_ready, __ATOMIC_ACQUIRE) == 0) > + ; > + > + priority = RTE_THREAD_PRIORITY_NORMAL; > + RTE_TEST_ASSERT(rte_thread_set_priority(thread_id, priority) == 0, > + "Failed to set thread priority"); > + RTE_TEST_ASSERT(rte_thread_get_priority(thread_id, &priority) == 0, > + "Failed to get thread priority"); > + RTE_TEST_ASSERT(priority == RTE_THREAD_PRIORITY_NORMAL, > + "Priority set mismatches priority get"); > + > + priority = RTE_THREAD_PRIORITY_REALTIME_CRITICAL; > +#ifndef RTE_EXEC_ENV_WINDOWS > + RTE_TEST_ASSERT(rte_thread_set_priority(thread_id, priority) == -ENOTSUP, > + "Priority set to critical should fail"); > + RTE_TEST_ASSERT(rte_thread_get_priority(thread_id, &priority) == 0, > + "Failed to get thread priority"); > + RTE_TEST_ASSERT(priority == RTE_THREAD_PRIORITY_NORMAL, > + "Failed set to critical should have retained normal"); > +#else > + RTE_TEST_ASSERT(rte_thread_set_priority(thread_id, priority) == 0, > + "Priority set to critical should succeed"); > + RTE_TEST_ASSERT(rte_thread_get_priority(thread_id, &priority) == 0, > + "Failed to get thread priority"); > + RTE_TEST_ASSERT(priority == RTE_THREAD_PRIORITY_REALTIME_CRITICAL, > + "Priority get set mistmatches priority get"); mistmatches -> mismatches will fix this when addressing more substantial feedback > +#endif > + > + priority = RTE_THREAD_PRIORITY_NORMAL; > + RTE_TEST_ASSERT(rte_thread_set_priority(thread_id, priority) == 0, > + "Failed to set thread priority"); > + RTE_TEST_ASSERT(rte_thread_get_priority(thread_id, &priority) == 0, > + "Failed to get thread priority"); > + RTE_TEST_ASSERT(priority == RTE_THREAD_PRIORITY_NORMAL, > + "Priority set mismatches priority get"); > + > + return 0; > +} > + > +static int > test_thread_affinity(void) > { > pthread_t id; > @@ -31,6 +81,7 @@ > rte_cpuset_t cpuset0; > rte_cpuset_t cpuset1; > > + thread_id_ready = 0; > RTE_TEST_ASSERT(pthread_create(&id, NULL, thread_main, &thread_id) == 0, > "Failed to create thread"); > > @@ -68,6 +119,7 @@ > .teardown = NULL, > .unit_test_cases = { > TEST_CASE(test_thread_affinity), > + TEST_CASE(test_thread_priority), > TEST_CASES_END() > } > }; > -- > 1.8.3.1 ^ permalink raw reply [flat|nested] 15+ messages in thread
* [PATCH v2 0/2] add thread priority accessors 2022-05-23 10:46 [PATCH 0/2] add thread priority accessors Tyler Retzlaff 2022-05-23 10:46 ` [PATCH 1/2] eal: get/set thread priority per thread identifier Tyler Retzlaff 2022-05-23 10:46 ` [PATCH 2/2] test/threads: add unit test for get set priority Tyler Retzlaff @ 2022-05-23 13:07 ` Tyler Retzlaff 2022-05-23 13:07 ` [PATCH v2 1/2] eal: get/set thread priority per thread identifier Tyler Retzlaff 2022-05-23 13:07 ` [PATCH v2 2/2] test/threads: add unit test for get set priority Tyler Retzlaff 2022-05-24 11:08 ` [PATCH v3 0/2] add thread priority accessors Tyler Retzlaff 3 siblings, 2 replies; 15+ messages in thread From: Tyler Retzlaff @ 2022-05-23 13:07 UTC (permalink / raw) To: dev; +Cc: thomas, dmitry.kozliuk, anatoly.burakov, Tyler Retzlaff this series introduces accessors for get and set of thread priority based on original patches from Narcisa Vasile. v2: * fix typo in test assert string mistmatch -> mismatch * add new experimental exports names to lib/eal/version.map Tyler Retzlaff (2): eal: get/set thread priority per thread identifier test/threads: add unit test for get set priority app/test/test_threads.c | 52 +++++++++++++++++ lib/eal/include/rte_thread.h | 50 +++++++++++++++++ lib/eal/unix/rte_thread.c | 101 +++++++++++++++++++++++++++++++++ lib/eal/version.map | 2 + lib/eal/windows/rte_thread.c | 130 +++++++++++++++++++++++++++++++++++++++++++ 5 files changed, 335 insertions(+) -- 1.8.3.1 ^ permalink raw reply [flat|nested] 15+ messages in thread
* [PATCH v2 1/2] eal: get/set thread priority per thread identifier 2022-05-23 13:07 ` [PATCH v2 0/2] add thread priority accessors Tyler Retzlaff @ 2022-05-23 13:07 ` Tyler Retzlaff 2022-05-23 13:07 ` [PATCH v2 2/2] test/threads: add unit test for get set priority Tyler Retzlaff 1 sibling, 0 replies; 15+ messages in thread From: Tyler Retzlaff @ 2022-05-23 13:07 UTC (permalink / raw) To: dev Cc: thomas, dmitry.kozliuk, anatoly.burakov, Tyler Retzlaff, Narcisa Vasile Add functions for setting and getting the priority of a thread. Priorities on multiple platforms are similarly determined by a priority value and a priority class/policy. Currently in DPDK most threads operate at the OS-default priority level but there are cases when increasing the priority is useful. For example, high performance applications may require elevated priority levels. For these reasons, EAL will expose two priority levels which are named suggestively "normal" and "realtime_critical" and are computed as follows: On Linux, the following mapping is created: RTE_THREAD_PRIORITY_NORMAL corresponds to * policy SCHED_OTHER * priority value: (sched_get_priority_min(SCHED_OTHER) + sched_get_priority_max(SCHED_OTHER))/2; RTE_THREAD_PRIORITY_REALTIME_CRITICAL corresponds to * policy SCHED_RR * priority value: sched_get_priority_max(SCHED_RR); On Windows, the following mapping is created: RTE_THREAD_PRIORITY_NORMAL corresponds to * class NORMAL_PRIORITY_CLASS * priority THREAD_PRIORITY_NORMAL RTE_THREAD_PRIORITY_REALTIME_CRITICAL corresponds to * class REALTIME_PRIORITY_CLASS (when running with privileges) * class HIGH_PRIORITY_CLASS (when running without privileges) * priority THREAD_PRIORITY_TIME_CRITICAL Note that on Linux the resulting priority value will be 0, in accordance to the documentation that mention the value should be 0 for SCHED_OTHER policy. Signed-off-by: Narcisa Vasile <navasile@linux.microsoft.com> Signed-off-by: Tyler Retzlaff <roretzla@linux.microsoft.com> --- lib/eal/include/rte_thread.h | 50 +++++++++++++++++ lib/eal/unix/rte_thread.c | 101 +++++++++++++++++++++++++++++++++ lib/eal/version.map | 2 + lib/eal/windows/rte_thread.c | 130 +++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 283 insertions(+) diff --git a/lib/eal/include/rte_thread.h b/lib/eal/include/rte_thread.h index 7888f7a..9e261bf 100644 --- a/lib/eal/include/rte_thread.h +++ b/lib/eal/include/rte_thread.h @@ -31,6 +31,16 @@ } rte_thread_t; /** + * Thread priority values. + */ +enum rte_thread_priority { + RTE_THREAD_PRIORITY_NORMAL = 0, + /**< normal thread priority, the default */ + RTE_THREAD_PRIORITY_REALTIME_CRITICAL = 1, + /**< highest thread priority allowed */ +}; + +/** * TLS key type, an opaque pointer. */ typedef struct eal_tls_key *rte_thread_key; @@ -115,6 +125,46 @@ int rte_thread_get_affinity_by_id(rte_thread_t thread_id, #endif /* RTE_HAS_CPUSET */ /** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Get the priority of a thread. + * + * @param thread_id + * Id of the thread for which to get priority. + * + * @param priority + * Location to store the retrieved priority. + * + * @return + * On success, return 0. + * On failure, return a positive errno-style error number. + */ +__rte_experimental +int rte_thread_get_priority(rte_thread_t thread_id, + enum rte_thread_priority *priority); + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Set the priority of a thread. + * + * @param thread_id + * Id of the thread for which to set priority. + * + * @param priority + * Priority value to be set. + * + * @return + * On success, return 0. + * On failure, return a positive errno-style error number. + */ +__rte_experimental +int rte_thread_set_priority(rte_thread_t thread_id, + enum rte_thread_priority priority); + +/** * Create a TLS data key visible to all threads in the process. * the created key is later used to get/set a value. * and optional destructor can be set to be called when a thread exits. diff --git a/lib/eal/unix/rte_thread.c b/lib/eal/unix/rte_thread.c index 9e5fa47..2b19f4a 100644 --- a/lib/eal/unix/rte_thread.c +++ b/lib/eal/unix/rte_thread.c @@ -16,6 +16,64 @@ struct eal_tls_key { pthread_key_t thread_index; }; +static int +thread_map_priority_to_os_value(enum rte_thread_priority eal_pri, + int *os_pri, int *pol) +{ + /* Clear the output parameters */ + *os_pri = sched_get_priority_min(SCHED_OTHER) - 1; + *pol = -1; + + switch (eal_pri) { + case RTE_THREAD_PRIORITY_NORMAL: + *pol = SCHED_OTHER; + + /* + * Choose the middle of the range to represent + * the priority 'normal'. + * On Linux, this should be 0, since both + * sched_get_priority_min/_max return 0 for SCHED_OTHER. + */ + *os_pri = (sched_get_priority_min(SCHED_OTHER) + + sched_get_priority_max(SCHED_OTHER))/2; + break; + case RTE_THREAD_PRIORITY_REALTIME_CRITICAL: + *pol = SCHED_RR; + *os_pri = sched_get_priority_max(SCHED_RR); + break; + default: + RTE_LOG(DEBUG, EAL, "The requested priority value is invalid.\n"); + return EINVAL; + } + return 0; +} + +static int +thread_map_os_priority_to_eal_priority(int policy, int os_pri, + enum rte_thread_priority *eal_pri) +{ + switch (policy) { + case SCHED_OTHER: + if (os_pri == (sched_get_priority_min(SCHED_OTHER) + + sched_get_priority_max(SCHED_OTHER))/2) { + *eal_pri = RTE_THREAD_PRIORITY_NORMAL; + return 0; + } + break; + case SCHED_RR: + if (os_pri == sched_get_priority_max(SCHED_RR)) { + *eal_pri = RTE_THREAD_PRIORITY_REALTIME_CRITICAL; + return 0; + } + break; + default: + RTE_LOG(DEBUG, EAL, "The OS priority value does not map to an EAL-defined priority.\n"); + return EINVAL; + } + + return 0; +} + rte_thread_t rte_thread_self(void) { @@ -29,6 +87,49 @@ struct eal_tls_key { } int +rte_thread_get_priority(rte_thread_t thread_id, + enum rte_thread_priority *priority) +{ + int ret; + int policy; + struct sched_param param; + + ret = pthread_getschedparam((pthread_t)thread_id.opaque_id, &policy, + ¶m); + if (ret != 0) { + RTE_LOG(DEBUG, EAL, "pthread_getschedparam failed\n"); + goto cleanup; + } + + return thread_map_os_priority_to_eal_priority(policy, + param.sched_priority, priority); + +cleanup: + return ret; +} + +int +rte_thread_set_priority(rte_thread_t thread_id, + enum rte_thread_priority priority) +{ + int ret; + int policy; + struct sched_param param; + +/* Realtime priority can cause crashes on non-Windows platforms. */ + if (priority == RTE_THREAD_PRIORITY_REALTIME_CRITICAL) + return ENOTSUP; + + ret = thread_map_priority_to_os_value(priority, ¶m.sched_priority, + &policy); + if (ret != 0) + return ret; + + return pthread_setschedparam((pthread_t)thread_id.opaque_id, + policy, ¶m); +} + +int rte_thread_key_create(rte_thread_key *key, void (*destructor)(void *)) { int err; diff --git a/lib/eal/version.map b/lib/eal/version.map index d49e30b..ab27a4b 100644 --- a/lib/eal/version.map +++ b/lib/eal/version.map @@ -423,8 +423,10 @@ EXPERIMENTAL { # added in 22.07 rte_thread_get_affinity_by_id; + rte_thread_get_priority; rte_thread_self; rte_thread_set_affinity_by_id; + rte_thread_set_priority; }; INTERNAL { diff --git a/lib/eal/windows/rte_thread.c b/lib/eal/windows/rte_thread.c index a616703..495225e 100644 --- a/lib/eal/windows/rte_thread.c +++ b/lib/eal/windows/rte_thread.c @@ -61,6 +61,58 @@ struct eal_tls_key { return thread_translate_win32_error(error); } +static int +thread_map_priority_to_os_value(enum rte_thread_priority eal_pri, + int *os_pri, DWORD *pri_class) +{ + /* Clear the output parameters */ + *os_pri = -1; + *pri_class = -1; + + switch (eal_pri) { + case RTE_THREAD_PRIORITY_NORMAL: + *pri_class = NORMAL_PRIORITY_CLASS; + *os_pri = THREAD_PRIORITY_NORMAL; + break; + case RTE_THREAD_PRIORITY_REALTIME_CRITICAL: + *pri_class = REALTIME_PRIORITY_CLASS; + *os_pri = THREAD_PRIORITY_TIME_CRITICAL; + break; + default: + RTE_LOG(DEBUG, EAL, "The requested priority value is invalid.\n"); + return EINVAL; + } + + return 0; +} + +static int +thread_map_os_priority_to_eal_value(int os_pri, DWORD pri_class, + enum rte_thread_priority *eal_pri) +{ + switch (pri_class) { + case NORMAL_PRIORITY_CLASS: + if (os_pri == THREAD_PRIORITY_NORMAL) { + *eal_pri = RTE_THREAD_PRIORITY_NORMAL; + return 0; + } + break; + case HIGH_PRIORITY_CLASS: + RTE_LOG(WARNING, EAL, "The OS priority class is high not real-time.\n"); + /* FALLTHROUGH */ + case REALTIME_PRIORITY_CLASS: + if (os_pri == THREAD_PRIORITY_TIME_CRITICAL) { + *eal_pri = RTE_THREAD_PRIORITY_REALTIME_CRITICAL; + return 0; + } + break; + default: + RTE_LOG(DEBUG, EAL, "The OS priority value does not map to an EAL-defined priority.\n"); + return EINVAL; + } + return 0; +} + rte_thread_t rte_thread_self(void) { @@ -72,6 +124,84 @@ struct eal_tls_key { } int +rte_thread_get_priority(rte_thread_t thread_id, + enum rte_thread_priority *priority) +{ + HANDLE thread_handle = NULL; + DWORD pri_class; + int os_pri; + int ret; + + pri_class = GetPriorityClass(GetCurrentProcess()); + if (pri_class == 0) { + ret = thread_log_last_error("GetPriorityClass()"); + goto cleanup; + } + + thread_handle = OpenThread(THREAD_SET_INFORMATION | + THREAD_QUERY_INFORMATION, FALSE, thread_id.opaque_id); + if (thread_handle == NULL) { + ret = thread_log_last_error("OpenThread()"); + goto cleanup; + } + + os_pri = GetThreadPriority(thread_handle); + if (os_pri == THREAD_PRIORITY_ERROR_RETURN) { + ret = thread_log_last_error("GetThreadPriority()"); + goto cleanup; + } + + ret = thread_map_os_priority_to_eal_value(os_pri, pri_class, priority); + if (ret != 0) + goto cleanup; + +cleanup: + if (thread_handle != NULL) + CloseHandle(thread_handle); + + return ret; +} + +int +rte_thread_set_priority(rte_thread_t thread_id, + enum rte_thread_priority priority) +{ + HANDLE thread_handle; + DWORD priority_class; + int os_priority; + int ret = 0; + + thread_handle = OpenThread(THREAD_SET_INFORMATION | + THREAD_QUERY_INFORMATION, FALSE, + thread_id.opaque_id); + if (thread_handle == NULL) { + ret = thread_log_last_error("OpenThread()"); + goto cleanup; + } + + ret = thread_map_priority_to_os_value(priority, &os_priority, + &priority_class); + if (ret != 0) + goto cleanup; + + if (!SetPriorityClass(GetCurrentProcess(), priority_class)) { + ret = thread_log_last_error("SetPriorityClass()"); + goto cleanup; + } + + if (!SetThreadPriority(thread_handle, os_priority)) { + ret = thread_log_last_error("SetThreadPriority()"); + goto cleanup; + } + +cleanup: + if (thread_handle != NULL) + CloseHandle(thread_handle); + + return ret; +} + +int rte_thread_key_create(rte_thread_key *key, __rte_unused void (*destructor)(void *)) { -- 1.8.3.1 ^ permalink raw reply related [flat|nested] 15+ messages in thread
* [PATCH v2 2/2] test/threads: add unit test for get set priority 2022-05-23 13:07 ` [PATCH v2 0/2] add thread priority accessors Tyler Retzlaff 2022-05-23 13:07 ` [PATCH v2 1/2] eal: get/set thread priority per thread identifier Tyler Retzlaff @ 2022-05-23 13:07 ` Tyler Retzlaff 1 sibling, 0 replies; 15+ messages in thread From: Tyler Retzlaff @ 2022-05-23 13:07 UTC (permalink / raw) To: dev; +Cc: thomas, dmitry.kozliuk, anatoly.burakov, Tyler Retzlaff Add unit tests to exercise and demonstrate rte_thread_{get,set}_priority(). Signed-off-by: Tyler Retzlaff <roretzla@linux.microsoft.com> --- app/test/test_threads.c | 52 +++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 52 insertions(+) diff --git a/app/test/test_threads.c b/app/test/test_threads.c index e1a2ea5..dbc20e6 100644 --- a/app/test/test_threads.c +++ b/app/test/test_threads.c @@ -24,6 +24,56 @@ } static int +test_thread_priority(void) +{ + pthread_t id; + rte_thread_t thread_id; + enum rte_thread_priority priority; + + thread_id_ready = 0; + RTE_TEST_ASSERT(pthread_create(&id, NULL, thread_main, &thread_id) == 0, + "Failed to create thread"); + + while (__atomic_load_n(&thread_id_ready, __ATOMIC_ACQUIRE) == 0) + ; + + priority = RTE_THREAD_PRIORITY_NORMAL; + RTE_TEST_ASSERT(rte_thread_set_priority(thread_id, priority) == 0, + "Failed to set thread priority"); + RTE_TEST_ASSERT(rte_thread_get_priority(thread_id, &priority) == 0, + "Failed to get thread priority"); + RTE_TEST_ASSERT(priority == RTE_THREAD_PRIORITY_NORMAL, + "Priority set mismatches priority get"); + + priority = RTE_THREAD_PRIORITY_REALTIME_CRITICAL; +#ifndef RTE_EXEC_ENV_WINDOWS + RTE_TEST_ASSERT(rte_thread_set_priority(thread_id, priority) == -ENOTSUP, + "Priority set to critical should fail"); + RTE_TEST_ASSERT(rte_thread_get_priority(thread_id, &priority) == 0, + "Failed to get thread priority"); + RTE_TEST_ASSERT(priority == RTE_THREAD_PRIORITY_NORMAL, + "Failed set to critical should have retained normal"); +#else + RTE_TEST_ASSERT(rte_thread_set_priority(thread_id, priority) == 0, + "Priority set to critical should succeed"); + RTE_TEST_ASSERT(rte_thread_get_priority(thread_id, &priority) == 0, + "Failed to get thread priority"); + RTE_TEST_ASSERT(priority == RTE_THREAD_PRIORITY_REALTIME_CRITICAL, + "Priority set mismatches priority get"); +#endif + + priority = RTE_THREAD_PRIORITY_NORMAL; + RTE_TEST_ASSERT(rte_thread_set_priority(thread_id, priority) == 0, + "Failed to set thread priority"); + RTE_TEST_ASSERT(rte_thread_get_priority(thread_id, &priority) == 0, + "Failed to get thread priority"); + RTE_TEST_ASSERT(priority == RTE_THREAD_PRIORITY_NORMAL, + "Priority set mismatches priority get"); + + return 0; +} + +static int test_thread_affinity(void) { pthread_t id; @@ -31,6 +81,7 @@ rte_cpuset_t cpuset0; rte_cpuset_t cpuset1; + thread_id_ready = 0; RTE_TEST_ASSERT(pthread_create(&id, NULL, thread_main, &thread_id) == 0, "Failed to create thread"); @@ -68,6 +119,7 @@ .teardown = NULL, .unit_test_cases = { TEST_CASE(test_thread_affinity), + TEST_CASE(test_thread_priority), TEST_CASES_END() } }; -- 1.8.3.1 ^ permalink raw reply related [flat|nested] 15+ messages in thread
* [PATCH v3 0/2] add thread priority accessors 2022-05-23 10:46 [PATCH 0/2] add thread priority accessors Tyler Retzlaff ` (2 preceding siblings ...) 2022-05-23 13:07 ` [PATCH v2 0/2] add thread priority accessors Tyler Retzlaff @ 2022-05-24 11:08 ` Tyler Retzlaff 2022-05-24 11:08 ` [PATCH v3 1/2] eal: get/set thread priority per thread identifier Tyler Retzlaff ` (2 more replies) 3 siblings, 3 replies; 15+ messages in thread From: Tyler Retzlaff @ 2022-05-24 11:08 UTC (permalink / raw) To: dev; +Cc: thomas, dmitry.kozliuk, anatoly.burakov, Tyler Retzlaff this series introduces accessors for get and set of thread priority based on original patches from Narcisa Vasile. v3: * synchronize exit of thread_main to maintain thread life for duration of affinity and priority tests. * fix test check of errno return value should be non-negative ENOTSUP instead of -ENOTSUP v2: * fix typo in test assert string mistmatch -> mismatch * add new experimental exports names to lib/eal/version.map Tyler Retzlaff (2): eal: get/set thread priority per thread identifier test/threads: add unit test for get set priority app/test/test_threads.c | 57 +++++++++++++++++++ lib/eal/include/rte_thread.h | 50 +++++++++++++++++ lib/eal/unix/rte_thread.c | 101 +++++++++++++++++++++++++++++++++ lib/eal/version.map | 2 + lib/eal/windows/rte_thread.c | 130 +++++++++++++++++++++++++++++++++++++++++++ 5 files changed, 340 insertions(+) -- 1.8.3.1 ^ permalink raw reply [flat|nested] 15+ messages in thread
* [PATCH v3 1/2] eal: get/set thread priority per thread identifier 2022-05-24 11:08 ` [PATCH v3 0/2] add thread priority accessors Tyler Retzlaff @ 2022-05-24 11:08 ` Tyler Retzlaff 2022-05-24 14:51 ` Stephen Hemminger 2022-05-24 11:08 ` [PATCH v3 2/2] test/threads: add unit test for get set priority Tyler Retzlaff 2022-06-07 10:39 ` [PATCH v3 0/2] add thread priority accessors David Marchand 2 siblings, 1 reply; 15+ messages in thread From: Tyler Retzlaff @ 2022-05-24 11:08 UTC (permalink / raw) To: dev Cc: thomas, dmitry.kozliuk, anatoly.burakov, Tyler Retzlaff, Narcisa Vasile Add functions for setting and getting the priority of a thread. Priorities on multiple platforms are similarly determined by a priority value and a priority class/policy. Currently in DPDK most threads operate at the OS-default priority level but there are cases when increasing the priority is useful. For example, high performance applications may require elevated priority levels. For these reasons, EAL will expose two priority levels which are named suggestively "normal" and "realtime_critical" and are computed as follows: On Linux, the following mapping is created: RTE_THREAD_PRIORITY_NORMAL corresponds to * policy SCHED_OTHER * priority value: (sched_get_priority_min(SCHED_OTHER) + sched_get_priority_max(SCHED_OTHER))/2; RTE_THREAD_PRIORITY_REALTIME_CRITICAL corresponds to * policy SCHED_RR * priority value: sched_get_priority_max(SCHED_RR); On Windows, the following mapping is created: RTE_THREAD_PRIORITY_NORMAL corresponds to * class NORMAL_PRIORITY_CLASS * priority THREAD_PRIORITY_NORMAL RTE_THREAD_PRIORITY_REALTIME_CRITICAL corresponds to * class REALTIME_PRIORITY_CLASS (when running with privileges) * class HIGH_PRIORITY_CLASS (when running without privileges) * priority THREAD_PRIORITY_TIME_CRITICAL Note that on Linux the resulting priority value will be 0, in accordance to the documentation that mention the value should be 0 for SCHED_OTHER policy. Signed-off-by: Narcisa Vasile <navasile@linux.microsoft.com> Signed-off-by: Tyler Retzlaff <roretzla@linux.microsoft.com> --- lib/eal/include/rte_thread.h | 50 +++++++++++++++++ lib/eal/unix/rte_thread.c | 101 +++++++++++++++++++++++++++++++++ lib/eal/version.map | 2 + lib/eal/windows/rte_thread.c | 130 +++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 283 insertions(+) diff --git a/lib/eal/include/rte_thread.h b/lib/eal/include/rte_thread.h index 7888f7a..9e261bf 100644 --- a/lib/eal/include/rte_thread.h +++ b/lib/eal/include/rte_thread.h @@ -31,6 +31,16 @@ } rte_thread_t; /** + * Thread priority values. + */ +enum rte_thread_priority { + RTE_THREAD_PRIORITY_NORMAL = 0, + /**< normal thread priority, the default */ + RTE_THREAD_PRIORITY_REALTIME_CRITICAL = 1, + /**< highest thread priority allowed */ +}; + +/** * TLS key type, an opaque pointer. */ typedef struct eal_tls_key *rte_thread_key; @@ -115,6 +125,46 @@ int rte_thread_get_affinity_by_id(rte_thread_t thread_id, #endif /* RTE_HAS_CPUSET */ /** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Get the priority of a thread. + * + * @param thread_id + * Id of the thread for which to get priority. + * + * @param priority + * Location to store the retrieved priority. + * + * @return + * On success, return 0. + * On failure, return a positive errno-style error number. + */ +__rte_experimental +int rte_thread_get_priority(rte_thread_t thread_id, + enum rte_thread_priority *priority); + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Set the priority of a thread. + * + * @param thread_id + * Id of the thread for which to set priority. + * + * @param priority + * Priority value to be set. + * + * @return + * On success, return 0. + * On failure, return a positive errno-style error number. + */ +__rte_experimental +int rte_thread_set_priority(rte_thread_t thread_id, + enum rte_thread_priority priority); + +/** * Create a TLS data key visible to all threads in the process. * the created key is later used to get/set a value. * and optional destructor can be set to be called when a thread exits. diff --git a/lib/eal/unix/rte_thread.c b/lib/eal/unix/rte_thread.c index 9e5fa47..2b19f4a 100644 --- a/lib/eal/unix/rte_thread.c +++ b/lib/eal/unix/rte_thread.c @@ -16,6 +16,64 @@ struct eal_tls_key { pthread_key_t thread_index; }; +static int +thread_map_priority_to_os_value(enum rte_thread_priority eal_pri, + int *os_pri, int *pol) +{ + /* Clear the output parameters */ + *os_pri = sched_get_priority_min(SCHED_OTHER) - 1; + *pol = -1; + + switch (eal_pri) { + case RTE_THREAD_PRIORITY_NORMAL: + *pol = SCHED_OTHER; + + /* + * Choose the middle of the range to represent + * the priority 'normal'. + * On Linux, this should be 0, since both + * sched_get_priority_min/_max return 0 for SCHED_OTHER. + */ + *os_pri = (sched_get_priority_min(SCHED_OTHER) + + sched_get_priority_max(SCHED_OTHER))/2; + break; + case RTE_THREAD_PRIORITY_REALTIME_CRITICAL: + *pol = SCHED_RR; + *os_pri = sched_get_priority_max(SCHED_RR); + break; + default: + RTE_LOG(DEBUG, EAL, "The requested priority value is invalid.\n"); + return EINVAL; + } + return 0; +} + +static int +thread_map_os_priority_to_eal_priority(int policy, int os_pri, + enum rte_thread_priority *eal_pri) +{ + switch (policy) { + case SCHED_OTHER: + if (os_pri == (sched_get_priority_min(SCHED_OTHER) + + sched_get_priority_max(SCHED_OTHER))/2) { + *eal_pri = RTE_THREAD_PRIORITY_NORMAL; + return 0; + } + break; + case SCHED_RR: + if (os_pri == sched_get_priority_max(SCHED_RR)) { + *eal_pri = RTE_THREAD_PRIORITY_REALTIME_CRITICAL; + return 0; + } + break; + default: + RTE_LOG(DEBUG, EAL, "The OS priority value does not map to an EAL-defined priority.\n"); + return EINVAL; + } + + return 0; +} + rte_thread_t rte_thread_self(void) { @@ -29,6 +87,49 @@ struct eal_tls_key { } int +rte_thread_get_priority(rte_thread_t thread_id, + enum rte_thread_priority *priority) +{ + int ret; + int policy; + struct sched_param param; + + ret = pthread_getschedparam((pthread_t)thread_id.opaque_id, &policy, + ¶m); + if (ret != 0) { + RTE_LOG(DEBUG, EAL, "pthread_getschedparam failed\n"); + goto cleanup; + } + + return thread_map_os_priority_to_eal_priority(policy, + param.sched_priority, priority); + +cleanup: + return ret; +} + +int +rte_thread_set_priority(rte_thread_t thread_id, + enum rte_thread_priority priority) +{ + int ret; + int policy; + struct sched_param param; + +/* Realtime priority can cause crashes on non-Windows platforms. */ + if (priority == RTE_THREAD_PRIORITY_REALTIME_CRITICAL) + return ENOTSUP; + + ret = thread_map_priority_to_os_value(priority, ¶m.sched_priority, + &policy); + if (ret != 0) + return ret; + + return pthread_setschedparam((pthread_t)thread_id.opaque_id, + policy, ¶m); +} + +int rte_thread_key_create(rte_thread_key *key, void (*destructor)(void *)) { int err; diff --git a/lib/eal/version.map b/lib/eal/version.map index d49e30b..ab27a4b 100644 --- a/lib/eal/version.map +++ b/lib/eal/version.map @@ -423,8 +423,10 @@ EXPERIMENTAL { # added in 22.07 rte_thread_get_affinity_by_id; + rte_thread_get_priority; rte_thread_self; rte_thread_set_affinity_by_id; + rte_thread_set_priority; }; INTERNAL { diff --git a/lib/eal/windows/rte_thread.c b/lib/eal/windows/rte_thread.c index a616703..495225e 100644 --- a/lib/eal/windows/rte_thread.c +++ b/lib/eal/windows/rte_thread.c @@ -61,6 +61,58 @@ struct eal_tls_key { return thread_translate_win32_error(error); } +static int +thread_map_priority_to_os_value(enum rte_thread_priority eal_pri, + int *os_pri, DWORD *pri_class) +{ + /* Clear the output parameters */ + *os_pri = -1; + *pri_class = -1; + + switch (eal_pri) { + case RTE_THREAD_PRIORITY_NORMAL: + *pri_class = NORMAL_PRIORITY_CLASS; + *os_pri = THREAD_PRIORITY_NORMAL; + break; + case RTE_THREAD_PRIORITY_REALTIME_CRITICAL: + *pri_class = REALTIME_PRIORITY_CLASS; + *os_pri = THREAD_PRIORITY_TIME_CRITICAL; + break; + default: + RTE_LOG(DEBUG, EAL, "The requested priority value is invalid.\n"); + return EINVAL; + } + + return 0; +} + +static int +thread_map_os_priority_to_eal_value(int os_pri, DWORD pri_class, + enum rte_thread_priority *eal_pri) +{ + switch (pri_class) { + case NORMAL_PRIORITY_CLASS: + if (os_pri == THREAD_PRIORITY_NORMAL) { + *eal_pri = RTE_THREAD_PRIORITY_NORMAL; + return 0; + } + break; + case HIGH_PRIORITY_CLASS: + RTE_LOG(WARNING, EAL, "The OS priority class is high not real-time.\n"); + /* FALLTHROUGH */ + case REALTIME_PRIORITY_CLASS: + if (os_pri == THREAD_PRIORITY_TIME_CRITICAL) { + *eal_pri = RTE_THREAD_PRIORITY_REALTIME_CRITICAL; + return 0; + } + break; + default: + RTE_LOG(DEBUG, EAL, "The OS priority value does not map to an EAL-defined priority.\n"); + return EINVAL; + } + return 0; +} + rte_thread_t rte_thread_self(void) { @@ -72,6 +124,84 @@ struct eal_tls_key { } int +rte_thread_get_priority(rte_thread_t thread_id, + enum rte_thread_priority *priority) +{ + HANDLE thread_handle = NULL; + DWORD pri_class; + int os_pri; + int ret; + + pri_class = GetPriorityClass(GetCurrentProcess()); + if (pri_class == 0) { + ret = thread_log_last_error("GetPriorityClass()"); + goto cleanup; + } + + thread_handle = OpenThread(THREAD_SET_INFORMATION | + THREAD_QUERY_INFORMATION, FALSE, thread_id.opaque_id); + if (thread_handle == NULL) { + ret = thread_log_last_error("OpenThread()"); + goto cleanup; + } + + os_pri = GetThreadPriority(thread_handle); + if (os_pri == THREAD_PRIORITY_ERROR_RETURN) { + ret = thread_log_last_error("GetThreadPriority()"); + goto cleanup; + } + + ret = thread_map_os_priority_to_eal_value(os_pri, pri_class, priority); + if (ret != 0) + goto cleanup; + +cleanup: + if (thread_handle != NULL) + CloseHandle(thread_handle); + + return ret; +} + +int +rte_thread_set_priority(rte_thread_t thread_id, + enum rte_thread_priority priority) +{ + HANDLE thread_handle; + DWORD priority_class; + int os_priority; + int ret = 0; + + thread_handle = OpenThread(THREAD_SET_INFORMATION | + THREAD_QUERY_INFORMATION, FALSE, + thread_id.opaque_id); + if (thread_handle == NULL) { + ret = thread_log_last_error("OpenThread()"); + goto cleanup; + } + + ret = thread_map_priority_to_os_value(priority, &os_priority, + &priority_class); + if (ret != 0) + goto cleanup; + + if (!SetPriorityClass(GetCurrentProcess(), priority_class)) { + ret = thread_log_last_error("SetPriorityClass()"); + goto cleanup; + } + + if (!SetThreadPriority(thread_handle, os_priority)) { + ret = thread_log_last_error("SetThreadPriority()"); + goto cleanup; + } + +cleanup: + if (thread_handle != NULL) + CloseHandle(thread_handle); + + return ret; +} + +int rte_thread_key_create(rte_thread_key *key, __rte_unused void (*destructor)(void *)) { -- 1.8.3.1 ^ permalink raw reply related [flat|nested] 15+ messages in thread
* Re: [PATCH v3 1/2] eal: get/set thread priority per thread identifier 2022-05-24 11:08 ` [PATCH v3 1/2] eal: get/set thread priority per thread identifier Tyler Retzlaff @ 2022-05-24 14:51 ` Stephen Hemminger 2022-05-26 6:29 ` Tyler Retzlaff 0 siblings, 1 reply; 15+ messages in thread From: Stephen Hemminger @ 2022-05-24 14:51 UTC (permalink / raw) To: Tyler Retzlaff Cc: dev, thomas, dmitry.kozliuk, anatoly.burakov, Narcisa Vasile On Tue, 24 May 2022 04:08:36 -0700 Tyler Retzlaff <roretzla@linux.microsoft.com> wrote: > +static int > +thread_map_priority_to_os_value(enum rte_thread_priority eal_pri, > + int *os_pri, int *pol) > +{ > + /* Clear the output parameters */ > + *os_pri = sched_get_priority_min(SCHED_OTHER) - 1; > + *pol = -1; > + > + switch (eal_pri) { > + case RTE_THREAD_PRIORITY_NORMAL: > + *pol = SCHED_OTHER; > + > + /* > + * Choose the middle of the range to represent > + * the priority 'normal'. > + * On Linux, this should be 0, since both > + * sched_get_priority_min/_max return 0 for SCHED_OTHER. > + */ > + *os_pri = (sched_get_priority_min(SCHED_OTHER) + > + sched_get_priority_max(SCHED_OTHER))/2; > + break; > + case RTE_THREAD_PRIORITY_REALTIME_CRITICAL: > + *pol = SCHED_RR; > + *os_pri = sched_get_priority_max(SCHED_RR); > + break; Many people have experimented with realtime priorities with DPDK and Linux, and have never heard of any one not having problems. Recommend that this either be an error add a warning log message that "Setting real time priority may break" > + default: > + RTE_LOG(DEBUG, EAL, "The requested priority value is invalid.\n"); > + return EINVAL; > + } > + return 0; > +} ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH v3 1/2] eal: get/set thread priority per thread identifier 2022-05-24 14:51 ` Stephen Hemminger @ 2022-05-26 6:29 ` Tyler Retzlaff 2022-05-26 15:21 ` Stephen Hemminger 0 siblings, 1 reply; 15+ messages in thread From: Tyler Retzlaff @ 2022-05-26 6:29 UTC (permalink / raw) To: Stephen Hemminger Cc: dev, thomas, dmitry.kozliuk, anatoly.burakov, Narcisa Vasile On Tue, May 24, 2022 at 07:51:05AM -0700, Stephen Hemminger wrote: > On Tue, 24 May 2022 04:08:36 -0700 > Tyler Retzlaff <roretzla@linux.microsoft.com> wrote: > > > +static int > > +thread_map_priority_to_os_value(enum rte_thread_priority eal_pri, > > + int *os_pri, int *pol) > > +{ > > + /* Clear the output parameters */ > > + *os_pri = sched_get_priority_min(SCHED_OTHER) - 1; > > + *pol = -1; > > + > > + switch (eal_pri) { > > + case RTE_THREAD_PRIORITY_NORMAL: > > + *pol = SCHED_OTHER; > > + > > + /* > > + * Choose the middle of the range to represent > > + * the priority 'normal'. > > + * On Linux, this should be 0, since both > > + * sched_get_priority_min/_max return 0 for SCHED_OTHER. > > + */ > > + *os_pri = (sched_get_priority_min(SCHED_OTHER) + > > + sched_get_priority_max(SCHED_OTHER))/2; > > + break; > > + case RTE_THREAD_PRIORITY_REALTIME_CRITICAL: > > + *pol = SCHED_RR; > > + *os_pri = sched_get_priority_max(SCHED_RR); > > + break; > > Many people have experimented with realtime priorities with DPDK > and Linux, and have never heard of any one not having problems. > > Recommend that this either be an error add a warning log message > that "Setting real time priority may break" so i went back through the feedback when the priority change was originally introduced. you're request was that you didn't want it to be possible for realtime priority to be set on linux. this particular function is just about mapping the abstracted representation of realtime priority in eal to the platform specific linux representation. so if you take a look at the other parts of the patch you will notice that the rte_thread_set_priority() for linux will in fact fail if you try to use realtime priority. additionally, the unit test exercises that attempting to use realtime priority does fail for linux. take another look and let me know if you don't agree. thanks ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH v3 1/2] eal: get/set thread priority per thread identifier 2022-05-26 6:29 ` Tyler Retzlaff @ 2022-05-26 15:21 ` Stephen Hemminger 2022-05-27 11:15 ` Tyler Retzlaff 0 siblings, 1 reply; 15+ messages in thread From: Stephen Hemminger @ 2022-05-26 15:21 UTC (permalink / raw) To: Tyler Retzlaff Cc: dev, thomas, dmitry.kozliuk, anatoly.burakov, Narcisa Vasile On Wed, 25 May 2022 23:29:57 -0700 Tyler Retzlaff <roretzla@linux.microsoft.com> wrote: > On Tue, May 24, 2022 at 07:51:05AM -0700, Stephen Hemminger wrote: > > On Tue, 24 May 2022 04:08:36 -0700 > > Tyler Retzlaff <roretzla@linux.microsoft.com> wrote: > > > > > +static int > > > +thread_map_priority_to_os_value(enum rte_thread_priority eal_pri, > > > + int *os_pri, int *pol) > > > +{ > > > + /* Clear the output parameters */ > > > + *os_pri = sched_get_priority_min(SCHED_OTHER) - 1; > > > + *pol = -1; > > > + > > > + switch (eal_pri) { > > > + case RTE_THREAD_PRIORITY_NORMAL: > > > + *pol = SCHED_OTHER; > > > + > > > + /* > > > + * Choose the middle of the range to represent > > > + * the priority 'normal'. > > > + * On Linux, this should be 0, since both > > > + * sched_get_priority_min/_max return 0 for SCHED_OTHER. > > > + */ > > > + *os_pri = (sched_get_priority_min(SCHED_OTHER) + > > > + sched_get_priority_max(SCHED_OTHER))/2; > > > + break; > > > + case RTE_THREAD_PRIORITY_REALTIME_CRITICAL: > > > + *pol = SCHED_RR; > > > + *os_pri = sched_get_priority_max(SCHED_RR); > > > + break; > > > > Many people have experimented with realtime priorities with DPDK > > and Linux, and have never heard of any one not having problems. > > > > Recommend that this either be an error add a warning log message > > that "Setting real time priority may break" > > so i went back through the feedback when the priority change was > originally introduced. you're request was that you didn't want it to be > possible for realtime priority to be set on linux. > > this particular function is just about mapping the abstracted > representation of realtime priority in eal to the platform specific > linux representation. > > so if you take a look at the other parts of the patch you will notice > that the rte_thread_set_priority() for linux will in fact fail if you > try to use realtime priority. > > additionally, the unit test exercises that attempting to use realtime > priority does fail for linux. > > take another look and let me know if you don't agree. > > thanks Thanks for clarifying. Is there any analog for SCHED_DEADLINE worth considering? Acked-by: Stephen Hemminger <stephen@networkplumber.org> ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH v3 1/2] eal: get/set thread priority per thread identifier 2022-05-26 15:21 ` Stephen Hemminger @ 2022-05-27 11:15 ` Tyler Retzlaff 0 siblings, 0 replies; 15+ messages in thread From: Tyler Retzlaff @ 2022-05-27 11:15 UTC (permalink / raw) To: Stephen Hemminger Cc: dev, thomas, dmitry.kozliuk, anatoly.burakov, Narcisa Vasile On Thu, May 26, 2022 at 08:21:27AM -0700, Stephen Hemminger wrote: > > Thanks for clarifying. > Is there any analog for SCHED_DEADLINE worth considering? i hadn't considered it. for now i probably won't since this patch is just a blocker for getting the thread lifetime series up. which is higher priority as i understand. ^ permalink raw reply [flat|nested] 15+ messages in thread
* [PATCH v3 2/2] test/threads: add unit test for get set priority 2022-05-24 11:08 ` [PATCH v3 0/2] add thread priority accessors Tyler Retzlaff 2022-05-24 11:08 ` [PATCH v3 1/2] eal: get/set thread priority per thread identifier Tyler Retzlaff @ 2022-05-24 11:08 ` Tyler Retzlaff 2022-06-07 10:39 ` [PATCH v3 0/2] add thread priority accessors David Marchand 2 siblings, 0 replies; 15+ messages in thread From: Tyler Retzlaff @ 2022-05-24 11:08 UTC (permalink / raw) To: dev; +Cc: thomas, dmitry.kozliuk, anatoly.burakov, Tyler Retzlaff Add unit tests to exercise and demonstrate rte_thread_{get,set}_priority(). Signed-off-by: Tyler Retzlaff <roretzla@linux.microsoft.com> --- app/test/test_threads.c | 57 +++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 57 insertions(+) diff --git a/app/test/test_threads.c b/app/test/test_threads.c index e1a2ea5..b9d8b4e 100644 --- a/app/test/test_threads.c +++ b/app/test/test_threads.c @@ -20,10 +20,65 @@ *(rte_thread_t *)arg = rte_thread_self(); __atomic_store_n(&thread_id_ready, 1, __ATOMIC_RELEASE); + while (__atomic_load_n(&thread_id_ready, __ATOMIC_ACQUIRE) == 1) + ; + return NULL; } static int +test_thread_priority(void) +{ + pthread_t id; + rte_thread_t thread_id; + enum rte_thread_priority priority; + + thread_id_ready = 0; + RTE_TEST_ASSERT(pthread_create(&id, NULL, thread_main, &thread_id) == 0, + "Failed to create thread"); + + while (__atomic_load_n(&thread_id_ready, __ATOMIC_ACQUIRE) == 0) + ; + + priority = RTE_THREAD_PRIORITY_NORMAL; + RTE_TEST_ASSERT(rte_thread_set_priority(thread_id, priority) == 0, + "Failed to set thread priority"); + RTE_TEST_ASSERT(rte_thread_get_priority(thread_id, &priority) == 0, + "Failed to get thread priority"); + RTE_TEST_ASSERT(priority == RTE_THREAD_PRIORITY_NORMAL, + "Priority set mismatches priority get"); + + priority = RTE_THREAD_PRIORITY_REALTIME_CRITICAL; +#ifndef RTE_EXEC_ENV_WINDOWS + RTE_TEST_ASSERT(rte_thread_set_priority(thread_id, priority) == ENOTSUP, + "Priority set to critical should fail"); + RTE_TEST_ASSERT(rte_thread_get_priority(thread_id, &priority) == 0, + "Failed to get thread priority"); + RTE_TEST_ASSERT(priority == RTE_THREAD_PRIORITY_NORMAL, + "Failed set to critical should have retained normal"); +#else + RTE_TEST_ASSERT(rte_thread_set_priority(thread_id, priority) == 0, + "Priority set to critical should succeed"); + RTE_TEST_ASSERT(rte_thread_get_priority(thread_id, &priority) == 0, + "Failed to get thread priority"); + RTE_TEST_ASSERT(priority == RTE_THREAD_PRIORITY_REALTIME_CRITICAL, + "Priority set mismatches priority get"); +#endif + + priority = RTE_THREAD_PRIORITY_NORMAL; + RTE_TEST_ASSERT(rte_thread_set_priority(thread_id, priority) == 0, + "Failed to set thread priority"); + RTE_TEST_ASSERT(rte_thread_get_priority(thread_id, &priority) == 0, + "Failed to get thread priority"); + RTE_TEST_ASSERT(priority == RTE_THREAD_PRIORITY_NORMAL, + "Priority set mismatches priority get"); + + __atomic_store_n(&thread_id_ready, 2, __ATOMIC_RELEASE); + + return 0; +} + +static int test_thread_affinity(void) { pthread_t id; @@ -31,6 +86,7 @@ rte_cpuset_t cpuset0; rte_cpuset_t cpuset1; + thread_id_ready = 0; RTE_TEST_ASSERT(pthread_create(&id, NULL, thread_main, &thread_id) == 0, "Failed to create thread"); @@ -68,6 +124,7 @@ .teardown = NULL, .unit_test_cases = { TEST_CASE(test_thread_affinity), + TEST_CASE(test_thread_priority), TEST_CASES_END() } }; -- 1.8.3.1 ^ permalink raw reply related [flat|nested] 15+ messages in thread
* Re: [PATCH v3 0/2] add thread priority accessors 2022-05-24 11:08 ` [PATCH v3 0/2] add thread priority accessors Tyler Retzlaff 2022-05-24 11:08 ` [PATCH v3 1/2] eal: get/set thread priority per thread identifier Tyler Retzlaff 2022-05-24 11:08 ` [PATCH v3 2/2] test/threads: add unit test for get set priority Tyler Retzlaff @ 2022-06-07 10:39 ` David Marchand 2 siblings, 0 replies; 15+ messages in thread From: David Marchand @ 2022-06-07 10:39 UTC (permalink / raw) To: Tyler Retzlaff; +Cc: dev, Thomas Monjalon, Dmitry Kozlyuk, Burakov, Anatoly On Tue, May 24, 2022 at 1:09 PM Tyler Retzlaff <roretzla@linux.microsoft.com> wrote: > > this series introduces accessors for get and set of thread > priority based on original patches from Narcisa Vasile. > > v3: > * synchronize exit of thread_main to maintain thread > life for duration of affinity and priority tests. > * fix test check of errno return value should be > non-negative ENOTSUP instead of -ENOTSUP > > v2: > * fix typo in test assert string mistmatch -> mismatch > * add new experimental exports names to lib/eal/version.map > > Tyler Retzlaff (2): > eal: get/set thread priority per thread identifier > test/threads: add unit test for get set priority > > app/test/test_threads.c | 57 +++++++++++++++++++ > lib/eal/include/rte_thread.h | 50 +++++++++++++++++ > lib/eal/unix/rte_thread.c | 101 +++++++++++++++++++++++++++++++++ > lib/eal/version.map | 2 + > lib/eal/windows/rte_thread.c | 130 +++++++++++++++++++++++++++++++++++++++++++ > 5 files changed, 340 insertions(+) Series applied, thanks. -- David Marchand ^ permalink raw reply [flat|nested] 15+ messages in thread
end of thread, other threads:[~2022-06-07 10:39 UTC | newest] Thread overview: 15+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2022-05-23 10:46 [PATCH 0/2] add thread priority accessors Tyler Retzlaff 2022-05-23 10:46 ` [PATCH 1/2] eal: get/set thread priority per thread identifier Tyler Retzlaff 2022-05-23 10:46 ` [PATCH 2/2] test/threads: add unit test for get set priority Tyler Retzlaff 2022-05-23 10:52 ` Tyler Retzlaff 2022-05-23 13:07 ` [PATCH v2 0/2] add thread priority accessors Tyler Retzlaff 2022-05-23 13:07 ` [PATCH v2 1/2] eal: get/set thread priority per thread identifier Tyler Retzlaff 2022-05-23 13:07 ` [PATCH v2 2/2] test/threads: add unit test for get set priority Tyler Retzlaff 2022-05-24 11:08 ` [PATCH v3 0/2] add thread priority accessors Tyler Retzlaff 2022-05-24 11:08 ` [PATCH v3 1/2] eal: get/set thread priority per thread identifier Tyler Retzlaff 2022-05-24 14:51 ` Stephen Hemminger 2022-05-26 6:29 ` Tyler Retzlaff 2022-05-26 15:21 ` Stephen Hemminger 2022-05-27 11:15 ` Tyler Retzlaff 2022-05-24 11:08 ` [PATCH v3 2/2] test/threads: add unit test for get set priority Tyler Retzlaff 2022-06-07 10:39 ` [PATCH v3 0/2] add thread priority accessors David Marchand
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.