Differences

This shows you the differences between two versions of the page.

--- realtime:documentation:howto:applications:application_base [2017/07/15 01:37]
jithu [Example]
+++ realtime:documentation:howto:applications:application_base [2024/05/31 15:54] (current)
alison [Scheduling and priority]
@@ Line 14: / Line 14: @@
 ==== Scheduling and priority ====
-The [[realtime:documentation:technical_basics:sched_policy_prio|scheduling policy]] as well as the priority
+The [[realtime:documentation:technical_basics:sched_policy_prio:start|scheduling policy]] as well as the priority
 must be set by the application explicitly. There are two possibilities
 for this:
@@ Line 30: / Line 30: @@
 the pthread attributes and not to use the inherit scheduling of the
 thread which created the real-time thread.
+</wrap>
+  - **Problems with pthread condition variables** \\ <wrap>
+Multithreaded applications which rely on glibc's libpthread are prone to unexpected latency delays since its condition variable implementation does not honor priority inheritance ([[https://sourceware.org/bugzilla/show_bug.cgi?id=11588|bugzilla]]). Unfortunately glibc's DNS resolver and asynchronous I/O implementations depend in turn on these condition variables.
+[[https://github.com/dvhart/librtpi|librtpi]] is an alternative LGPL-licensed pthread implementation which supports priority inheritance, and whose API is as close to glibc's as possible.  The alternative [[https://www.musl-libc.org/|MUSL libc]] has a pthread condition variable implementation similar to glibc's.
 </wrap>
 ==== Memory locking ====
-In real-time applications it is important to avoid non-deterministic
+See [[realtime:documentation:howto:applications:memory#Memory Locking | here]]
-behavior.  If the memory that is needed by the real-time application
-is not locked in the RAM, this memory could be paged out.  If the
+==== Stack for RT thread ====
-memory is not paged in when the application tries to access the
-memory, a page fault occurs causing non-deterministic high latency.
+See [[realtime:documentation:howto:applications:memory#Stack Memory for RT threads | here]]
-For this reason memory should be locked in real-time applications.
-The memory lock persists until the process owning it terminates or
+==== Capabilities: running the app with RT priority as a non-root user ====
-explicitly unlocks it by calling ''munlock()'' or ''munlockall()''.
-Be aware that page faults due to paged out memory occur in systems
-with swap as well as in systems without swap.  In addition, the binary
-of the executed application itself could be paged out.
-The following call of ''mlockall()'' locks all current pages mapped
+Several of the Pthread APIs, like ''mlockall()'', ''pthread_attr_setschedpolicy()'', by default and convention require root in order to successfully get their work done. Thus, RT apps - which need to set an RT sched policy and priority -  are often run via ''sudo''.
-into the address space of the process as well as all pages that will
-be mapped in the future.
-  mlockall(MCL_CURRENT|MCL_FUTURE);
+There's a far better approach to this; ''sudo'' gives the process root capabilities. This interests hackers :-).
+Instead, you should leverage the powerful POSIX **Capabilities** model! This way, the process (and threads) get _only_ the capabilities they require and nothing more. This follows the infosec best practice, the //principle of least privilege//.
-==== Stack prefaulting ====
+Apps start out with no capabilities by default; also note that capabilities are a per-thread resource (essentially translating to bitmasks with the task structure,which is per-thread of course). Among the various capability bits, the man page on ''capabilities(7)'' shows that **''CAP_SYS_NICE''** is the appropriate capability to use in this circumstance; a snippet from the ''capabilities(7)'' man page reveals this:
-Since page faults cause non-deterministic behavior, the stack should
+...
-be prefaulted before the real-time critical section starts.
+**CAP_SYS_NICE**
-In case several real-time threads are used, it should be done for each
+  * Lower the process nice value (nice(2), setpriority(2)) and change the nice value for arbitrary processes;
-thread individually.  In the following example, a memory block of a
+  * **set  real-time  scheduling  policies  for  calling process, and set scheduling policies and priorities for arbitrary processes (sched_setscheduler(2), sched_setparam(2), sched_setattr(2));**
-certain size is allocated.  All of its pages are touched to get them
+  * set CPU affinity for arbitrary processes (sched_setaffinity(2));
-mapped into RAM ensuring that no page faults occur later.
+  * set I/O scheduling class and priority for arbitrary processes (ioprio_set(2));
+  * apply migrate_pages(2) to arbitrary processes and allow processes to be migrated to arbitrary nodes;
+  * apply move_pages(2) to arbitrary processes;
+  * use the MPOL_MF_MOVE_ALL flag with mbind(2) and move_pages(2). ...
-  void *buffer;
-  buffer = mmap(NULL, MSIZE, PROT_READ | PROT_WRITE,
+**//Ok, great, but how exactly is this capability bit to be set on the app?//
-                MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
+**
-  memset(&buffer, 0, MSIZE);
+  - One approach is to do so programatically, via the ''capget()/capset()'' system calls. (Note that's it's generally easier to use the libcap library wrappers, ''cap_[g|s]et_proc(3)'': [[https://man7.org/linux/man-pages/man3/cap_get_proc.3.html]]. This man page even provides a small example of doing so).
+  - Another easy way is to leverage systemd and run your app as a service; in the service unit, specify the capability (see the man page on systemd.exec(5); [[https://www.freedesktop.org/software/systemd/man/systemd.exec.html#Capabilities]].
+  - Perhaps the easiest way: via the ''setcap(8)'' utility (it's man page: [[https://man7.org/linux/man-pages/man8/setcap.8.html]]). The setcap/getcap are typically part of the libcap package. For example:
+    ''sudo setcap CAP_SYS_NICE+eip <your-app-binary-executable>''
-The prefaulted stack can be assigned to a thread.  (''&attr'' is a
+You could put this line in the app Makefile (or equivalent).
-''pthread_attr_t'' pointer; the pthread attribute needs to have been
+(The ''getcap(8)'' utility can be used to verify that the 'dumb-capability' binary now has the ''CAP_SYS_NICE'' bit set)!
-previously initialized):
-  pthread_attr_setstack(&attr, buffer, PTHREAD_STACK_MIN);
+And you're all set to run it as non-root now, a much more secure approach.
 ===== Example =====

Wiki

User Tools

Site Tools

Differences

Page Tools