This shows you the differences between two versions of the page.
Both sides previous revision Previous revision Next revision | Previous revision | ||
realtime:documentation:howto:applications:application_base [2023/08/29 07:25] kaiwan |
realtime:documentation:howto:applications:application_base [2024/05/31 15:54] (current) alison [Scheduling and priority] |
||
---|---|---|---|
Line 30: | Line 30: | ||
the pthread attributes and not to use the inherit scheduling of the | the pthread attributes and not to use the inherit scheduling of the | ||
thread which created the real-time thread. | thread which created the real-time thread. | ||
+ | </wrap> | ||
+ | - **Problems with pthread condition variables** \\ <wrap> | ||
+ | Multithreaded applications which rely on glibc's libpthread are prone to unexpected latency delays since its condition variable implementation does not honor priority inheritance ([[https://sourceware.org/bugzilla/show_bug.cgi?id=11588|bugzilla]]). Unfortunately glibc's DNS resolver and asynchronous I/O implementations depend in turn on these condition variables. | ||
+ | [[https://github.com/dvhart/librtpi|librtpi]] is an alternative LGPL-licensed pthread implementation which supports priority inheritance, and whose API is as close to glibc's as possible. The alternative [[https://www.musl-libc.org/|MUSL libc]] has a pthread condition variable implementation similar to glibc's. | ||
</wrap> | </wrap> | ||
Line 42: | Line 46: | ||
==== Capabilities: running the app with RT priority as a non-root user ==== | ==== Capabilities: running the app with RT priority as a non-root user ==== | ||
- | Several of the Pthread APIs, like mlockall(), pthread_attr_setschedpolicy(), by default and convention require root in order to successfully get their work done. Thus, RT apps - which need to set an RT sched policy and priority - are often run via sudo. | + | Several of the Pthread APIs, like ''mlockall()'', ''pthread_attr_setschedpolicy()'', by default and convention require root in order to successfully get their work done. Thus, RT apps - which need to set an RT sched policy and priority - are often run via ''sudo''. |
- | There's a far better approach to this; sudo gives the process root capabilities. This interests hackers :-). | + | There's a far better approach to this; ''sudo'' gives the process root capabilities. This interests hackers :-). |
- | Instead, you should leverage the powerful POSIX Capabilities model! This way, the process (and threads) get _only_ the capabilities they require and nothing more. This follows the infosec best practice, the 'principle of least privilege'. | + | Instead, you should leverage the powerful POSIX **Capabilities** model! This way, the process (and threads) get _only_ the capabilities they require and nothing more. This follows the infosec best practice, the //principle of least privilege//. |
- | Apps start out with no capabilities by default; also note that capabilities are a per-thread resource (essentially translating to bitmasks with the task structure,which is per-thread of course). Among the various capability bits, the man page on capabilities(7) shows that **CAP_SYS_NICE** is the appropriate capability to use in this circumstance; a snippet from the man page reveals this: | + | Apps start out with no capabilities by default; also note that capabilities are a per-thread resource (essentially translating to bitmasks with the task structure,which is per-thread of course). Among the various capability bits, the man page on ''capabilities(7)'' shows that **''CAP_SYS_NICE''** is the appropriate capability to use in this circumstance; a snippet from the ''capabilities(7)'' man page reveals this: |
- | ... | + | |
+ | ... | ||
**CAP_SYS_NICE** | **CAP_SYS_NICE** | ||
* Lower the process nice value (nice(2), setpriority(2)) and change the nice value for arbitrary processes; | * Lower the process nice value (nice(2), setpriority(2)) and change the nice value for arbitrary processes; | ||
- | * set real-time scheduling policies for calling process, and set scheduling policies and priorities for arbitrary processes (sched_setscheduler(2), sched_setparam(2), sched_setattr(2)); | + | * **set real-time scheduling policies for calling process, and set scheduling policies and priorities for arbitrary processes (sched_setscheduler(2), sched_setparam(2), sched_setattr(2));** |
* set CPU affinity for arbitrary processes (sched_setaffinity(2)); | * set CPU affinity for arbitrary processes (sched_setaffinity(2)); | ||
* set I/O scheduling class and priority for arbitrary processes (ioprio_set(2)); | * set I/O scheduling class and priority for arbitrary processes (ioprio_set(2)); | ||
* apply migrate_pages(2) to arbitrary processes and allow processes to be migrated to arbitrary nodes; | * apply migrate_pages(2) to arbitrary processes and allow processes to be migrated to arbitrary nodes; | ||
* apply move_pages(2) to arbitrary processes; | * apply move_pages(2) to arbitrary processes; | ||
- | * use the MPOL_MF_MOVE_ALL flag with mbind(2) and move_pages(2). | + | * use the MPOL_MF_MOVE_ALL flag with mbind(2) and move_pages(2). ... |
- | ... | + | |
+ | |||
+ | **//Ok, great, but how exactly is this capability bit to be set on the app?// | ||
+ | ** | ||
+ | - One approach is to do so programatically, via the ''capget()/capset()'' system calls. (Note that's it's generally easier to use the libcap library wrappers, ''cap_[g|s]et_proc(3)'': [[https://man7.org/linux/man-pages/man3/cap_get_proc.3.html]]. This man page even provides a small example of doing so). | ||
+ | - Another easy way is to leverage systemd and run your app as a service; in the service unit, specify the capability (see the man page on systemd.exec(5); [[https://www.freedesktop.org/software/systemd/man/systemd.exec.html#Capabilities]]. | ||
+ | - Perhaps the easiest way: via the ''setcap(8)'' utility (it's man page: [[https://man7.org/linux/man-pages/man8/setcap.8.html]]). The setcap/getcap are typically part of the libcap package. For example: | ||
+ | ''sudo setcap CAP_SYS_NICE+eip <your-app-binary-executable>'' | ||
+ | |||
+ | You could put this line in the app Makefile (or equivalent). | ||
+ | (The ''getcap(8)'' utility can be used to verify that the 'dumb-capability' binary now has the ''CAP_SYS_NICE'' bit set)! | ||
- | //Ok, great, but how exactly is this capability bit to be set on the app?// | + | And you're all set to run it as non-root now, a much more secure approach. |
- | - One approach is to do so programatically, via the capget()/capset() system calls. (Note that's it's generally easier to use the libpcap library wrappers, cap_[g|s]et_proc(3): [[https://man7.org/linux/man-pages/man3/cap_get_proc.3.html]]. This man page even provides a small example of doing so). | + | |
- | - Another is to use systemd and run your app as a service; in the service unit, specify the capability (see the man page on systemd.exec(5); [[https://www.freedesktop.org/software/systemd/man/systemd.exec.html#Capabilities]]. | + | |
- | - Perhaps the easiest way: via the setcap(8) utility (it's man page: [[https://man7.org/linux/man-pages/man8/setcap.8.html]]). | + | |
===== Example ===== | ===== Example ===== |