====== cpusets ======

Cpusets is a Linux kernel facility and a subset of cgroups-v1, that
provides a [[realtime:documentation:howto:tools:cpu-partitioning:start|mechanism to assign a set of CPUs]] and NUMA memory nodes (if
NUMA is available) to a set of tasks. This means in effect, that the
tasks are constrained to the assigned cpusets. For a detailed documentation of cpusets,
please refer to the [[https://docs.kernel.org/admin-guide/cgroup-v1/cpusets.html|CPUSETS kernel doc]].

===== Enable cpusets =====

The following kernel configuration Parameter are required to enable cpusets:

  * CONFIG_CGROUPS
  * CONFIG_CPUSETS


===== Sysfs interface =====

To handle the cpusets in userspace, the functionality is available in sysfs:
"/sys/fs/cgroup/cpuset/". Each cpuset is represented by a directory in the cgroup file system containing (on top of the standard cgroup files) the following files describing that cpuset: 

  * cpus: list of CPUs in that cpuset
  * mems: list of Memory Nodes in that cpuset
  * memory_migrate flag: if set, move pages to cpusets nodes
  * cpu_exclusive flag: is cpu placement exclusive?
  * mem_exclusive flag: is memory placement exclusive?
  * mem_hardwall flag:  is memory allocation hardwalled
  * memory_pressure: measure of how much paging pressure in cpuset
  * memory_spread_page flag: if set, spread page cache evenly on allowed nodes
  * memory_spread_slab flag: if set, spread slab cache evenly on allowed nodes
  * sched_load_balance flag: if set, load balance within CPUs on that cpuset
  * sched_relax_domain_level: the searching range when migrating tasks


In addition, the root cpuset only has the following file:

  * memory_pressure_enabled flag: compute memory_pressure?


===== Realtime Example =====

The following bash commands show exemplary how cpusets can be used for realtime processes on a 4 CPU machine. To simplify the use of cpusets the Python tool [[realtime:documentation:howto:tools:cpu-partitioning:cset|cset]] can be used.

<code bash>
 $ mkdir /dev/cpuset/rt0
 $ echo 0 > /dev/cpuset/rt0/cpus
 $ echo 0 > /dev/cpuset/rt0/mems
 $ echo 1 > /dev/cpuset/rt0/cpu_exclusive
 $ echo $RT_PROC_PID > /dev/cpuset/rt0/tasks
 $ mkdir /dev/cpuset/system
 $ echo 1-3 > /dev/cpuset/system/cpus
 $ echo 0 > /dev/cpuset/system/mems
 $ echo 1 > /dev/cpuset/system/cpu_exclusive
 $ for pid in $(cat /dev/cpuset/tasks); do /bin/echo $pid > /dev/cpuset/system/tasks; done 
</code>

===== References ======
  * [[https://docs.kernel.org/admin-guide/cgroup-v1/cpusets.html|CPUSETS kernel doc]]
  * [[https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/tree/include/linux/cpuset.h|include/linux/cpuset.h]]
  * [[https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/tree/kernel/cgroup/cpuset.c|kernel/cgroup/cpuset.c]]