RT preemption

edit

The Linux Foundation's Real-Time Linux (RTL) collaborative project is focused on improving the real-time capabilities of Linux and advancing the adoption of real-time Linux in various industries, including aerospace, automotive, robotics, and telecommunications.

Parameter CONFIG_PREEMPT_RT id enables real-time preemption.

RT scheduling policies

edit

Scheduling policies for RT:

SCHED_FIFO id, SCHED_RR id
implemented in kernel/sched/rt.c src
SCHED_DEADLINE
implemented in kernel/sched/deadline.c src


API:

man 1 chrt – manipulate the real-time attributes of a process
man 2 sched_rr_get_interval – get the SCHED_RR interval for the named process
man 2 sched_setscheduler, sched_getscheduler – set and get scheduling policy/parameters
man 2 sched_get_priority_min, sched_get_priority_max – get static priority range

Testing RT capabilities

edit

The testing process for Real-Time Linux typically involves several key aspects. First and foremost, it is crucial to verify the accuracy and stability of the system's timekeeping mechanisms. Precise time management is fundamental to real-time applications, and any inaccuracies can lead to timing errors and compromise the system's real-time capabilities.

Another essential aspect of testing is evaluating the system's scheduling algorithms. Real-Time Linux employs advanced scheduling policies to prioritize critical tasks and ensure their timely execution. Testing the scheduler involves assessing its ability to allocate resources efficiently, handle task prioritization correctly, and prevent resource contention or priority inversion scenarios.

Furthermore, latency measurement is a critical part of Real-Time Linux testing. Latency refers to the time delay between the occurrence of an event and the system's response to it. In real-time applications, minimizing latency is crucial to achieving timely and predictable behavior. Testing latency involves measuring the time it takes for the system to respond to various stimuli and identifying any sources of delay or unpredictability.

Additionally, stress testing plays a significant role in assessing the system's robustness under heavy workloads. It involves subjecting the Real-Time Linux system to high levels of concurrent activities, intense computational loads, and input/output operations to evaluate its performance, responsiveness, and stability. Stress testing helps identify potential bottlenecks, resource limitations, or issues that might degrade the real-time behavior of the system.


RTLA – The realtime Linux analysis tool:
rtla timerlat doc – CLI for the kernel's timerlat tracer doc
rtla osnoise doc – CLI for the kernel's osnoise tracer doc. Kernel function run_osnoise id measures time with function trace_clock_local id in loop.
rtla hwnoise doc – CLI for the osnoise tracer doc with interrupts disabled
Implementation: tools/tracing/rtla src and kernel/trace/trace_osnoise.c src
Linux scheduling latency debug and analysis
RT-Tests, source
cyclictest
some RT-Tests man pages:
cyclictest – measures man 2 clock_nanosleep or man 2 nanosleep delay
hackbench – scheduler benchmark/stress test
hwlatdetect – CLI for /sys/kernel/tracing/hwlat_detector doc / kernel/trace/trace_hwlat.c src. Kernel function kthread_fn id measures time delays with function trace_clock_local id in loop.
oslat – measures delay with RDTSC in busy loop
RT Tracing Tools with eBPF
realtime ltp


Further reading about real-time Linux:

linux/spinlock_rt.h inc used via linux/spinlock_types.h inc
linux/rtmutex.h inc used via linux/mutex_types.h inc
linux/rwbase_rt.h inc used via linux/rwlock_types.h inc
Introduction to Real-Time Linux: Unleashing Deterministic Computing
Power Management and Scheduling in the Linux Kernel (OSPM)
the Real-Time Linux wiki
CPU partitioning and isolation
Realtime@LWN
linux-stable-rt.git
linux-rt-devel.git
Realtime kernel patchset, Arch Linux
https://www.kernel.org/pub/linux/kernel/projects/rt/ - RT patches for upstream kernel
High Precision Event Timer (HPET)
Demystifying the Real-Time Linux Scheduling Latency
Real-time kernel tuning in RHEL 9
Linux subsystems related to real-time
Linux kernel scheduling and preemption
Interrupts
Deferred works
Non-maskable interrupt handler (NMI)
System management interrupt (SMI)
man 7 sched
latency @ LKML
PREEMPT_RT @ LKML
QA about PREEMP_RT, LPC'23, State of the onion, pdf

💾 Historical

The PREEMPT_RT patch has been partially merged into the mainline Linux kernel, starting from version 5.15. Lazy preemption (CONFIG_PREEMPT_LAZY) remains in the external patchset.