Embedded Linux
Embedded Linux systems can range from consumer electronics like smartphones and smart TVs to industrial machinery, medical devices, automotive systems, communication equipment and more. Linux brings the power, flexibility, and stability to these resource-constrained devices.
Core components of an Linux on embedded systems are boot loader, custom Linux kernel and root file system. Source code is compiled by GCC cross compiler. Common boot loader is Das U-Boot – the Universal Boot Loader. Root FS usually is based on BusyBox – software suite that provides many Unix utilities in a single executable file. Local debugging often is limited, so gdbserver can be used.
Basic embedded Linux system can be compiled manually from modules above. There are tools for complete system generation from old and simple Buildroot to huge Yocto Project, which creates customized embedded Linux distributions. PC distributions Debian, Ubuntu ant others have build for ARM processors.
Raspberry Pi, BeagleBoard and Nvidia Jetson are examples of popular single-board computers supporting embedded Linux.
Unlike regular desktop or server distributions of Linux these versions of Linux usually have much fewer libraries and application, firmware is smaller and typically boot from flash memory or SD card.
Projects
editBuild frameworks and environments
edit- Wear OS – Android for smartwatches and other wearables.
- AOSP – Android Open Source Project
- Buildroot – old simple patch and make based build environment
- Yocto Project and OpenEmbedded – a BitBake based build framework and environment
- BitBake – make-like build tool
- OpenWrt – open wireless router
- Embeddable Linux Kernel Subset – for ancient 16-bit x86 machines and emulators
Components
edit- The Linux Kernel
- "init" implementations: OpenRC, runit
- GUIs: LXQt, LXDE, Xfce, FLTK
- coreboot – lightweight BIOS
C standard library variants
edit- General purpose GNU C Library glibc, 7.9 MB
- uClibc, 560 KB
- musl, 527 KB
- dietlibc, 185 KB
- Newlib
- klibc – primarily for booting Linux systems
- Bionic – originally developed by Google for the Android embedded system operating system
Further reading:
Distributions
editDistributions for ARM processors:
- Arch Linux ARM – port of Arch Linux for ARM processors
- Armbian – Debian and Ubuntu based
- RedSleeve – port of Red Hat Enterprise Linux
- emteria.OS – proprietary Android based
Lightweight cloud oriented systems:
- Alpine Linux – uses musl, BusyBox and OpenRC.
- Fedora CoreOS
- RHCOS – Red Hat Enterprise Linux CoreOS for OpenShift Container Platform
- https://MicroShift.io/ – lightweight Kubernetes distribution designed for edge and IoT computing environments
Lightweight multi-platform distributions:
Common multi-platform distributions:
Features of Linux kernel for embedded systems
edit- I/O and buses:
- drivers/usb/gadget src – peripheral USB device implementation
- drivers/iio src – Industrial I/O
- drivers/pwm src – Pulse-width modulation
- drivers/regulator src – Generic Voltage and Current Regulator support
- drivers/fpga src
- drivers/slimbus src
- SLOB – Simple List Of Blocks memory allocator
- Devicetree – describes hardware components for the kernel
Multimedia
edit- Audio
- ASoC – ALSA System on Chip
- snd_soc_card id
- ASoC – ALSA SoC Layer doc
- https://www.alsa-project.org/wiki/ASoC
- TFT (LCD) – Thin-film-transistor liquid-crystal display
File systems and storage
edit- UBIFS – Unsorted Block Image FS
- SquashFS – a compressed read-only FS with low overhead
- drivers/mtd src – Memory Technology Device block device
- More: YAFFS, JFFS2
MMU less
editLinux was originally designed on a processor with a memory management unit (MMU). Most embedded systems do not have a MMU, as we discussed earlier (Embedded Systems/Memory) and are called microcontrollers. Microcontrollers without a MMU are cheaper consumes less power.
Benefits of using a processor with a MMU:
- can isolate running "untrusted" machine code from running "critical" code, so the "untrusted" code is guaranteed (in the absence of hardware failures) not to interfere with the "critical" code
- makes it easier for the OS to present the illusion of virtual memory
- can run "normal" Linux (could also run "μClinux", but what's the point?)
μClinux – "MicroController Linux" is a version of the Linux kernel that supports Altera NIOS, ARM, Freescale M68K (including DragonBall, ColdFire, PowerQUICC and others), Hitachi H8, MIPS, and Xilinx MicroBlaze processors.
Disabling parameter CONFIG_MMU enables nommu mode.
Real-time
editPeople use a variety of methods to combine real-time tasks with Linux:
- Run the real-time tasks on a dedicated microcontroller; communicate with a (non-real-time) PC that handles non-real-time tasks. This is pretty much your only choice if you need real-time response times below 1 microsecond.
- Run the real-time tasks in a "underlying" dedicated real-time operating system; run Linux as a "nested operating system" inside one low-priority task on top of the real-time operating system. Some of these systems claim real-time response times below 500 microseconds.
- use a Linux kernel designed to emphasize real-time tasks, and run the real-time tasks with a high priority (perhaps even as a kernel thread).
Examples of RT distributions:
Several RT features provide real time capabilities:
- RT preemption
- RT scheduling
- CPU partitioning and isolation
- Dynticks (Tickless, nohz)
RT preemption
editThe Linux Foundation's Real-Time Linux (RTL) collaborative project is focused on improving the real-time capabilities of Linux and advancing the adoption of real-time Linux in various industries, including aerospace, automotive, robotics, and telecommunications.
Parameter CONFIG_PREEMPT_RT id enables real-time preemption.
RT scheduling policies
editScheduling policies for RT:
- SCHED_FIFO id, SCHED_RR id
- implemented in kernel/sched/rt.c src
- SCHED_DEADLINE
- implemented in kernel/sched/deadline.c src
API:
- man 1 chrt – manipulate the real-time attributes of a process
- man 2 sched_rr_get_interval – get the SCHED_RR interval for the named process
- man 2 sched_setscheduler, sched_getscheduler – set and get scheduling policy/parameters
- man 2 sched_get_priority_min, sched_get_priority_max – get static priority range
Testing RT capabilities
editThe testing process for Real-Time Linux typically involves several key aspects. First and foremost, it is crucial to verify the accuracy and stability of the system's timekeeping mechanisms. Precise time management is fundamental to real-time applications, and any inaccuracies can lead to timing errors and compromise the system's real-time capabilities.
Another essential aspect of testing is evaluating the system's scheduling algorithms. Real-Time Linux employs advanced scheduling policies to prioritize critical tasks and ensure their timely execution. Testing the scheduler involves assessing its ability to allocate resources efficiently, handle task prioritization correctly, and prevent resource contention or priority inversion scenarios.
Furthermore, latency measurement is a critical part of Real-Time Linux testing. Latency refers to the time delay between the occurrence of an event and the system's response to it. In real-time applications, minimizing latency is crucial to achieving timely and predictable behavior. Testing latency involves measuring the time it takes for the system to respond to various stimuli and identifying any sources of delay or unpredictability.
Additionally, stress testing plays a significant role in assessing the system's robustness under heavy workloads. It involves subjecting the Real-Time Linux system to high levels of concurrent activities, intense computational loads, and input/output operations to evaluate its performance, responsiveness, and stability. Stress testing helps identify potential bottlenecks, resource limitations, or issues that might degrade the real-time behavior of the system.
- RTLA – The realtime Linux analysis tool:
- rtla timerlat doc – CLI for the kernel's timerlat tracer doc
- rtla osnoise doc – CLI for the kernel's osnoise tracer doc. Kernel function run_osnoise id measures time with function trace_clock_local id in loop.
- rtla hwnoise doc – CLI for the osnoise tracer doc with interrupts disabled
- Implementation: tools/tracing/rtla src and kernel/trace/trace_osnoise.c src
- Linux scheduling latency debug and analysis
- RT-Tests, source
- some RT-Tests man pages:
- cyclictest – measures man 2 clock_nanosleep or man 2 nanosleep delay
- hackbench – scheduler benchmark/stress test
- hwlatdetect – CLI for /sys/kernel/tracing/hwlat_detector doc / kernel/trace/trace_hwlat.c src. Kernel function kthread_fn id measures time delays with function trace_clock_local id in loop.
- oslat – measures delay with RDTSC in busy loop
- RT Tracing Tools with eBPF
- realtime ltp
...
editFurther reading about real-time Linux:
- linux/spinlock_rt.h inc used via linux/spinlock_types.h inc
- linux/rtmutex.h inc used via linux/mutex_types.h inc
- linux/rwbase_rt.h inc used via linux/rwlock_types.h inc
- Introduction to Real-Time Linux: Unleashing Deterministic Computing
- Power Management and Scheduling in the Linux Kernel (OSPM)
- the Real-Time Linux wiki
- Realtime@LWN
- linux-stable-rt.git
- linux-rt-devel.git
- Realtime kernel patchset, Arch Linux
- https://www.kernel.org/pub/linux/kernel/projects/rt/ - RT patches for upstream kernel
- High Precision Event Timer (HPET)
- Demystifying the Real-Time Linux Scheduling Latency
- Real-time kernel tuning in RHEL 9
- Linux subsystems related to real-time
- latency @ LKML
- PREEMPT_RT @ LKML
- QA about PREEMP_RT, LPC'23, State of the onion, pdf
💾 Historical
The PREEMPT_RT patch has been partially merged into the mainline Linux kernel, starting from version 5.15. Lazy preemption (CONFIG_PREEMPT_LAZY) remains in the external patchset.
...
editFurther readings about Embedded Linux: